Tuesday, June 1, 2010

PSAKE Template Expansion

Several months ago I was in the market for a build solution. The two that caught my attention were James Kovac’s PSAKE and Derick Bailey’s Albacore. Derick and crew have put a lot of work into the Albacore project and it shows. The feature set is remarkable and Albacore is a cinch to use. However, I need to keep my project dependencies to a minimum and for me Ruby is too much. So I went with PSAKE.
One thing I did reluctantly give up by using PSAKE was the template expansion features Derick put into Albacore. After a fairly thorough search I found that no one had a decent solution or they were keeping their find to themselves. The solutions that I did find required a bit of XML which I was aggressively staying away from. With shortage of time I had to make my own.
For my solution I chose to follow Derick’s choice of YAML files for the template data files and how these keys are represented in the templates. To me this is a more understandable format and I had most of my templates setup to use Albacore already. In case you are not familiar with how Albacore’s expandtemplates work:
Your template data file details key value pairs in YAML format.

value_1: this is some value 
value_2: this is another value

Your template file would represent these like :





If the process works correctly, you end up with:

 



What you are here for


The first step was getting the key value pairs in a usable format. To do this you have to read the data file and create a hashtable from each line. This is how you accomplish this in Powershell:

$data_values = @{}    
foreach($line in $template_data) {
$d= $line.IndexOf(":")
$length = $line.Length
$data_values.add($line.Substring(0,$d).Trim() ,
$line.Substring($d+1, ($length-($d+1))))
}

Once the data file has been hashed, you need to read the contents of the file and perform the replacements:
[string] $template_text = [System.IO.File]::ReadAllText($template_file)     
if ($debug -eq $true) {
echo $template_text
echo ""
echo "--------------------------------------------"
echo ""
echo "Replacing values"
}

foreach ($item in $data_values.Keys) {
if ($template_text -eq $null) {
throw "There is a problem with text template"
}

if ($debug -eq $true) {
echo " -- #{$item} with " $data_values.Get_Item($item)
}

$template_text = $template_text.Replace("#{$item}",
$data_values.Get_Item($item).Trim())
}

Finally you create the destination file:

$template_text | sc -path $destination_file


Conclusion

This is the first useful function I have created for my PSAKE builds and have placed it on github here.

Wednesday, April 7, 2010

Remove Unwanted Text from Repository Files

You may find that you need to delete content from a file that was committed to a Subversion repository.  If so, your options are limited.  Especially since Subversion does not support this kind of “malicious” activity.  The only option you have is to edit the dump file with a text editor like Vim and load the clean dump file.

First you need to dump your Subversion repository to a file:
>svnadmin dump live_repos > repos.dmp

Next you will open the file with Vim and replace the text with a mask of equal length.  It is important to keep the length the same. Otherwise you face more complications.

After you save your changes you can load the dump file into a temporary repository:
>svnadmin create temp_repos
>svnadmin load temp_repos < repos.dmp

Unfortunately this is not all there is to do. Eventually your load will stop with an error like :
 errormsg

Each file has a checksum in the dump file. The message above states that the load found a checksum for Web.Config other than the expected one. This is perfect because it tells us what to look for and what to replace it with.  In your dump file you will search for the hash that is shown after the expected: with the hash that follows the actual: Depending on how many times the text you want replaced occurs, you might have to do this more than once.

After you have a complete load you need to verify that your temp_repos ends with the same revision as your live_repos. If it does then you need to load the clean dump file into your live_repos.

Tuesday, February 9, 2010

War Games

A few months ago a server crashed and took down two blogs that I read.  Within a short time, years of hard work had come down to one thing, a simple backup that did not exist for either of these blogs.  This event was so catastrophic that the International Backup Awareness Day was declared in remembrance.  Surely the majority of developers do not worry about such things.  I do.  This event overwhelmed with “what if’s” that played through my head for days.  Unlike Phil and Jeff, I would not be lucky enough to pull my data from users’ cache or old virtual hard drives.  Even the SEO Toolkit would be of little help. My room for error is thin and failure would definitely lead to my demise.

I needed a good evaluation of my existing backup plan.  The US military conducts War Games to test scenarios, theories, and battle plans.  Taking from this and TDD, I decided to conduct my own War Games to test my disaster recovery/prevention strategy. 

Scenario 1: Server Failure – FAILED

REQUIREMENT: If the server should fail, I should be able to restore the database on another server, loosing less than 1 hour of data.

To test this, I “pretended” my server was shutdown.  Many will ask, “How did you fail this?”  To be fair, I did have a backup from offsite storage.  However, it was more than my required limit of 1 hour or less.  This or any other lame ass excuse is not good enough.  I would be in a tight spot trying to explain why the past four hours of transactions were lost.

I run regular backups of my databases.  Where I went wrong was not having a recent backup somewhere other than the database server.  Since our offsite storage solution allows me to schedule a pickup, it would be easy to use it for this.  I think there are better solutions that would cost less money and be more accessible to those that might need these backups.

Currently I am using Microsoft's Sync Toy to sync a shared folder with the database server’s backup folder.  The backups are small and take very little bandwidth on a 10gb network.  This is not my ideal method, but it works with the time constraints I have now.

Another option is to setup Microsoft SQL Server to save the backups to a shared folder.  This requires MSSQL to run under an account that has READ/WRITE privileges to the shared folder.  This seems to be a simple solution.

Scenario 2: Server Failure - FAILED

REQUIREMENT: If the server should fail, someone other than me would be able to restore the database on another server, loosing less than 1 hour of data.

There is a chance that you might not be available when something happens.  It is important that your disaster recovery/prevention plan be readily accessible to key players in your organization.  Besides being accessible, your plan needs to provide useful information and instructions.

    At a minimum your disaster recovery/prevention plan should include:
  1. List of databases and the applications they support
  2. Ordered list of who is responsible for getting the databases up and running with at least two contact numbers for each
  3. Details of where the backups are located
  4. Detailed instructions on how to restore the databases from backups

I failed this scenario because I did not have a document that my organization’s key players could read.   I have discussed changes in backup plans with my boss as well as the IT department.  However, these conversations will likely be forgotten when the shit hits the fan.

To say the least, I did not fare so well on these two scenarios.  Yes there are many other scenarios to play against.  But these were the two that I assume would be the most beneficial.  In the spirit of actual military War Games, my organization’s key players will take part in future games, acting in response to possible real world events extending beyond just database recovery.  At least that’s my plan.  It will definitely require active participation from key players and IT.  Surely they will see the benefits from testing what you set to do.

Wednesday, August 12, 2009

Open Command Prompt Here

Navigating through folders from the command prompt is a pain.  I would rather browse the directories using point and click via Windows Explorer then open the command prompt from there.  The Windows 7 dev team made plenty of improvements, but somehow overlooked this.  All you have to do is:

  1. Open RegEdit
  2. Navigate to [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell].
    1. Alternatively you can also add the key to [HKEY_CLASSES_ROOT\Directory\shell].
  3. Add a key and name it cmd.
  4. Set the default value to something descriptive like “Open Command Prompt Here”.
  5. Add a key under the cmd key you just created and name it command.
  6. Set the value of the default value to cmd.exe /k cd %1

This has been as valuable as being able to open a folder as a VS web site from windows explorer.

Tuesday, July 21, 2009

Comparison of Date Ranges

A tricky problem that arises is figuring out if Item-B is available between 11:00 and 12:00 on a given day. If Item-B is not checked out at all you don’t have a problem. But what if it was checked out from 8:00 to 11:15 or 9:00 to 16:00?

Table1
Item StartDate EndDate
Book 7/26/2009 8:00 7/26/2009 16:00
Pencil 7/26/2009 6:00 7/26/2009 20:00
Paper 7/26/2009 10:00 7/26/2009 14:00
Eraser 7/26/2009 6:00 7/26/2009 13:00
Ruler 7/26/2009 12:00 7/26/2009 18:00

Lasse Karlsen provided an excellent solution and explanation in this StackOverflow question.

If I wanted to see what was checked out between 11:00 and 12:00 from Table1 I would use:


SELECT * FROM Table1
WHERE Table1.StartDate <= @enddate
AND Table1.EndDate >= @startdate

Thursday, March 12, 2009

Open Folder as VS Web Site

Thank you Brad for this.
From VS go Tools --> Macros IDE
Right click My Macros
Add new module named WebSite
Add reference to VsWebSite.Interop.dll
Copy the following into the the WebSite module


Sub OpenWebsite(Optional ByVal path As String = "")
If (String.Compare(path, String.Empty) = 0) Then
MsgBox("Must supply a folder path to the OpenWebsite macro",
MsgBoxStyle.OkOnly)
Else
Dim webPkg As VsWebSite.VSWebPackage
webPkg = DTE.GetObject("WebPackage")
webPkg.OpenWebSite(path,
VsWebSite.OpenWebsiteOptions.OpenWebsiteOption_None,
False)
End If
End Sub

Create a new file named OpenWebSite.reg

Copy the following into this file:



Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\OpenVSWeb] @="Open as Visual Studio Website"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\OpenVSWeb\command] @="devenv.exe /command \\\"Macros.MyMacros.Website.OpenWebsite %1\\\""

Double click on OpenWebSite.reg

Thursday, March 5, 2009

WSS 3.0 Installation

I recently installed WSS 3.0 on my laptop that is running Server 2008. By default the installation installed the databases in Windows Internal Database which is not what I wanted. I want the tables running SQL Server 2008 instance and I want them named WSS_Config and WSS_AdminContent. Here's how to do it.

  1. If you have not done so already, add C:\Program Files\Common Files\microsoft shared\web server extensions\12\bin to the path System Variable. This is a must if you plan to work with SharePoint.
  2. run psconfig.exe -cmd configdb -create -server SQLServerInstanceName -database WSS_Config -admincontentdatabase WSS_AdminContent
  3. Administrative Tools ---> SharePoint Products and Technologies Configuration Wizard
After this completes goto Administrative Tools ----> SharePoint 3.0 Central Administration to create a new site or upgrade an existing web app.