Automating Code Changes with StudioShell

§ October 3, 2011 04:27 by beefarino |

imageIn the last few talks I’ve given on PowerShell for Developers, I’ve focused on build, test, and code automation.  When discussing StudioShell, I’ve pulled a very effective demo from a real-life project…

After wrapping up an iteration with a client, they returned to me with a new 300,000-line code base spread over about 200 (very messy) files.  They were in the process of implementing a set of “corporate” coding standards and wanted some help getting their existing codebase to these standards.

One standard in particular was that and property that meets these criteria:

  1. is public
  2. returns a boolean

should start with “Is”.  So in this example code:

public class Demo 
{
   public bool Enabled { get;set; }
   public bool Valid { get;set; } 
   public bool IsRunning { get;set; } 
   public string Name { get;set; } 
   private bool Applied { get;set; } 
}

The properties Enabled and Valid need to be renamed to IsEnabled and IsValid (lines 3 & 4).  The IsRunning property already meets the standard (line5), the Name property returns a non-boolean type so the standard doesn’t apply (line 6), and the Applied property is private and excluded from the standard.  In the end this file needs to look as follows:

public class Demo 
{ 
   public bool IsEnabled { get;set; }
   public bool IsValid { get;set; } 
   public bool IsRunning { get;set; }
   public string Name { get;set; }
   private bool Applied { get;set; }
}

While it’s entirely possible to apply this standard by hand, the thought of scouring 300,000+ lines of code for properties that meet this criteria is not an attractive task.  FxCop can probably find these properties for us, but it can’t apply a fix – that still requires human intervention.  Best case scenario, I’m looking at days of mind-numbing monkey-on-a-keyboard work.

This is the kind of problem that screams for automation.  This is where StudioShell shines.  Here’s how I slashed through this Gordian knot…

First, I isolated a single code file in my StudioShell console, so I could work out the pipeline logic I needed without obliterating the codebase:

> cd DTE:\solution\codemodel\demoapp\program.cs

Second, I use PowerShell’s filtering capabilities to isolate all public properties in the file:

# spacing added for readability
> ls -recurse | where { 
     $_-match 'property' -and
     $_.access -match 'public' }

Location: dte:\solution\codemodel\demoapp\program... 
File: program.cs 
Code Container: Class Demo 
Available Operations: d+ < 
        Kind                         Name 
-----  ----                          --------- 
d+ < vsCMElementProperty Enabled 
d+ < vsCMElementProperty Valid 
d+ < vsCMElementProperty IsRunning 
d+ < vsCMElementProperty Name
Once I have all the properties isolated, I need to filter the properties to those that returning booleans.  This required a little investigation using get-member, but once I found the Type property of the CodeProperty class, I was home free:
 
#spacing added for readability
> ls -recurse | where { $_-match 'property' -and
    $_.access -match 'public' -and
    $_.type.asFullName -match 'bool' } 

Location: dte:\solution\codemodel\demoapp\program... 
File: program.cs 
Code Container: Class Demo 
Available Operations: d+ < 

        Kind                         Name 
        --------------              ---- 
d+ < vsCMElementProperty Enabled 
d+ < vsCMElementProperty Valid 
d+ < vsCMElementProperty IsRunning

The last factor to consider is the property name – I only need names that do not start with “Is”.  Easily done:

#spacing added for readability
> ls -recurse | where { $_-match 'property' -and
   $_.access -match 'public' -and
   $_.type.asFullName -match 'bool' -and
   $_.name -notmatch '^Is' } 

Location: dte:\solution\codemodel\demoapp\program... 
File: program.cs 
Code Container: Class Demo 
Available Operations: d+ < 
        Kind                         Name 
        -------------               ----- 
d+ < vsCMElementProperty Enabled 
d+ < vsCMElementProperty Valid
At this point, my pipeline is filtering out the correct properties – only public properties returning a boolean that do not start with “Is” are being included in the pipeline output.  The next step is to take corrective action against these offending property names.  The RenameSymbol method will execute the appropriate rename activity for me, ensuring that the new name is propagated across the entire codebase that is loaded in Visual Studio:
 
#spacing added for readability
> ls -recurse | where { $_-match 'property' -and
   $_.access -match 'public' -and
   $_.type.asFullName -match 'bool' } |
foreach { $_.RenameSymbol( "Is"+$_.Name ) } 

> ls 
Location: dte:\solution\codemodel\demoapp\program... 
File: program.cs 
Code Container: Class Demo 
Available Operations: d+ < 
        Kind                         Name 
        --------------              ---- 
d+ < vsCMElementProperty IsEnabled 
d+ < vsCMElementProperty IsValid 
d+ < vsCMElementProperty IsRunning 
d+ < vsCMElementProperty Name 
d+ < vsCMElementProperty Applied
Once I verified that this worked as expected, applying the changes across the entire code base was a trivial matter of changing the root of my recursive search to the root of the code base:
 
#spacing added for readability
> ls dte:/solution/codemodel -recurse | where { 
   $_-match 'property' -and
   $_.access -match 'public' -and
   $_.type.asFullName -match 'bool' 
} | foreach { $_.RenameSymbol( "Is"+$_.Name ) }

Done.  The total time it took me to complete the task across the 200+ files, including the necessary research, prototyping, and the full script execution: less than 30 minutes.  My input focus never moved from the StudioShell console, and my hands never left the keyboard (except when the script was running across the code base, during which time I helped my kids study their math facts).

This is why I made StudioShell – so I can scale iterative code and IDE solutions the same way an IT pro scales administrative solutions.  If a solution can be applied to a single line of your code, StudioShell can be apply it to all lines of your code.



Summer (and Fall) of Speaking

§ September 12, 2011 03:10 by beefarino |

SOOOOoooooOOOO… I’ve been a bit incognito lately and I thought I’d take a moment to summarize what’s been happening in the last three months,and give everyone a head’s up for the rest of the year.

May was consumed largely by two things.  The first was volunteer work at my kids’ school during EOG testing.  I was basically there for 5 hours every day for three weeks, so the teachers could administer 4 exams for each of their 24 students one-on-one as mandated by the school board.  I couldn’t stop the testing, but I did what I could to ease the pain on the school, teachers, and kids.  The second thing was prepping talks for the summer conferences – CodeStock, MADExpo, and DEVlink – and various code camps in the Carolinas and Georgia.  I headed out to CodeStock at the end of May, where I spoke about StudioShell; it was another fantastic conference experience, the highlight for me was getting to kick Charles Petzold out of the room so I could give my talk immediately after his.  I mean, freaking Charles Petzold.

June was full of CodeStock and MADExpo.  MADExpo went off rather well for a first-time conference.  The maker-esque focus is something I hope they elaborate upon next year – the sessions on netduino, robotics, and the like were a real hit across the board, and the kid’s area was a blast with snap circuits, legos, etc.  I’m considering submitting an origami session next summer, just to do something completely different…

July was comprised of work, work, work … and a vacation in the east Texas desert, during which I managed to line up a substantial contract that consumed most of August … except of course for DEVlink.  This was my first DEVlink experience and it was a good one (despite the lack of wireless and cell coverage and no, I didn’t stay at the Choo-Choo either).  I definitely made more connections – social and professional – at DEVlink than at any other conference thus far.  I did another StudioShell session there.  Jaws hit the floor, it was well-received.  I also attended PowerShell sessions by other speakers –  Sarah Dutkiewicz and Joe Webb – I love seeing how different people approach teaching this technology to others.

With the summer over and school back in session, my speaking focus is shifting back to user groups and events.  This Thursday, September 15, I’ll be speaking at the WNC .NET User Group about PowerShell for Developers.  I’ve expanded this talk to include PSake, Pester, and StudioShell; I look forward to the feedback on the new topics.

Then on Saturday, September 17th, it’s SQL Saturday #89 in Atlanta.  This will be my first SQL Saturday ever, and I’m so glad to be bringing the PowerShell and development love to the database community.  I have two sessions – the first is using StudioShell to automate Denali, and the second is a brand-spankin’ new talk titled “Stupid PowerShell Tricks”.  I got the idea from going over some of my other talks and realizing that the most frequent question I get asked is “What was that you just did there?”  I also find myself asking this question quite a bit of others – especially when I’m around admins or DBAs.  Since there is obviously much we can learn from each other, I decided to make a session out of it.  I’ll show you my stupid tricks and you show me yours, and we’ll all walk away with new ways to get stuff done!

Later this month – September 22nd to be exact - I’m heading to the Triad Developer’s Group to spread more PowerShell love to Carolina software developers.  In October it looks like I’ll be a guest on Talk TechNet (details to follow).  In November I’m heading up to the Raleigh .NET User Group to demo and discuss StudioShell.

Whew… at some point I’ll need to fit in some project work… I’ve got a couple of decent irons in the fire, some will go open-source and some won’t … more on that in another post…



It’s no joke…

§ April 1, 2011 09:56 by beefarino |

imageI received confirmation from Microsoft today that I’m the newest PowerShell MVP!

I sincerely appreciate this recognition, and hope I meet the expectations of the community over the coming year.  I already have several PowerShell talks and projects on the near horizon…

First, I’ll be rescheduling my talk for the Charleston ALT.NET group that I had to cancel due to illness.  Look for an update on this in the next week or so.

In June I’ll be giving a StudioShell talk at CodeStock.  This is shaping up to be a demo-driven talk, so expect to leave with lots of practical knowledge and ideas.  With any luck I’ll be giving similar talks at DEVLink and MADExpo later this summer.  I’m also slated to talk to the Atlanta PowerShell UG in July, where I’d like to discuss using PowerShell to add agility and flexibility to your .NET applications.

In a few weeks I will be releasing MoSh, a MongoDB PowerShell provider that maps MongoDB servers, databases, and collections as PowerShell drives.  I’m having tons of fun with this one, and it’s proving to be a useful and intuitive way of caching and filtering data objects between PowerShell sessions.

As long as we’re talking about databases, I’m collaborating with Brady Gaster on a “Domain Console.”  The idea is seductive: a PowerShell module that mounts your NHibernate repositories as drives, allowing you to perform CRUD and native filtering operations using common PowerShell syntax.  We’re still early on this one, but the work Brady’s done thus far is making us both giggle with joy! 

And of course I’m continuing to work on StudioShell – I appreciate everyone’s continued involvement and issue submissions.

Again, thank you for the support, and I look forward to serving the PowerShell community for the next year!



StudioShell and the Debugger

§ March 15, 2011 05:12 by beefarino |

creepybugOne of the more tantalizing potentials in StudioShell is using it with the Visual Studio debugger.  This post will describe some of the features that are currently available. 

The debugger features are accessible at the path dte:/debugger.  A quick call to get-childitems shows how the functionality is broken down:

§ DTE:\debugger>ls Container: PSDTE::dte:\debugger Name ---------- ---- d+ < Breakpoints d < DebuggedProcesses d < LocalProcesses

There are two major subcategories of debugger features exposed through StudioShell: breakpoints and processes.

Breakpoints

StudioShell lets you add, remove, and modify breakpoints from script using the standard item cmdlets.  Here is a simple example that adds a breakpoint to line 25 of Program.cs (the ` line-continuation is used for readability):

new-item -path dte:/debugger/Breakpoints ` -file program.cs -line 25

There is also a condition parameter that causes a debug break for a specific condition, such as when the local variable “v” is less than 65:

new-item -path dte:/debugger/Breakpoints ` -file program.cs -line 25 ` -condition "v < 65"

That’s really the tip of the iceberg – breakpoints are very dynamic beasts.  In fact, I had no idea the extent to which you can customize breaking into the debugger until I started researching the Debugger node of the dte: drive.  To see all of your options, type:

get-help -path dte:/debugger/breakpoints new-item

You can remove breakpoints using the remove-item cmdlet; this example removes all breakpoints in the Program.cs file:

ls dte:/debugger/breakpoints ` | where{ $_.file -match 'program.cs' } ` | remove-item

You can also disable and enable breakpoints using the clear-item and invoke-item cmdlets, respectively:

$bkpts = ls dte:/debugger/breakpoints; # disable all breakpoints with a hitcount > 1 $bkpts | ?{ $_.currenthitcount -gt 1 } | clear-item; # enable all breakpoints in the Program.Main function $bkpts | ?{ $_.functionName -match 'Program.Main' } ` | invoke-item;

Debugged processes

When you start the debugger, the debuggedprocesses node of the dte: drive springs to life.  This node contains a list of all processes currently hooked by the debugger, so naturally its contents vary depending on what startup projects you have specified in your solution:

§ DTE:\debugger\DebuggedProcesses>ls Location: PSDTE::dte:\debugger\DebuggedProcesses Available Operations: d < ID Name ---------- -- ---- d < 4988 DemoSolution.vshost.exe

The DebuggedProcesses node doesn’t support many operations; it is simply a reflection of the current debugger state.  In fact, the only things you can do at this node is get-item (to fetch a lightweight process object) and set-location to enter a debugged process and start snooping around the threads:

§ DTE:\debugger\DebuggedProcesses>cd .\DemoSolution.vshost.exe § DTE:\debugger\DebuggedProcesses\DemoSolution.vshost.exe>ls Location: PSDTE::dte:\debugger\DebuggedProcesses\DemoSolution.vshost.exe Threads for Process: DemoSolution.vshost.exe Available Operations: d < Frozen Alive Thread ID Name ---------- ------ ----- --------- ---- d < * 3560 3560 d < * 1980 1980 d < * 464 vshost.RunParkingWindow d < * 5464 .NET SystemEvents d < * 5300 5300

Threads

Like processes, thread operations are limited to exploration: get-item will fetch a lightweight thread object for the specified thread, and set-location will drill into the stack for each thread:

§ ...\DemoSolution.vshost.exe>cd 5300 § ...\DemoSolution.vshost.exe\5300>ls Location: PSDTE::dte:\debugger\DebuggedProcesses\DemoSolution.vshost.exe\5300 Frames for Thread: Available Operations: d < Language Location Name ---------- -------- -------- ---- d < C# DemoSolution.Program.Main Current d < [Native to Managed Transition] Frame1 d < [Managed to Native Transition] Frame2 d < System.AppDomain.ExecuteAssembly Frame3 d < Microsoft.VisualStudio.HostingProcess... Frame4 d < System.Threading.ThreadHelper.ThreadS... Frame5 d < System.Threading.ExecutionContext.Run Frame6 d < System.Threading.ExecutionContext.Run Frame7 d < System.Threading.ThreadHelper.ThreadS... Frame8 d < [Native to Managed Transition] Frame9

Stack Frames

Stack frames are named by their current index, with the topmost frame always named “Current”, the next frame named “Frame1”, and so on.  Each frame is a container, so you can set-location again into a specific stack frame to probe further:

§ ...\DemoSolution.vshost.exe\5300>cd current § ...\DemoSolution.vshost.exe\5300\current>ls Container: PSDTE::dte:\debugger\DebuggedProcesses\DemoSolution.vshost.exe\5300\current Name ---------- ---- d < Arguments d < Locals

Which brings us to the meaty part…

Locals and Arguments

Once you navigate into a specific stack frames, you can access the local variables and arguments at that frame:

§ ...\5300\current>ls locals Location: PSDTE::dte:\debugger\DebuggedProcesses\DemoSolution.vshost.exe\5300\current\locals Available Operations: d < Value Type Name ---------- ----- ---- ---- < {string[0]} string[] args < 77 int v < "This is a v... string value

Once you’re this deep in the path hierarchy, you’re able to do some pretty neat things.  These locals values can be modified from the console, just like they can be modified from the locals or immediate window:

§ ...\5300\current>set-itemproperty -path v ` -name value -value 125 § ...\5300\current>ls Location: PSDTE::dte:\debugger\DebuggedProcesses\DemoSolution.vshost.exe\5300\current\locals Available Operations: d < Value Type Name ---------- ----- ---- ---- < {string[0]} string[] args < 125 int v < "This is a v... string value

Of course, being accessible in PowerShell means you gain so much more, such as the ability to search the stack for specific variables and values:

1 § ...\5300>$v = ls | ` #get all stack frames 2 join-path -child 'locals' | ` #create paths to local variables on each stack frame 3 ls | ` #list all locals... 4 where { $_.name -eq 'v' }#.. named "v" 5 § ...\5300>$v | select pspath,value 6 7 PSPath Value 8 ----- ----- 9 ...\Current\locals\v 123 10 ...\Frame1\locals\v 124 11 ...\Frame2\locals\v 125

This looks complex, but it simple when you break it down. 

  1. On line 1 we use get-childitems at the thread node to get a list of all stack frames. 
  2. Line 2 appends the node “locals” to each stack frame path, resulting in a list of StudioShell paths for the local variables at each stack frame. 
  3. Line 3 performs a get-childitems on each of these paths, resulting in a list of all locals across all stack frames. 
  4. Line 4 selects those local variables with a name of “v”. 
  5. Line 5 creates a simple report of the local variable path and its value. 

The result is the history of the values of variables named “v” at every stack location.  This isn’t something you can easily get from the IDE alone.

Looking Ahead

Hopefully this gives you a few ideas.  I’m still exploring this space in StudioShell, but I find myself giddy with ideas. 

One that’s got me drooling is the notion of replacing the breakpoint condition string with a scriptblock, enabling me to break at specific code locations in response to WMI events for instance.  Not sure how I will accomplish this yet – the condition doesn’t appear to be pluggable, but I’m sure there is something that can be done.