Introduction

Click here to view the source code

So, I came across quite an interesting situation approaching a new project that had been developed for several years. All of the library projects were never updated past .Net 3.5. Apparently, the developers did not want to take on the arduous task of not only updating all their projects, but also not deal with upgrading one of their ORMs, nHibernate, and the changes the software made between full releases. Which is understandable considering there were close to 200 projects and much of the code infrastructure was not encapsulated. This form of neglect seemed quite troubling to realize since these projects had to get upgraded eventually and left an absurd technical debt on future developers. Well, once the original programmers decided to leave I decided to take it upon myself to try and update these projects myself. Partly out of curiosity, but  mostly because I am a bit of a masochist. In order to start this process though, I decided on letting Visual Studio handle the upgrade process for me. In order to do that I first needed to figure out how to place all my projects into a single solution.

Project Creation with EnvDTE

The first thing I decided was, given the disorganization of the code base, manually upgrading projects was not realistic. Knowing that, I started figuring out how to automate searching through all projects and adding them to a single solution. I knew I would have to make use of Visual Studio’s SDK, more specifically the EnvDTE namespace, which is used for Visual Studio’s core automation process. So I went ahead and started creating a separate class library dedicated to using EnvDTE objects. The first thing I had to add was the base EnvDTE projects:

As I was going through the process of building out my code, I ran into an issue with the EnvDTE.Constants references:

With my lack of experience with Interop Types, I fortunately managed to find a solution here. Apparently Visual Studio just has an issue attempting to embed certain assemblies and the way to get around it is to set the property “Embed Interop Types” on the assembly to false. Through a matter of trial and error I managed to find the ‘envdte’ assembly was the one causing my issues and fixed this promptly:

With this I can now begin creating wrappers for the features I want to employ with the EnvDTE namespace.

Wrappers and Factory

Two of the main features I wanted to take advantage of the EnvDTE namespace are the EnvDTE.Project and _DTE.Solution interfaces. So I went ahead and created two class files to represent these; ProjectWrapper and SolutionWrapper. The ProjectWrapper class is fairly straight forward, which merely exposes one property of the EnvDTE.Project. I started out this path to decouple any issues I may run into for future projects:

public class ProjectWrapper
{
     private Project _project;

     public ProjectWrapper(Project project)
     {
         _project = project;
     }

     public string FullName { get { return _project.FullName; } }
}

As for the SolutionWrapper, much of the project aggregation process exists in this class. To start, we first must create our DTE object, which will contain the Visual Studio solution. I went ahead and place the implementation inside a factory:

internal static class EnvDTEFactory
{
    internal static DTE Create(VisualStudioVersion visualStudioVersion)
    {
        var vsProgID = visualStudioVersion.ToDescription();
        var type = Type.GetTypeFromProgID(vsProgID, true);
        var obj = Activator.CreateInstance(type, true);

        return obj as DTE;
    }
}

The VisualStudioVersion is an enumeration I created to represent the different versions of Visual Studio. The enumeration contains a description attribute that represents the Visual Studio version’s ProgID:

public enum VisualStudioVersion
{
    [Description("VisualStudio.DTE.12.0")]
    VisualStudio2013,
    [Description("VisualStudio.DTE.14.0")]
    VisualStudio2015
}

Since this automation process does not always perform consistently when attempting to add old projects into newer versions of visual studio, I had to have the flexibility to either use Visual Studio 2013 or 2015 (the solutions were at least kept fairly up to date).

Before we can begin aggregating all the projects into our solution, first we have to gather all the project files that exist in our root directory:

private IEnumerable<FileInfo> GetProjectsMissingFromSolution(string rootPath)
{
    var projectsInSolution = GetProjectNamesInSolution();
    var projectsInDirectory = GetProjectFilesInDirectory(rootPath);

    foreach (var projectFile in projectsInDirectory)
    {
        if (!projectsInSolution.Any(p => p.Contains(projectFile.Name)))
        {
            yield return projectFile;
        }
    }
}

As the method name GetProjectNamesInSolution suggests, we first look inside the given solution file, if it already exists, and read the contents through a stream to find all the current projects files. Afterwards, we find all projects in our root directory then ignore any projects that are already inside our solution.

At this point, we first need to open the solution to allow for the addition of projects:

_dte.Solution.Open(_solutionName);

When this occurs successfully, you can actually see the solution in your Task Manager Details tab, but not in the Processes tab. This is good to know in case the application closes abruptly. Although I do my best to clean up these solutions on completion, if you happen to force close the process, these instances will continue to exist until you clean them up yourself:

At this point we are ready to iterate through our projects and include them into the solution:

private void AddProjects(FileInfo[] missingProjects)
{
    foreach (var project in missingProjects)
    {
        try
        {
            AttemptTo(() =>
            {
                if (_projectWrappers.All(p => p.FullName != project.FullName))
                {
                    AddProjectFromFile(project.FullName);
                    _logger.Log("\tProject Added: {0}", project.Name);
                }
            }, 3).Wait();
        }
        catch (Exception)
        {
            _logger.Log("Skipping adding this project for now");
        }
    }
}

You will notice the method AttemptTo used throughout the SolutionWrapper class. This method will simply attempt to re-execute a given action an X number of times. The reason I need this is because as quickly as it may seem to execute actions through the EnvDTE interfaces, in the background sometimes the process takes longer than expected to complete your last request. So, when this occurs and you attempt to perform another action, an exception is thrown with this message:

The message filter indicated that the application is busy. (Exception from HRESULT: 0x8001010A (RPC_E_SERVERCALL_RETRYLATER))

Although I did find a solution, which involves using the COM interface IMessageFilter. Using this, I could read in the RetryRejectedCall event, or even the Servercall_RetryLater and force the thread to retry the event on its own without throwing an exception. Unfortunately I was never able to get this to work, so instead I accommodated by catching the specific exception, sleeping the thread for a few seconds, then trying again.

For each project, I only need to tell the solution to add it based on the file location:

var project = _dte.Solution.AddFromFile(fileName, false);

From here, I wrap the returned EnvDTE.Project and attempt to go through the aggregation process again if any projects were missed:

for (int i = 0; i < iterations && _missingProjects.Any(); i++) { _logger.Log("************ Attempt {0} ************", i + 1); _logger.Log("Number of projects missing from the solution: {0}", _missingProjects.Count()); AddProjects(_missingProjects); await AttemptTo(() =>
    {
        _missingProjects = _missingProjects.Where(p => _projectWrappers.All(pw => pw.FullName != p.FullName)).ToArray();
    });
}

The reason why I am going through the aggregation process multiple times is because some projects refuse to get loaded unless another dependent project is already included in the solution. When this occurs, sometimes going through the process again with the missing projects will turn out successful after X many iterations.

After all this we simply need to Save the solution:

_dte.Solution.SaveAs(_solutionName);

And finally close it completely, this includes both the solution and the DTE object that contains it:

_dte.Solution.Close();
_dte.Quit();

Then you’re finished. At this point you should be able to go to the location you specified for your solution, open it, and witness all the projects included.

Issues

Although this process works fairly seamlessly with newer, less complicated architectures, when I attempted to apply this solution to this projects code base mentioned earlier I ran into a plethora of problems and tiny gotchas.

• First of all, when attempting to aggregate almost 200 libraries it can take quite a long time to run, not to mention how many times you have to re-iterate over the project files that end up refusing to add because they need X, Y, and Z projects before they can be included.
• I also found a Solution User Options (.suo) file that was sometimes created after the process finished was absolutely necessary to open the solution again in a timely manner. Initially, efficient habits led me to remove this file when attempting to check it into source control. But the next time I tried to open the monolithic solution, it was taking almost twice as long as when I automated the process.
• I had to also specifically ignore certain project files because no matter how much I tried to fix the library itself, Visual Studio refused to include them into my solution. Some projects were just so old and unused that the IDE just could not understand them anymore.
• One of the biggest issues I was surprised I ran into was dealing with visual studio versions. I actually first started working on this problem under Visual Studio 2013, but as 2015 came out I naturally switched to that IDE, but found I could no longer aggregate more than 1/4 of all my projects. Even specifying ‘Visual Studio 2013’ as my vsProgTypeID made no difference. I had to both use Visual Studio 2013 and set my vsProgTypeID to be the same in order for this to work. It makes sense since the current Visual Studio may not know all the differences used from other versions. Nonetheless, this caught me off guard for a moment.

Conclusion

Overall, I am quite happy with the solution I made. I was able to aggregate all of the relevant libraries along with quite a few defunct ones, which I only figured out after noticing no usages for those projects. This was also a great debugging tool since so many common dependencies were sprawled across several solutions. If a change was ever made from one, it was extremely difficult to tell where that change would break elsewhere. With the aggregated solution, I was able to simply find usages of a change and make the appropriate updates. Also, at the time, I was able to leverage Visual Studio 2013 Ultimate’s Architecture Tools, which made weeding out unused libraries. Now that I was able to collect all my libraries into a single solution, the next step was to find a way to automate the upgrade process.