This post is part of a series in which I explore the building of DNArtwork.com. It’s about how I’ve decided to use Microsoft .NET and Azure as the infrastructure that supports the website because they’re cheaper and better than other technologies. There you go, that’s the conclusion of this article. If you’re reading this series for the DNA and the artwork, you can move along now.
If you’re interested in why this is surprising, then I’ll tell you a story about the computer industry. Not all that many years ago I was personally committed to fighting against Microsoft, which I and many others saw as a corrupting force in that industry. Now I’m actually quite excited about using their new software. This is quite surprising to me, and this article is my way of figuring out how this happened.
By the way, this is one of my occasional long rambling posts. You have been warned :o)
Part 1: the empire and the rebels
When I was a university student teaching myself computer programming, every tech enthusiast I knew believed Micro$oft to be evil. We might have got a little carried away with this belief, egged on by a Star Wars-inspired predilection towards plucky rebels (in the form of free software like Linux) taking on Big Mean Evil Empires. But we did have some good reasons for disliking the company.
Microsoft grew up in a world where IT was purchased by big companies with large budgets, purchasing decisions were made by men in suits who had never written a line of code in their lives, and professional developers lived with those decisions because they were paid to do so. Microsoft wasn’t interested in selling individual products to customers who’d pick and choose the best product for each of their requirements. Their strategy was to create a family of products that covered all common IT requirements and worked well with each other but not with the competition. Software teams would design their creations on Windows (~$300) using products like Office (~$500) & Project (~$600), communicate using Exchange (~$600), write code using Visual Studio (~$1000), share it with the team using Visual Source Safe (~$500), deploy it on Windows Server (~$1000) using an SQL Server database (~$6000 wait what?!?), and… well you get the idea. The pitch was “buy Microsoft for everything and all your software will just work together”.
Software developers tended to adopt the Microsoft product family wholesale and make software that only worked on Windows, or avoid Microsoft as much as possible. I was in the latter camp, and me and my young idealistic techie friends would refer to Microsoft as The Borg (after a race of parasitic aliens in Star Trek) and to developers who worked only with Microsoft technologies as having drunk the Kool-Aid (explanation for non-American readers).
It wasn’t the price of Microsoft software that bothered the young techies since they’d just pirate the software anyway*, it was the feeling that Microsoft put the minimum necessary effort into their software and relied on marketing to middle managers and various kinds of unfair competition to achieve high market share.
* I’m told. «cough»
For example, Microsoft internally referred to their process for bringing new product categories into their ecosystem as embrace, extend and extinguish. They’d enter new markets where the existing vendors had agreed on a standard that made their software interoperable. Microsoft would extend the standard with their own modifications. The effect would be that if you used the Microsoft product then you could read files from the other vendors, but users of the other vendors couldn’t read your files. Moving to Microsoft became a one-way street. New customers would buy the Microsoft product even if it was technically inferior and more expensive (and it often was) as that was the only way of being able to guarantee that they could read files from both MS and non-MS users. The open market for that technology would wither and die.
All this isn’t to say that Microsoft didn’t have some excellent technology. Some of the best developers in the world work for Microsoft and they produce some amazing software. Microsoft are particularly good at making developer tools: software for making software. Windows, .NET and Visual Studio are all excellent works of software engineering. But Microsoft weren’t interested in making them easy to use on their own – they’re supposed to be the carrots that lure developers into the Microsoft product family. Unhappy with this “all or nothing” proposition and the dirty feeling that came with supporting a company that seemed to be on track to turn the whole software industry into a monopoly, the more idealistic developers tended to ignore the whole Microsoft realm and focus their attention on open source software like Linux. It was often messy, unpolished and frustrating to use, but it was free in both senses of the word.
Then, the web happened.
Microsoft was able to dominate the desktop software world because it was there right from the start of the personal computer revolution. It was clear to Microsoft’s executives knew where the value of PCs lay, and could notice new product categories as they became important (word processors, spreadsheets, project management tools, etc) and move in to dominate the new markets.
But the web began by stealth in 1991 at CERN (the European Organisation for Nuclear Research) as a platform for sharing academic documents and gradually morphed into a platform for building and delivering software.
Early development of the tools to create web pages was done by a community of open source developers, using a philosophy for developing software that is in many ways the antithesis of Microsoft’s. This is:
- Software should be developed by, or at least in close collaboration with, its community of users.
- Make small tools that do one thing well and integrate well with other tools.
- Release your experimental software early, even if it doesn’t yet work properly and seek help in finishing it. Later, maintain a stable version for most users and an experimental version for enthusiasts and early adopters.
- Friendly competition with other tools is healthy, and should be based on the merits of the software and its community.
- Developer experience matters just as much as features and benefits.
By the time anyone in a suit realised that there was money to be made in this Internet thing, open source software had an unassailable lead in the tools to create websites. Microsoft tried to create compelling tools like ASP.NET and IIS to lure web developers inside the Microsoft walled garden. They tried the old Embrace Extend and Extinguish trick with Internet Explorer, adding IE-only features to HTML and encouraging developers to make websites that only ran on Windows computers. But it was too little too late. Every exciting development in web technology – Ruby on Rails, Node.js, NoSQL and containers to name a few – has happened in Linux land. Microsoft’s once effective strategy of requiring people to buy into their product family wholesale is now their biggest weakness. Developers aren’t prepared to forgo the new and exciting technologies, so they tend to ignore all Microsoft products.
One of the hallmarks of open source web tools is that they’re focussed on providing a good experience for developers. Since the early 2000’s, large companies like Google and Amazon and smaller startups have been competing fiercely for engineering talent. In addition to paying well, one way that they compete is to adopt the open source tools that developers want to use. Requirements for Microsoft technologies like C#, .NET and IIS all but vanished off job adverts and were replaced with things like Python, Node.js and ${fashionable framework du jour}.
But Microsoft isn’t stupid. They saw what was happening and responded with what appears to be a wholesale cultural change in the way that they deliver web tools. The new Microsoft tactic is softer than the old: make great products, let people combine your products freely with your competitors’ ones, and try and win the battle for market share based on quality, not lock-in.
To this end, the new .NET framework is a series of small open source modules that work on Windows, Mac and Linux. All development is done in the open – the Microsoft employees working on .NET all have GitHub accounts and commit their code to publicly available Git repositories like any other open source developer. You can go to, for example, the .NET Core repo and see all the design conversations and early attempts at implementation, and even contribute if you’re so inclined. This goes beyond .NET – Microsoft’s cloud hosting service Azure supports non-Microsoft frameworks like Node.js, and lets you choose between hosting your site on Windows or Linux. Visual Studio on a PC is now free for individual developers and they’ve released free code editors for Mac and Linux too. Quite a change from 2001 when Microsoft CEO Steve Ballmer said that “Linux is a cancer”.
The result is that Microsofts tools for making websites are now available to non Windows users like me, and I feel they’re making an concerted effort to demonstrate to me that they’re worth adopting. I’m aware of course that Microsoft hasn’t become a charity – the idea of all these free tools they’re giving me is still to lure me into the Microsoft product family so that I pay them lots of money. But the stick is gone, and only the carrot is left. It’s actually quite inspirational to see them fight back in such a positive manner, and I think I know why. Star Wars may be the most popular example of the “plucky rebels take on the evil empire” narrative, but it also has one of the most famous instances of another common movie trope: “bad guy sees the error of his ways at the last moment and and turns around to fight for Good”. Jaws is a better example of this trope but I wanted to keep the Star Wars theme. I’m talking about Jaws the James Bond character here, the shark was a dick right up to the end.
Part 2: hosting web applications
OK, so Microsoft was Evil and now they’re not. Well they also used to be overpriced, let’s see if that’s changed.
Traditionally, the most common model for web hosting is shared accounts on a web server.
The “web server” bit means that you just upload your files to the server, which is as simple as dragging a folder from your computer desktop, and they immediately become available over the Internet. If I connect to berniesumption.com and upload a file “somefile.txt”, that file becomes available at http://berniesumption.com/somefile.txt.
Web applications can be created by uploading files containing code, so if I upload a file called “hello.php” then when someone goes to http://berniesumption.com/hello.php the file is interpreted as a computer program written in the PHP language, and the result of running that program is shown to the user. It’s really simple – if you can arrange files on a computer, you can manage a website.
The “shared accounts” bit means that each project takes up space on the same server, and shares a common set of resources. You buy some resources from a web host – some amount of space and processing power. These resources are shared between all the sites that you upload to your account. In theory, if all the sites on your account experience a spike in traffic at the same time, they’ll all compete for the same resources and will slow down noticeably. But this is rare – a well written website should consume very few resources most of the time, and only very occasionally go into a brief flurry of activity when something changes, like adding a new page or being linked to from a popular site. Sharing resources for multiple sites is a great way to save money, and you always have the choice of putting important sites in their own account to guarantee that they won’t be affected by other sites.
This model worked really well for two reasons. Firstly it was super cheap, especially for hobbyists with many small sites, web design companies that host many client sites, and companies with one big site and a number of small microsites. Secondly, it was really easy to manage – it’s amazing what you can get up and running just by copying some files onto a server. For example, this blog you’re reading now runs on WordPress, and took me all of 5 minutes to set up.
Since around 2005, a number of new frameworks like Ruby on Rails and Django have become popular that break out of the traditional file-based web server model. These frameworks emphasise developer productivity, allowing you to throw together a prototype site in hours rather than days. It was a very exciting time to be a developer – how often do you become 2-5 times more productive practically overnight? But apps created using these new frameworks were much harder to deploy onto the Internet than traditional websites. Instead of copying files onto a web server you have to install, configure and run a program on your server, then set up all kinds of fiddly things like load balancing and scaling. In the early days this was a serious barrier to people adopting these new frameworks, so a new breed of web host product has arisen called Platform as a Service, or PaaS for short.
PaaS providers restore the ease of deployment that we had with file-based web servers. You upload some files to the PaaS provider and it figures out what kind of app you’ve created, installs and configures it, sets up fiddly things like load balancers, and makes it available on the Internet. They’re wonderful products, but in the transition from web servers to PaaS, the economies of shared hosting have been lost. PaaS providers seem to be targeting startup web companies who have a single valuable app. If you have 5 employees, then $50 per month for hosting is nothing if it saves you a day or two of server administration per month. But if you’re a hobbyist with a collection of small projects, you’re still going to pay $50 for each of them, and it quickly becomes crazy expensive.
As far as I can tell, there’s only one provider that uses the old shared hosting model of “buy some capacity and share it between all of your apps”. And that provider is Microsoft, with their Azure App Service.
Again, I’m surprised. When it came to products like application hosting that are useful for big companies, The Microsoft Of Old was famous for pricing that focussed on the needs of big businesses and left hobbyists either pirating software or using alternatives.
Part 3: throwing together an app
So, Microsoft aren’t evil any more, and they’re cheap. But is their stuff good? As I said earlier, most of the exciting developments in web technology happened in Linux land, and Microsoft were stuck playing catch-up. Well it turns out that there’s an advantage to being the last one to arrive at the party.
Developers often use the phrase “throw together” when referring to building a prototype app, and the phrase is pretty descriptive of what happens. We cobble together a site by taking code from wherever we can find it and using it to build the repetitive parts of the app, because if we didn’t then we’d never get any app started, let alone finished.
Imagine you want to build a web app, it’s called Frobulator. Users log in and upload their widgets, and other users can frob those widgets. It’s a brilliant idea, timed just right to capitalise on this frobbing craze that the kids are so wild about these days. But you need to get it done fast while it’s still relevant. You write down some requirements and there are only 5 key features that are essential for the first version. Great, you can get this done in a weekend. You start coding.
Day 1. Item 1: “users can log into the website”.
Here’s where you meet a pattern that happens over and over in software development: the requirement that looks really simple but turns out to be a lot of work. To log in with a username and password you’ll need a login form. Then you’ll need a signup form for new users. And you probably want to validate the email address that people enter by requiring that they click a link that you send to them in an email. You’ll need to store the passwords in a cryptographically secure way so that even if you get hacked, the hacker can’t get the original passwords back. And send password reset emails if people forget their login. And if you’re in the EU you’ll be legally required to provide a way for an administrator to delete a user account and associated data if requested to. Then there are a lot of optional but really nice to have features like logging in with Facebook. Behind the requirement “users can log into the site” there are tens of features that must be implemented and hundreds of ways to mess up the user experience or security. That’s why good developers on tight budgets don’t do this themselves, they use existing frameworks that do it for you.
I’ve used login as an example, but there are loads of other examples of similar issues that look simple, turn out to be complex, and are only tangentially related to the actual user experience you’re trying to build.
One popular way of getting user login and similar features “for free” is to build your site as a plugin to a content management system that provides all of these features. WordPress and Drupal are great for this purpose if you like PHP (perhaps you’re nostalgic for what software development was like in the 90’s). Other languages have good CMSes too, though it’s surprising how far ahead the PHP ones are given that PHP is arguably the worst designed language ever to achieve popularity. Building an app on top of a CMS is a great way to get started quickly, but more often than not you end up fighting the CMS because it thinks you want a content managed website, and really all you wanted was a login box that worked properly.
So what you want is a language or framework that lets you pick and choose useful modules to add to your site so that you can throw together a prototype. One that lets you use, for example, just the user login system and the database access layer but not the whole kitchen sink. It needs to provide common features like login as part of its core feature set, and also have a good community of developers creating extensions to cover the more esoteric needs. It has to be well documented, and have a helpful community of users to give you a hand if you get stuck on a detail.
I can think of exactly three such frameworks: Django, Meteor and ASP.NET (and arguably Ruby on Rails). Of these, ASP.NET has recently been completely rewritten as part of the effort to support Mac and Linux. It’s incredibly well designed (it ought to be, as a new product it can learn from the mistakes that the competitors made). It’s well documented. And it’s the fastest too.
The bottom lines
They say that every cell in the body is replaced in ten years, so we’re effectively new people every decade. It’s not true, but they say it. The point is, I wonder what the staff turnover rate at a company like Microsoft is? I suspect a company like that turns over most of its staff every 10-15 years or so, which means that the Microsoft of today really is a different company to the Microsoft of 2000.
It’s early days, but I figure The Borg deserves another chance. It’s hot, there’s an iced pitcher of Kool-Aid on the Microsoft table, and it’s looking tasty.
