Tuesday, 26 March 2013

Software Procurement in the Public Sector

Last week, I was a member of the panel at a workshop hosted by ITAC Health.  The goal of the workshop was to bring public sector buyers and sellers together to fix a broken procurement system.  I was a member of the vendor panel, representing mid-sized Canadian software companies.  The buyer side was well represented, including an Auditor General of Ontario and the Assistant Deputy Minister for Ontario Shared Services.  Details of the event can be found here.  Below is the talk I gave at the event and some suggestions vendors made for improving the public sector procurement process.

Building a Software System is not like Building a Subway System

As a Software Engineer, the main barrier I see to successful software delivery in the public sector is a misunderstanding about the nature of software.  Software projects in the public sector tend to be managed like major construction projects: like building a new subway system.  How do you manage risk when building a new subway system?  You do years of up front planning, specifying all the details of the entire project long before the first shovel breaks earth.  Why do you do this?  Because re-routing a subway tunnel is very, very expensive.  You have to get it right the first time.

The mistake we make in the public sector, is that we treat software systems the same way. Software is different.  Unlike a major construction project, it is in fact relatively inexpensive to alter a software system after it has been built.  Building a software system is more like building a successful political campaign platform.  Successful political campaigns commit very little up front: They throw out teasers and then poll intensively to suss out the public mood--which parts of the new platform does the electorate hate, which parts get the public excited--then based on this feedback, the direction of the campaign is altered.  It is not a subway tunnel, planned years in advance.  It is built incrementally, guided by constant feedback from the electorate.

To give a specific example, consider Ontario's Medication Management RFP.  Ontario started writing this RFP 6 years ago.  The RFP was finally issued 3 years ago.  Today it is still not awarded.  We've been planning this projects for 6 years now, and all we have to show for it is a stack of paper.  If a middle-aged person arrives at an EMR today unconscious, we have *no* way of knowing what medications they are on.  Why are we at this point?  It's because the scope of the Ontario Medication Management system is huge.  It's being procured like a new subway system: with massive scope and comprehensive specifications.  The scope includes real-time transactional integrations into pharmacy systems, real-time e-prescribing integration into physician systems, real-time patient lookup into patient registries etc.  These are ambitious plans.

Imagine, if instead of this, 6 years ago we decided to build the software incrementally.  We started with a nightly batch upload of all prescriptions from all pharmacies to a central database and then gave EMRs access to this database.  Pharmacies already batch upload their prescriptions to other partners, so adding a provincial drug database to the feed would be a simple project for them.  Had Ontario instead started with this scope, I believe we would have had a comprehensive province-wide prescription database within a year. Sure the prescription data is a day old, but day old data is a heck of a lot better than *no* data.  More significant than that, having a real system out there, in the hands of users, allows you to start polling your users--which parts of the new platform do the users hate?  Which parts get the users excited.  Based on this feedback, you can steer the evolution of the system, often in ways you could never have anticipated at the start.

6 years is an eternity in the world of technology.  6 years ago, there were no iPhones.  3 years ago, when the Medication Management RFP hit the streets, there were no iPads.  Subway trains don't change that much.  But software changes a lot.  When you're planning a software project, particularly something as important as a public health system, if you plan too big for too long, your system will be obsolete by the time it's launched.  Start small, get feedback, and  incrementally improve.

Fix the Q&A Process

I also wanted to touch on an aspect of that process that I think is particularly broken, and that is the RFP Q&A process.  The intent of the RFP Q&A rules are to level the playing field--to ensure all vendors have a fair chance at winning.  The effect of the RFP Q&A rules is exactly the opposite.  Vendors on the inside already understand what the customer needs, and those on the outside have no way of finding out.

The rules require that any question a vendor has about an RFP must be submitted on paper on the public record, with the answers to the questions being shared with all bidders.  While this process looks good on paper, this is what actually happens:

  • Meaningful clarification questions are rarely asked for fear of giving away your requirements analysis advantage to your competitors.
  • On the rare occasion when meaningful clarification questions are asked, 9 times out of 10 the answers don't help.
Software requirements cannot be understood through a public Q&A process.  Delivering software is fundamentally different from delivering milk.  What's required is a conversation, with only the vendor and the customer in the room, where the vendor can ask insightful questions and the customer can answer honestly. It is very, very hard to successfully bid on a project when you don't understand what the customer actually needs, and a public Q&A process has proven that it can't get us there.

To make this work, there would need to be some sort of short-list qualifying process so there are a manageable number of meetings.  But often times there are only 2 or 3 bidders.  Even a confidential Q&A would be more fair than what we currently have.

Contract Templates

We heard a lot at the workshop about how attaching a unique contract in each RFP adds delay to the bidding process, and how many clauses in these contracts are showstoppers for many vendors, especially clauses regarding unlimited liability, unlimited indemnification, and IP ownership.

Why not publish, say, 5 contract templates that are be used for all public sector procurements, and then have the RFP simply refer to the contract template by name.  This would allow vendors to pre-qualify which of those templates they can bid on and which they can't.  Such pre-qualification could even be announced by vendors if they chose to, so that when selecting a contract template, governments would know in advance which vendors they are excluding.

RFP Star Rating

Those of us who regularly read RFPs know that there is a huge variety in the quality.  Vendors prefer RFPs that describe the business problem and leave the implementation details to the vendor.  Vendors prefer RFPs with clear scope and deliverables.

One of the things Marion Macdonald (ADM Ontario Shared Services) mentioned at the meeting is that Ontario plans to change the system it uses to manage the RFP process.  I would suggest to Marion that she consider allowing vendors to anonymously assign a star rating to posted RFPs.  I imagine a future where budding RFP authors download all the 5 star RFPs as guides for how to write a really good RFP.

References vs. Innovation

My co-panelist Michael Martineau raised an important point, which is that we make a lot of noise about fostering innovation, but then when it comes time to RFP, the RFP inevitably requires "three sites in Canada where the software has been installed for over a year" or somesuch reference requirement.  You're not going to introduce innovative solutions into the marketplace, particularly from other countries, so long as RFPs contain language like that.


  1. This is an insightful blog Ken. Keep up the good blogging!

  2. Ken is right on the money with this post. In my former role, I bid on several eHealth initiatives and found the process incredibly painful. I would add that the eHealth architecture team imposes such a technical straight-jacket on potential solutions that systems that work in other jurisdictions cannot be used "out of the box". This causes extensive rework that adds significantly to the cost that taxpayers must bear as well as adding to the risk of the project.

    Confine the RFP's to business requirements and let the vendors sort out the technical solution. Please.

  3. This comment has been removed by the author.

  4. This post should be required reading. In particular, your insight into MMS. I also like your idea in terms of star ratings on RFP's. Let's make the stars a compensation factor for those writing the RFPs. And my feedback wouldn't need to be anonymous. :-)