I have nothing against Go (in fact I like it) but they seem to have built GOV.UK with every damn bit of technology there is available. I've seen Varnish, Go, Ruby, Python, Scala, Java, Mongo, MySQL, nginx, rails, sinatra, Django and PILES of Not Invented Here.
Also I'm not sure it's a great use of public money to build stuff like this when they should have nginx/apache up front and some configuration.
In fact this whole thing stinks of crappy information architecture resulting in a massive front end router hack that they did in Scala, binned and moved to Varnish, then binned and moved to Go.
This is not one "cohesive" web front end. It's a rat's nest and an increasingly expensive one.
This is all outlined in the blog post - perhaps not as clearly as I would have liked, though!
Varnish is different as that requires restarting.
We do the same with commercial kit (Riverbed) with over 140 HTTP application endpoints and that is higher friction than the equivalent setup yet we manage it with a mere 1 person...
Deployment is a solved problem. No offense but you're not Google!
It seems they're allowing team members to play with "language of the week" (lotw), build something that's then used in production without the rest of the team knowing the lotw, that member moving on and then rebuilding it in a new lotw 6 months later. no doubt this GO implementation will go the same way at a later date.
I'm not opposed to using a lotw in production with other languages and such, but it needs to be done in a thought out manner with the whole "what if the project lead gets hit by a big red double decker" mentality.
source: I used to work for local government
However it doesn't excuse people working on a web site funded by the taxpayer using it as a technology playground.
Establishing technical standards and homogenity is important on projects of this scale.
Though they do seem to be using all the new toys maybe a bit more thought about which technology to use rather than jumping from technology to technology.
We publish several huge legacy applications, several huge new applications, several integrations, public web site, documentation, online support. A mere 80-100 million HTTP transactions a day.
Not once have we built something to do it all.
People make changes to their infrastructure. The govuk team are at least brave enough to blog about it, and offer the code up for inspection. Can you post links to similar public contributions you have made?
I'm sure if you joined the team you'd wave your magic wand and everyone would fall in line, and things would be beautiful. Right.
Oh, and I'm sure the golang community are really, really pleased that "in fact, [you] like it" - very charitable of you!
I can't wait to see what you've been up to.
Uk.gov is just the cherry on the top of an old and creaky enterprise design that hasn't kept pace with changes to its mission or the world around it.
The same applies to people working under me.
... something something blub something something agile rockstar ...
Scala is a large and complex language, encompassing multiple programming idioms. Teams programming Scala (as with C++ and many other "large" languages) often have difficulties nailing down what portion of the language they are going to use. "Java-without-semicolons" all the way through to idiomatic, functional Scala.
Go is a relatively small, simple language. You could easily learn Go's basic syntax and semantics in an afternoon or two. Simplicity and suitability for programmers is an explicit design goal.
Switching away from Scala due to lack of experience while simultaneously picking up Go is not as illogical as everyone seems to be implying.
I think the argument in the blog post is weak because it does not quantify what makes Go more suitable for learning than say, Erlang or Clojure, which are dismissed outright, before moving on to discuss some of the advantages of Go in depth.
They lack an architectural oversight, that they should really be using one(ish) language rather than a big mix of languages so that this doesn't occur again when no one on the team knows GO because the project leads taken a new job.
A crappy PHP dev will learn Go a hell of a lot quicker than Scala, IMO. A safer choice, for their team and position, IMO.
There are good arguments for trying new technologies, but this particular instance seems to be a very good example of how such a strategy can go terribly wrong.
[Edit] On second thought, I might be jumping to conslusions as I don't know anything about how successful they are in actually achieving their goals, doing it on budget or how fun it is to work there. Their actual approach may be less confused than the blog post makes it sound.
Hell, some people like Perl! :p
* No one having a deep knowledge of Scala is mentioned as a problem,
but they will learn Go.
* They rule out a language because of its syntax.That and I suspect there was some wishlists on what people wanted to do in the mix, like COBOL, still used but people just do not want to learn that as it is deemed so last century in comparsision to say java, though they are altogether different fish. Bit like the difference between a proxy and a router, what sounds better on a CV.
I suspect that from a maintenance aspect that the GO code will be much easier to support and grow as and when needed as well as scaleability.
Still it is most encouraging to see such a young language like GO get so much respect and use so quickly in a enviroment that has to many in the past and present been slow to adapts new technology. But there again many Govermental departments still have XP, albeit used as a fancy terminal to some old mainframe application that just ticks along.
* Reverse proxy, forwarding requests to and serving responses from multiple backend servers on a single domain.
* Redirector, serving HTTP 301 and 302 redirects to new URLs.
* Gone responder, serving HTTP 410 responses for resources that used to but no longer exist.
Looks like they didn't actually try it or know about it which is worrying.
Either that or it was more interesting to do it in Go which is not a valid reason in a taxpayer funded site.
"Scala is great for performance, but quite bad at resource usage"
"No-one in the core GOV.UK team had a deep knowledge of Scala, and particularly how the old router worked"
Oh, then it's OK. It's always reasonable and a safe bet to rewrite something instead of understanding how something works, because after all, your predecessors were a dumb bunch and you (using Go! how cool is that!?) obviously know much more.
Why? There are other resources than CPU time, e.g. memory.
Is there a different definition I'm missing? I think, a car can be "fast" and take too much fuel, but (in my book) a programming language cannot be "great for performance" while being bad at resource usage.
If it's good enough for Cloudflare it probably is good enough here.
Besides, Varnish also would have excelled (yes I read the first post in the series) if they had just stored the configuration in a saner format and then output the required VCL when the config changed and reloaded on the fly.
Both of the technologies I would've picked (nginx and Varnish) were in their stack.
I love working in Go, but reading this feels like a reinvention of the wheel. Go would be my choice only if these other things didn't exist already and you had to start from scratch.
The biggest pain identified was the writing, maintaining and loading of the router config. Openresty could've solved that.
Also I'd Erlang is state of the art, even though it is so old (Haskell too), but I wouldn't call it an entirely safe bet because it requires quite a different style of programming and experience, so might not work well for certain developers.