Edit: Please note that the question is not whether or not Rails and Django developers will still be in demand in 2016. I'm confident they will be. The question is will the best new projects in 2016 consider using Rails and Django.
The impression I get, is that in general, these languages and frameworks will age, get boring, and eventually be something used because of legacy, the existing codebases. Is this bad? Of course not! It means there will always be a wealth of well-documented, high quality software to work with.
But really, the only way to fight off not being 'boring' is by trying to always do the brand new technologies, always having a way of implementing them. Rails with Turbolinks, to pseudo-mimic frontend JS frameworks. Rails also implementing server side events, moving web sockets closer to 'the norm'.
Will they still be the hot go-to languages/frameworks 5-10 years from now? Probably not. Rails will probably feel then like PHP does now('old and safe', with better options available for most cases), and something like Go or some other incredibly thin means of maintaining a backend API to facilitate frontend Javascript frameworks. And after that, who knows. Maybe the web as pages will die 10-15 years from now, instead being replaced by something more akin to a pluggable game or OS engine. Maybe holograms. Maybe we just get robots and call it a day.
Having worked on Rails since 2005, I think the bigger problem is that it has become more complex, and harder to get started with, as the pendulum swings from eliminated.
One bigger change is probably as you state, that we will get away from frameworks whose primary responsibility is generating HTML, which affects all of these things similarly, and node.js would get caught up in the same thing.
Another big change is the pendulum swing from Object/Relational systems to Functional/BigData systems. This is a big mental shift for programmers trained in the Object-Oriented approach.
I'm a Ruby/Rails developer (and I run MemphisRuby) and while I seriously enjoy working with Rails I don't expect it to be as popular ten years from now.
Rails is still tremendously good at what it does and I don't think it's likely that another framework or language is going to beat it at its own game. The real question is whether or not the things that Rails does best will continue to be important in the marketplace.
We've already seen parts of Rails's core competencies carved out by other tools: Erlang/Node/Go enable saner async programming. Backbone/Angular/Ember enable rich web applications. Scala/Clojure/Go/etc. on the "fast compiled backend services" front. Even PHP is keeping pace with the times (Composer, Laravel) just enough to keep its adherents from having a real reason to jump ship.
Ruby and Rails are still pretty much as good as it gets for rapid development of web applications (the backend parts of MVC, anyway), and there's a growing Ruby niche for configuration management (Chef, Puppet, etc).
Personally I find my investment in learning and working with Ruby has been a great way for me to experience a much wider variety of technologies and architectures than I would have otherwise. Thanks to Rails work I've learned tons about AMQP, Postgres, Chef, Backbone, JavaScript, CSS, Ansible, DNS (bind), and more that I just wouldn't have had the same exposure to in a slower-paced development environment.
Your ending line about "still in wide use but not frequently selected for the best new projects" is a bit complicated. Certainly any language stays in wide use once it falls out of fashion and its projects categorically move to the "legacy" phase, but "best new projects" is a bit much. A lot of what we might think of as the 'best' projects are driven by fashion just as much as technical merits.
But the question how much demand will there be for that style of app in the future?
In a similar manner, I was asked the other day if I still did any Delphi development - for the first time in about eight years. Delphi was a best-of-class tool in its day, but there just isn't the same demand for database-driven desktop Windows apps any more.
http://www.indeed.com/jobtrends?q=ruby%2C+python%2C+php%2C+n...
The shift to front-end dev with Backbone and Angular is an important insight, as server-side frameworks may dumb down over time and/or be replace by pure data access like Parse.
Here is the same graph with absolute values, which shows that Scala is completely invisible compared to the competition:
http://www.indeed.com/jobanalytics/jobtrends?q=ruby%2C+pytho...
http://www.indeed.com/jobtrends?q=ruby%2C+python%2C+php%2C+n...
1) Rails (I don't know Django) will continue to be the quickest way to have a quick full-stack demo/MVP. It is opinionated, and has tools like railsapps that let you customize it. So you can have a solution that includes bootstrap, authentication, stripe-payments, etc., almost completely out of the box.
2) Rails will continue to be plagued with the problems (security, slow dynamic language) etc. but it will always have workarounds like jruby, celluloid, rails-api, in-memory-models (I wish this was more common). The community is vibrant enough to keep plugging along and fixing the gaps.
3) I don't think it's a coincidence that the two most influential frameworks (Rails and Sinatra) were written in Ruby.
4) Eventually people's MVP will move to something like Scala, following the example of Twitter and LinkedIn. Static languages that encourage functional behavior improve speed and safety.
5) Play is interesting. I am checking it out now, and seeing how I can move a rails app to it.
6) Node serves the lowest-common-denominator in my opinion (it caters to non-polyglots). I use it for certain things (a workflow for prototyping phonegap apps, and client-side tooling) but I personally think it is way overhyped. However it definitely has a place, and will continue to have one.
7) If Rails does someday "die" that's okay for dhh's legacy. His framework is spiritually part of whatever comes next and raised the bar as much as Seinfeld did for television.
You admit that you're a Ruby developer, so you must see that this is slightly blinkered. Influential on what? There's not much Rails brought to the table which was entirely unique, it was just a very good MVC framework in a pleasant language that came along at the right time. And lets not forget that development on Django (which was also very influential), began long before Rails appeared on the scene.
Sinatra I've only heard of in passing, but in the Python world, web.py, Tornado and flask have been also been very significant.
On frameworks in other languages, both in terms of design/motivation and expectations; one indication of this is the frequency of phrases like "sinatra style framework" or "rails style framework" to either (a) describe an existing web framework for a different langauge, or (b) describe what someone is looking for in a web framework in a different language.
I'm not sure I've ever seen the same thing with Django, web.py, or Tornado. I've seen it with Flask, but its been basically a subset of the times I've seen it with Sinatra (that is, I've seen things like "Sinatra-style framework" and "Flask/Sinatra-style framework", but not "Flask-style framework" on its own.)
As a company, if you push yourself into a corner with newer tech like Node, Meteor, etc. you may come up against hiring issues. Not only that, but Rails and Django are also evolving with time so they too get better with time.
The "age" of a technology isn't the only reason why it gets selected, it gets selected because of a number of factors including features, programmer availability, programmer expenses (in terms of both time and skill), proven track record, etc. Usually age is a plus point in many ways.
Rails and Django will certainly be around and competitive. However, if you're asking from a programmer salary perspective then yes, there will be many more Rails and Django programmers by 2016, so moving into more edgy tech might give you a USP, but you may also end up without a place to work in. Maybe knowing Rails/Django + "shiny new tech" is a better combo ?
For prototyping, maybe (and even that is arguable).
In practice, I see a lot more companies moving away from these frameworks toward JVM-based frameworks than the other way around.
These frameworks simply don't scale past the size of a small project.
These are not small projects.
That is incorrect. From the Django side, see Instagram (http://instagram-engineering.tumblr.com/post/13649370142/wha...) and Disqus (http://blog.disqus.com/post/62187806135/scaling-django-to-8-...).
Django is sadly hampered by two fairly complex problems. The first, is their insistence on backwards compatibility and stability. While it's becoming obvious that the basic design of Django creates serious problems when moving away from the request/response cycle, it's a very hard problem to fix if you can't fundamentally change core assumptions within the framework. Second, Python 3 (sans Tulip) just isn't up to snuff for asynchronous coding. Twisted is a mess, Tornado is far too simple.
The Rails team seems more willing to break with the past between releases, but its still hampered by Ruby's inherent performance disadvantages.
Ultimately, while none of these frameworks are disappearing any time soon, it'd be fair to say that many engineers, particularly those working at the edge of the industry, are choosing things like Node, or Go.
Some of the reasons Rails and Django might fall into legacy could be the due to the underlying language itself. If Python continues to be affected by GIL related multiprogramming issues or lack of stronger types, then the associated web frameworks like Django would also be affected. Then there are always non-technical factors like hype and community interest.
Currently, Rails and Django have crossed a certain critical mass in adoption, they are effective for a majority of web programming usecases and have a thriving third party developer community. So my guess is that it would be pretty much a solid choice in 2016 as well.
[1]: https://docs.djangoproject.com/en/dev/misc/design-philosophi...
Certainly not Node.js. The tooling isn't there yet, the javascript language is still a barrier for maintainable code and the asynchronous model as a default comes with very high complexity costs.
Play requires Scala, and I just don't see typical web folks switch to Scala (an otherwise great language) in droves.
Clojure? Really? Yes, Lisp is great. But it's not going to get popular in 2 years and it seems to require some intellectual fortitude that is probably harder to find on the job market than Django talent.
Sure, Django can be used to build single page sites and flow-heavy web apps. But that's not its strength. Just as you end up having to do a lot of custom programming to make Meteor or Node/Derby work for a site with 15 different content types and multiple backend administrators, but that's not their strength.
Many of these frameworks won't disappear but I think adapt to the newer trends.
Whether we like it or not java is still here and used, I don't see why rails and django won't be.
The whole point of software is it evolves. These projects both have enough of a following to receive updates to fit those new paradigms.
They may not be as suited for something built from the ground up for that specific use case, but that doesn't mean they will disappear.
We designed our JSON model differently than Ember Data expected, though, so we basically had to make our own Ruby GEMs and EmberData adapter. But with those in place it's fairly easy to use Ruby as just a back end.
There's a lot of great libraries out there for this use case. Not surprised you had to do a bit of customizing though.
I think over time this will be come easier if not baked in to the frameworks themselves.
Were you using the Rails asset-pipeline to server the Ember app, or did you separate the Ember app (served statically and hitting API)?
Backends as HTTP APIs with a javascript heavy front end are becoming very common; REST, not as much, though it seems to be popular now to call any non-SOAP HTTP API a "REST API".
For awhile, I didn't touch Rails but went back to it and was pleasantly surprised that even though I enjoyed using Sinatra and Padrino, the weight that Rails's magic adds is counter-balanced by convenience that is most welcome to developers.
I contrast this with Drupal/Wordpress, in which the version upgrades make the platforms easier to build out for clients, but not much easier to develop for, in terms of either developing plugins or customizing the install (not sure with Drupal, I quit at around v5).
To me, it seems like Rails is adding some things I may never use, but those things seem to be optional...And from 3 to 4, they did cut back on some magic-hacks, such as the magic-finders that were too susceptible to security flaws.
I think a big part of why Rails may have better longevity than previous uber-platforms is because of its focus on testing. So it's quite feasible that Rails can always adapt (by being more modular) without causing unmitigated chaos in the developer ecosystem.
For example, at aha.io we use Rails, and wanted to get the front-end performance of a Javascript-heavy app, but by taking advantage of what we know (which is writing ActiveRecord models, Rails controllers and ERB views). So we borrowed a technique from Meteor and made our views reactive - using Rails for the view rendering. That way we get the best of both worlds: the code only needs to be written once in Ruby, but the performance and reactivity is like an app that has a Javascript front-end.
This is just one example, but there lots of similar examples (think about the increasing use of functional programming styles). The incredibly rich ecosystem around Rails means that stuff just gets done faster - the ecosystem has done most of the work for you. Apart from Django, every other framework is still playing catchup.
Do you have any links or resource to share on that style?
The beauty is that it supports even the most complex reactive update use-cases. E.g. reactively updating sort order of a list for one user when another user renames an item in the list. The key benefit though is that you allows you to just write Rails code - there is no need to write Javascript for each new reactive view.
Node.js is a joke, I see more and more people "get it". It sure serves a purpose, but the language not being for general purpose, the syntactical problems of JS and the single-trick concurrency model don't make it fit for true disruption.
Meteor might be a contender for "next level" (currently it is build on Node i think, but that might change one day), I also consider Yesod (Haskell) and some of the web stuff on Clojure to be good contenders.
I think it will not be merely a new FW that changes the scene, I expect it to be a new language along with that.
Play is an evolution for sure. But I'd consider it "beyond". Validation concerns are where they should be: At input. Not the Models. Comet, Websockets, etc are all trivial. Chunked Responses FTW. You don't need different responders. Render _anything_ by having a Writable in-scope.
Our API calls to render our models as JSON look like this:
api.photos.get(id) map(Ok(_))
Honestly, I find Play/Scala _more_ productive than Ruby MVC in general. Fewer bugs. Much better performance and memory usage, so fewer constraints (that one is huge). Rework is much easier, but also much less necessary. The language itself is a joy (IMO) to work with.
The Java ecosystem is a huge enabler. You can have complete control over resizing images in ImgScalr, faster, with no external dependencies, in less code and a flatter learning curve than anything I've ever used in Ruby. Scala itself presents a lot of new things to learn and get comfortable with language-wise until you feel you're back in the groove, but once you've got it, I've found that the ecosystem makes the complex things much easier, and the language itself makes the easy things engaging as well.
Ruby doesn't really help much trying to tackle hard problems IME. Image Resizing? Better shell out to ImageMagick. Dubbing a video? Hello shell my old friend. Cache coherency across a cluster? Fire up another external service (maybe Redis) to manage and start working through failure modes and failover scripts if you care about availability. Compare that to Hazelcast. Need to serve static assets? Better setup an NGinx proxy. In-memory near-caching? You don't have the memory for it, and the GC can't handle it anyways, stick to Redis.
Put a crew of experienced Play developers head-to-head with experienced Rails developers, give them the same goals (and non-goals), and you'll have a faster, more maintainable solution from the Play team in the same or less time for anything but the most trivial apps.
IMO. :-)
I really don't want to handle the side effects of dubbing videos in my web app. That kind of stuff belongs in a dedicated service anyhow - big monoliths are really the killer of maintainability.
In many ways, it is beyond Rails. For instance, it offers type-safe URLs and builds upon the Akka actor framework (allowing you to schedule asynchronous tasks, make reactive web applications, etc.).
This is really where Play shines. I wouldn't bother with Play for a basic CRUD app, it makes you do too much grunt work yourself (e.g., authentication). But when you have a backend built on Akka and want to build a reactive, event-driven front-end Play makes it trivial.
Akka is the key. Its actor model just makes reasoning about concurrency so easy because of how it deals with state. Add in remoting and clustering and you've got a highly scalable, event-driven clustered backend without too much extra work on your part. Play fits into that sort of ecosystem quite nicely.
However, before I can build the web framework I want in Rust, I'm having to write the HTTP library I want in Rust (with similar goals), and that's the stage I'm currently at: http://github.com/chris-morgan/rust-http.
Rails has quite a bit of life left. Sure, with more choices of frameworks available, there'll be fewer cases where it's the only viable choice for a project, but it will be competitive for some time.
So, let's first look at what Rails and Django can do. Ruby and Python are both slow languages compared to Java, Go, C, and even JavaScript (thanks to companies pouring a lot of work into it). However, computing power is increasing. In the Ruby and Rails world, there has been good work on JRuby, Puma, etc. to help concurrency and move away from the process-based concurrency model. I'm less familiar with Django here, but Python does have frameworks built for concurrency.
Now let's look at Node.js, Java, and Go. All three have good support for handling multiple connections within the same process (via. eventing, threading, and goroutines). Rather than taking up a process per connection, they just add a little bit of RAM overhead and get to stay in the same memory space with cheaper context switching (or none in an evented system). That deals well with long-running processes that would otherwise block a process, tie up RAM, etc.
So, Rails is improving its support for running in a threaded environment and while I don't know as much about Django, Python does have other options addressing this issue. But as it stands today, alternatives have a speed advantage and somewhat by-default operate in a better way for concurrency.
Now, if we're looking toward the future, what do we need this for? We don't need this for things like serving CRUD. If you're trying to keep lots of connections for a chat application waiting for a response, that can be better served by alternatives. However, it should be noted that 37signals does chat via Rails. When thinking about the future, you should think about what is happening in the application and how that affects things like context switching, RAM usage, etc. I don't want to predict the future. If you want to, think about how applications may work that would be better served by systems that don't rely on process-based concurrency.
As some unsolicited closing advice, I find that a lot of people spend an inordinate amount of time worrying over their tools. They want to pick the one set of tools that will last for eternity. Just as your laptop will age and be replaced, the code you write will. Obsessing over tools keeps many from doing anything. There's a lot of low hanging fruit. I'm guessing you see things you want to build - things you could improve upon. Don't worry about having to modify, extend, and do things differently in the future if it's keeping you from doing anything today. Tools are just that: tools. They aren't your product. They're meaningful. Using a better tool is more enjoyable and can create a better outcome. But don't obsess about it trying to guess the future and insulate yourself from future work. Build!
Although I personally believe that with present-day framework performance spanning multiple orders of magnitude, the lower-performance frameworks have a substantial amount of ground to cover before I personally would consider using them. Higher-performance frameworks have, for my use-cases and in my opinion, reached or nearly reached parity in developer efficiency while providing significant pain-reduction on the performance and scale side. In other words, why not enjoy both performance and efficiency?
That said, everything is evolving, so whatever we see today is subject to change. As you said, our answers to this question will be a matter of opinion and belief about what will happen in the future.
Looking forward with only the context of today's technologies, I personally believe that WebSocket and the various protocols built on top of it may eventually supersede the conventional HTTP request-response conversation. To me, this would mean simpler and more direct communication between client and server, putting the two on equal footing with respect to the conversation semantics (the server can take the lead, for example). It's something I have been wanting ever since I replaced an old client-server app with a web application in the late 1990s with a boggled sense of "Why am I shoehorning an application on top of this hypertext document fetching protocol?"
So when building an application today, I like selecting platforms and frameworks that have natively implemented WebSocket.
But echoing your point again, it's impossible to know what else will come up in a few years' time. Unless your system needs to survive for a very long time, technology is too unpredictable to expend that much worry on the long-term future.