If that is the case, surely we can find better ways to uncover the requirements, and better tooling will help solve the problem.
Experience tells me that people don’t know everything beforehand. Thus the key assumption is not valid.
Then the question we should be asking is: how do we most efficiently bring people to where they discover and understand the requirements?
Experience tells me people are much better at giving concrete, specific feedback to a running system than to an abstract requirements document.
Hence iterative development.
In essence requirements are not a document by a process.
No, people cannot know "everything in detail, in advance". That doesn't mean that they don't know anything. They know a lot. Nobody with any actual experience in requirements-gathering expects 100% perfection. So the underlying assumption about the underlying assumption is wrong.
After 20+ years in this industry, I'm long past believing the conventional wisdom that running systems are the best way to gather better requirements. It's not agile. Think about it. A key part of agile is to push everything to the left as much as possible - to catch problems as early as possible in the cycle. What's earlier than before you write the code at all? Writing code to find out what's wrong with it from a requirements perspective is really inefficient.
This isn't to say we shouldn't get working code out there as quickly as possible, or that feedback from working systems has no value. But this idea that it's the only way to get meaningful requirements, that's just BS.
Requirements aren't a document, or a process - they are a system.
Your push back to waterfall development is driving me crazy - we already tried that for decades and you can only get it to (kinda) work with a ridiculous investment that only makes sense for incredibly important systems, like launching a billion dollar rocket. And even then, you need iteration, just a more more careful, sandboxed type of iteration.
Humans are really bad at designing and building large improvements from paper requirements. Small improvements mean you understand most of the requirements are known and tested, and only small parts are uncertain.
The real problem is that testing requirements is really hard. You need to build the product to test the requirement. That's why most industries have an intermediate between requirements and product that is testable: this could be small scale prototypes, but more and more it's a virtual model that can be tested through software algorithms.
If we want to make real progress in the software industry, we need to move beyond word documents with requirements that are by definition not testable, to testable software models that don't require a full implementation. Low-code, model driven development is an example where this is happening.
1. accept the blame - beef up the requirements gathering process, attempt to gather ever more
2. reject the blame - move to an agile process where everything is learned on-the-hoof
3. reject the premise.
People tend to either land in 1) or 2) above, but I think 3) is the correct place. Gathering requirements is about figuring out how much we know, identifying what we don't know, and working the risks. On some projects the risk is that we don't know enough about what customers really need (= agile engagement required). On others, the risk is literally all about delivery.
Iterative development is great at addressing some risks. It doesn't address other risks at all; it's not well-suited in many instances where the information known up-front is substantial, or where it's difficult to engage users.
The key is to recognise what problems you need to manage, and choose a suitable methodology to do it.
The strategy I use its to scope out as much as you can up front. A list of high-level user stories. Give these a rough prioritization (MoSCoW works) and some rough estimates on each. Now estimate your velocity with a few possible team configurations. Also, assume your backlog will grow about 10% as you go when new stuff is uncovered.
Now if you need to schedule a launch or set a budget, set it deep into the non-mandatory features. If everything goes off the rails, you have cushion to avert failure. If everything goes ok, you will deliver a richer product. You'll also be able to track very accurately as you go how close you are to the plan week by week.
If a problem is less complex or can be released iteratively, than that's the lean way to do it, where you also have good learning. But often to solve the problem just a bit you already need a load of stuff to be taken care of.
Key to me is to stay in text or cheap click-dummies for long enough. Depending on the complexity I go through several stages:
generally:
- Always probe for details if you can imagine some already, you are trying to know as much as possible as soon as possible. File it all in to a "to take care of later"-list at least, better yet sort in properly already.
- write down everything (maybe others can remember everything, fine too :) )
- change and refine whenever something new has to be taken care of. IN the following steps it will always be easier now, than in the next step.
1. gather high level requirements with the stakeholders
2. sketch a rough workflow. I usually do a nested list.
3. write down a complete workflow
4. now you might know what you need, so define a rough UI, technology, interfaces
5. still in text: write your concept so someone else understands it
6. talk everyone involved through the concept (first stakeholders, than devs)
7. double-check if you cant simplify or leave out anything, at least for a first version
8. if necessary: do mockups, designs, schemas
9. only now start to program (for difficult stuff a prototype)
- On top it might be helpful to have a small checklist depending on your needs with entries like "reporting?, testing?, support?"
This is reasonable, it's human, but does anyone have a good approach to applying an iterative development approach on fixed price contracts?
I've been on many fixed price projects that are "agile" in name only, general issues I've observed:
- Iteration on requirements becomes confrontational (pay for change request) making it difficult to build a good product as we all learn what does/doesn't work for users throughout development.
- Upfront estimate is inaccurate causing time pressure on development resulting in rushed work which negatively impacts code quality and team learning.
The traditional answer is to have the client commit to specific requirements and hold them to it.
But what I'd really like to figure out is a way to acknowledge evolution of requirements will happen so we can work _with_ clients to build great products.
I struggle because this seems incompatible with fixed price and large companies seem to only want to do fixed price.
Whether the contract was time & materials (preferred) or fixed bid, that 25% rule worked well as an early indicator that the project was likely to go over budget. It allowed us to have early conversations with the client about cutting scope or expanding the budget to cover the unknown complexity.
We'd also dramatically increase our rate to reduce our risk on fixed price projects.
[0] Barebones, ugly, but functional from end-to-end.
This is highly context dependent. In some domains the business basically just needs to throw random things at the wall and see what sticks because nobody can know what they really "need" until it's tested in front of a customer. In other domains they have an incredibly detailed view of the behavior that they need.
In some businesses they're in a weird situation of just not being very good at figuring out what they need and an improved process would save tons of time and money.
In others nobody even thinks about any of this because their requirements are so simple and obvious nobody needs to.
Iterative development is a lot of the time but it's not a panacea and it's not a replacement for fixing a broken requirements process when it's needed.
by -> but
Misguided downvote, imho. (shrug)
You make a minimal cost mockup ASAP, for the client to try out, to see if it's what they wanted. Clients can't appreciate requirements, so (like, actual) architects make a scale model first.
The alternative, of doing requirements as a separate phase, was ridiculed as the "waterfall model". In reality, there's interactions between the so-called phases of req, spec, design, code, test, maintain etc.
The truth is that understanding the problem is most of the work - not just for the programming problem, but for the business problem. It's just difficult. And when the world changes, you just have to change with it. If you try to anticipate what's next, you'll invariably get it wrong.
Because the world is changing faster, software develoment has gone from beautifully engineered software that just keeps working, to slapped together solutions, and a fulltime team that runs alongside, slapping patches on patches, continuously.
What's the point of spending the time and money on beautiful engineering, if it's going to be scrapped tomorrow anyway (or even before it's finished)?
The only hope is for tools, libraries, frameworks and languages, that address lower levels of abstraction, that change less frequently. This isn't all good, but the JVM is one example.
A so-called FDD - frustration driven development.
Good luck maintaining that code. Don't forget about the team, a messy codebase and/or poor requirements, eventually, will break a team's morale until the day news devs come along and demand a refactor.
I start to love the idea of a waterfall model for software. People can still iterate on the problem with wireframes, even high fidelity ones, and speaking with customers, market research, etc.
There is a lot of value, time and money saved when you have a decent level of accuracy in the specification.
Honestly, in these cases, there's nothing better than waterfall, partly to save development time, but mostly for contractual protection: If a small change can cost millions more (and from experience, they can), you need to know who is responsible for paying those millions...
Edit: sibling is right as well, it depends on what you're building. Sometimes there is nothing better than waterfall.
* There's a ridiculous amount of code that either gets tossed or goes almost totally unusued and time spent on beautiful engineering on that is 100% waste. This is, I think, why languages with incredibly strict type systems that try to 'force' good design up front never end up being widely deployed. They make prototyping way too expensive.
* Only juniors demand to rewrite or refactor anything. Being a really good developer means being able to figure out creative ways to fix the most enormous technical debt incrementally and knowing that you shouldn't ever have to 'demand' to do that because you can just do it.
* Pretty much every project I've ever seen that does anything useful starts out with nasty code because the programmers who worry about making it beautiful before it's useful never make it useful to begin with.
* People who dream up architecture before writing the code dream up the worst architectures. The best architectures, on the other hand, happens as a result of a lot of time spent tediously refactoring code.
It's purpose is to capture design-rationale and bring it to life so that as you refine your reasoning, the changes ripple through your decisions. It can help communicating complicated design decisions that can be awkward to capture in prose.
It's intended to be very low-friction so more like markdown than old requirements management or decision support systems. My dream is that it can help us tackle decisions so complicated we usually give-up e.g. when multiple decisions impact each other.
I am currently in beta and I'd love to talk to anyone interested in the topic. Please drop me a line at duncan at thorny.io.
If beta-users find the web-app useful then I can release native apps but I must resist until its proven!
The stage I'm at is that it is experimental and I need to iterate features/usability to make it useful before going native. It has been written offline-first with tech suitable for shipping cross-platform native apps.
I almost have guilt over feeling like my work of design and scoping is effectively useless to a developer, all my mockup layouts have to be built ground-up, my requirements aren't actionable in any way unless they feel like reading them (I try to keep them as succinct as possible, but the nature of working for clients also means I have to be somewhat specific so that people know when they should actually pay us). I've looked into things like Cucumber (https://cucumber.io/) so that my requirements can actually be compiled as tests, but adoption is slow and arduous, and all I'm really doing is adding more work to a dev.
My latest line of thinking is that I need a way to show the user interface, and then the data flow and logic all the way back through the system (usually a back-end DB or a customer legacy system). It's vital that these are presented together, hence my current process is interactive mock-ups built in Sketch (https://sketchapp.com/) and hosted on Invision (https://www.invisionapp.com/) which allows the customer and developer to click around and see it on a mobile screen so they really get a feel for it. Finally I couple that with a BPMN diagram which has swim lanes not just for the traditional system swim lanes, but also for a user (i.e. User taps Submit) and for a user interface (i.e. shows the mockup screen that is displayed), and then the logic flows down through the diagram. (e.g. User, User Interface, Mobile App Logic, Server Logic, Server DB, etc.)
Shared understanding (passing what you've learnt on behalf of the rest of the team doesn't count) can help everyone, (including the customer!) understand the whys, whats and hows of their own problem space. Then as a whole team you can come up with and vet the solution - qualified by the deeper understanding everyone in the team has.
This deeper understanding will help you arrive at a better solution, wasting less time building the wrong thing and reduce the friction in "handing-off" to development (because there isn't a hand off).
I interviewed a bunch of firms for building a website last year; nothing particularly fancy. Several of them (at least the big firms) send in a guy who would always start off explaining their 'process' (all fancy sounding), that process essentially being 'you tell me what your problem is, then we will together design a solution, and then I'll hand you off to our project manager back at the office who will just have the programmers implement it; you'll never even have to see these guys face to face!'. Uh sure, probably to 95% of your customers, naive and gullible because of lack of experience, that sounds great and like it's an advantage, but no way I'm going to get caught with my pants down 6 months from now because there was some aspect we didn't cover in the 'design' but the programmers coded it up like that anyway because hey it say so in the spec, right?
Yes, this. It's what I've wanted as a software engineer and as a product manager. A unified overview.
I find the project materials become siloed, and cross-referencing is time-consuming, error prone, and not done enough. There is often a written spec, and a separate set of wireframes, with clickable areas to show interactions. Those two need to be merged together. Then a bunch of JIRA tickets, again separate. These need to be associated with the wireframes and the spec.
I have yet to find any system that makes this work.
It’s a good way to get stuff out of folks heads, validate the problem and solution with users, and end up with some stuff for everyone to execute on by the end of the process. You can google the term for more detailed descriptions.
As for tools, we’ve been having success with real time board, and pivotal tracker (maybe gives away where I work) and of course a ton of sticky notes.
https://pivotal.io/office-hours
(Free, staffed by a balanced team over a lunch hour)
I would be happy to give you a quick demo!
Add more screenshots and less "integrates with slack".
"I'll not book..."? I guess you meant "I'll book..."? =)
edit: looks like you have an admirer https://thorny.io/
In short: if you have good people it shouldn't matter! Experienced people will extend the assumed scope beyond the stated requirements without being asked, and either do so efficiently within the resources available to the project or bring an appropriate level of attention to the limitation before it becomes an issue.
The somewhat less PC and far snarkier explanation, in the immortal words of twitter: Don't pick the right tool for the job, pick the right framework which secures you extra work for being a tool. - @iamdevloper (via http://github.com/globalcitizen/taoup)
Use UML, SysML, etc to diagram out your requirements (starting at high level at first, more granular as the system matures). Now build a high-level model of your system design (software and hardware if required). Match parts of your system to the requirements. This lets you see gaps in two directions: requirements you haven't addressed (because they aren't connected as "satisfied by" anywhere on your model), and questions to ask for more detail on a given requirement that could drive the design.
As others have said, there will always be late requirements that change things, so it won't resolve those. It can, however, show stakeholders how much changing a requirement can cost in terms of rework/time by showing how the model has to change to accommodate.
The caveat with this is: the model needs to reflect your actual design, and you need to keep it up to date. The times I've used it so far it has been a useful exercise.
DDD, ubiquitous language and bounded contexts in particular, can be enormously helpful with defining better requirements.
I'm not so sure about BDD though in this context. While the notion of the customer / product owner writing specifications in this format in a way developers can use these specifications to test their code sounds great at face value I have yet to see a project where this is done consistently and continuously.
Moreover, some types of requirements can be better explained by using diagrams or UI mockups, which doesn't really fit the BDD paradigm.
To address the specific question, I think most of the time there's a problem because there is no one that takes care of making sure everyone has the same vision and understanding of what needs to be done. And because sometimes words fail us, I think mockups are a great way of uncovering and sharing requirements.
Engaging end users is a problem from a couple of directions. First, the very idea of "end user" - you need to engage customers, not end users. (Take Facebook, for example... you're not the customer, you're the product. If you build something for Facebook, your customer is Facebook, not Facebook's users.) This situation isn't exactly unusual. In this case, it's the customer's responsibility to gather feedback from end users - and put you in the feedback loop, as needed.
From an logical point of view I think it makes a lot of sense to start from clearly documented requirements, then work forward to design, implementation, and testing with links back to each requirement. I don't know how testing (higher level than unit testing) can really be effective unless they have requirements documents to use as their starting point. Use Cases go some of the way to providing this information but tend to be a bit less formal. Agile methodologies tend to be less formal than older methodologies that emphasised this kind of process more. A lot of Agile is good, but it does also tend to forget lessons from the past.
Transferring valid understanding into the final artefact is a key constraint in many projects (reading Goldratt and thinking of this transfer of understanding as a constraint was helpful for me)
There are many ways to fail. Some of the traditional ways to succeed are
i) rely on an individual who really understands the desired tool, and who has the authority and skill to communicate and be the final arbiter (including sometimes to write it all themselves). Sometimes this can be also done by a small group.
ii) write a really clear, well-written, very hard to misinterpret document and get professionals to develop and test this system (before the document goes significantly out of date)
iii) Run an agile process where the business can describe what they want in small pieces that can be delivered so quickly that little understanding is lost
Obviously what works is massive contextual, depending on the domain, funding, resource reqts etc. (Glen Alleman is good on this)
So I would argue 'good software tools for requirements' are critically dependent on your approach for how you are going to turn 'understanding' into a 'system', and you don't want to worry too much about them until you are happy with your approach. At that point you can start building your 'meta-tools'.
One awesome thing you can do is ping everyone interested in a feature, and ask them how they want it to work. See it in action here: https://feedback.canny.io/feature-requests/p/tags. (Scroll down to Sarah's comment on July 20.)
Whenever we're thinking about building a feature, we ping all the stakeholders. This gives us solid context on how they want it to work, which helps us define our MVP. If we need to follow up for more information, it's easy to do that via email / Intercom.
Hope this doesn't come off as salesy – I really felt it was relevant.
That said it is still incredibly helpful for everyone to agree on what we’re building in the present iteration/sprint. You mentioned BDD which can be very helpful when paired with a team practice such as Example Mapping[0].
Other practices such as DDD can help you bound the problem and define it. Also there are some helpful lessons in Feature Driven Design.
Source: I teach BDD workshops for a living
[0] https://cucumber.io/blog/2015/12/08/example-mapping-introduc...
The fit problem I'm having is getting technical threat hunters who populate the market to think about risk and security design, or product/project managers to reason even high level about a domain they delegate to technologists.
Still iterating for fit, but so glad there is a thread of people thinking about architecture.
http://hitchdev.com/hitchstory
The time when it was most useful as a "BDD tool" was when I was working with an extremely technical stakeholder who was proposing behavior in a command line tool he wanted.
I simply wrote 'tests' that described the command line tool's behavior and showed the YAML to him. He corrected the mistakes I'd made by misinterpreting his original requirements and then I built the tool to that spec and when it passed I was confident I'd nailed his requirements.
QA picked up bugs afterwards but they were all either (quickly rectified) mistakes he'd made in the spec or environment issues. I had zero spec<->programmer communication issues even though (and here's the crazy part) the domain was very complex and I didn't understand it. It had to do with a highly customized software system I didn't understand which enacted some sort of financial process which I also didn't understand.
Cucumber can do this in theory, but in practice the spec is not sufficiently expressive and the stories end up being ridiculously, unusably vague. Unit tests could also do this in theory I guess, but good fucking luck getting a stakeholder to read them even if you do manage to write them "readably".
I'm taking this process a step further. Although these YAML specifications were useful for me in the past to collaborate with stakeholders they're still not amazingly readable. For instance, the "YAML inheritance" makes it easy for programmers to maintain but harder for non-technical stakeholders to understand.
Instead of sacrificing maintainability for readability I created a process to easily generate readable stakeholder documentation from the 'YAML tests'. I used this in the libraries on the website above to generate always-in-sync-and-still-highly-readable documentation.
I think this could be used to essentially write low level "unit" tests which generate documentation which stakeholders can interpret (e.g. defining the behavior of a complex pricing algorithm) and use that to get a quicker turnaround on understanding, defining and modifying desired behavior in complex domains.
With most programs, you'll spend half your time battling its UI and trying to get around limitations.
I would argue that the reason things get called out as poorly defined or change is because risks are not addressed early, and hypotheses are assumed to be theorems.
Make sure your teams test the main assumptions early, with actual code if at all possible. That will call out why your stories aren't clear enough.
Tools are useful, but they won't solve your problem.
Bonus point for the interactivity: I can share the URL for remote work sessions, and the stakeholders on the other end of the line can see me create/adapt wireframes and notes in real time, while I'm explaining everything on the phone (using anonymous access for those not logged into 'The Google').
Google Docs has version management since this year (I think), so I can pinpoint the evolution of specifications over the time. When I'm done I'll just export a PDF and attach to a "Spec done!" mail to everyone.
Doing requirements gathering isn't an inherently bad process to adopt even in non-regulated settings. However I believe many developers will have a negative reaction due to prior experiences with, or having heard about, the waterfall method. The key to remember is that requirements don't, and shouldn't, have to come with an estimate. Great requirements demonstrate a thorough understanding of the problem. They're written from the perspective of the end-user.
Specifications are the dual of requirements and are what drive the implementation that solves the stated problem. Specifications is something I think we need to get better at. We some unit testing, some integration tests, and occasionally end-to-end user tests... some few to property tests; but rarely do we write formal, verifiable specifications in a modelling language that can be checked with a model checker. Rarer still do we write proofs as part of our specifications.
We often elide specifications or we do the minimum necessary to assure correctness. This is to the detriment of your team on a larger project. How do you know your designs don't contain flaws that could have been avoided? How much time are you spending before you realize that the locking mechanism you chose and shipped has a serious error in it that is causing your SLO's to slip?
For requirements I just use plain old Markdown with pandoc. For specifications I use a mixture of Markdown and TLA+. I use TLA+ for the hard parts of the system that we're unsure about and require correctness for. The rest of the specifications that aren't as interesting I simply use prose. It's a matter of taste but it does require an intuition for abstraction... when to do it and how to think about systems abstractly.
We could definitely use better tools for TLA+-style specifications, btw. Maye more libraries that can translate TLA+ state machines into the host language so that we can drive them from our integration tests, etc. Better IDE tooling. Better training.
then as the document is discussed and altered by the owner the use cases alter and flow
It's just trying to keep a prose discussion in line with everything else in dev friendly manner (it can be stored in the docs folder in git and be generated onwards etc)
still playing
I like the approach to embed requirements in source code (management).
Maybe you could engage customers qualitatively, and translate that information into software requirements and acceptance tests? I'm not entirely clear how you want to engage these kinds of users.
I believe if you can get the small team scenario working over-and-over again scaling will work itself out. So far I have no reason to believe this isn't the case -- and I've applied the principles in the book both to functional coding and program management.
mjul's comment is the key one: you can't know everything before you start. That doesn't mean you can't know anything. It means that there is a "progressive elaboration" that has to happen on an as-needed basis. Otherwise you're stuck either not knowing enough to get going -- or having created a monster or a tools/information system that ends up running the project instead of the team.
There are some sanity checks for whatever backlog/requirements system you are using. Instead of my continuing to pitch, I'll just list the things that your system should do no matter what kind of system you have.
- Handle whatever detail is needed before actual development happens.
- Be able to "flip-around" and start from scratch within an hour or so. (And "starting from scratch" means beginning with nothing and ending with the team starting coding) While keeping all that detail in test 1.
- Be reusable with other teams doing similar work. A backlog/requirements system can't make work fungible, but it can enable better conversations in other teams without being a burden.
- Limit meetings around organization to under an hour or so. Yep, you gotta have those "meta" meetings from time-to-time and talk about things like release plans. But they shouldn't take over your afternoon.
- Help the larger organization (if there is one) learn and grow over time. Orgs learn from the bottom-up. Good tools should facilitate this learning.
- Drive directly to acceptance tests. Good backlogs are testable. Things shouldn't be in there that don't drive tests.
- Have controls in place to prevent abuse. As soon as you create some tool for the requirements/scoping phase, somebody is going to go all architecture-astronaut and overuse it. There has to be controls to prevent this from happening. The tool should facilitate the work of understanding and scoping, not replace it. (I think even a lot of tools vendors get confused on this one).
I even wrote an analysis compiler that demonstrates all of this as part of writing the book. So if you think all of this is impossible -- happy to do a demo.
There are a few other testable criteria for whatever your requirements/backlog system is, but that should be enough to get you started deciding whether some system is better or worse than another system.
