I have been trying to create a command line tool to categorise my expenses and show a total by category. I’ve been using this exercise to practice TDD.
In my latest frustrated attempt, my first test was along the lines of “it can categorise a single transaction correctly”. The problem with this is that this is testing the “tip of the iceberg” that will be almost the whole, final app. I think I was attempting a more “top down” design strategy here.
Before that one, I tried the opposite: a more “bottom up” approach. I know I need to, at least, read a CSV, parse a CSV row into some data structure and then categorise it. So I TDDed my way through to complete these “sub problems”. However, when I was about to apply the categorisation rules I realised that the data structure I created didn’t help, was just bad and didn’t work at all :).
So, how do you do it when you have a “larger project” (here, “larger” means something that’s not your typical “2/3 points user story” at work)? I always end up feeling stuck whether I try bottom-up or top-down.
Sometimes that's building a full app and getting it to market. Sometimes that's isolating a bug in a legacy system to a unit test. Sometimes its the compiler failing to build your code. Sometimes it's a data science problem, and it means training a model in a notebook, to measure its accuracy.
Don't get hung up on using one particular feedback loop.
Getting stuck in the wrong feedback loop is a big anti pattern of software development. When it takes forever to submit a job to some system, then takes 20 minutes to give you a type error in a scripting language, that could be caught by a linter or type checker sooner. Those 20 minutes easily turn into 60 minutes of distraction. Imagine instead you could catch this in 5 seconds with a test / linter / whatever. Then you'd resolve the issue in minutes. OTOH spending hours decomposing simple code into unit tests can be a waste of time, when the most value is submitting that simple code to a batch processing system, to see if it produces output...
In my experience I get those benefits when testing at the system/use case level but not when doing unit tests. At that higher level automated tests works wonderfully. I haven’t had a bug in production for 5+ years because of test automation. Which means that I spend 95% or more of my time adding new features or optimisations instead of bug hunting and/or getting shouted at because of customer problems. Love it!
To go from a high level requirement like "command line application to track transactions and show expenses by category", you need to go through several levels of detailed design before you have anything that is implementable by a test-driven approach, which will necessarily be bottom-up.
Remember that tests can only confirm the presence of a defect, not its absence, and that not everything can be tested due to the intractability of exhaustive testing.
By blindly following test-driven design, you could end up with a useless, inflexible program that correctly categorizes expenses over only a small, fixed vocabulary of categories because it never occurred to anyone that a category must be something user-defined.
A user-defined category feature cannot be implemented with strict TDD because the space of user-defined categories is infinite. Say you have a test which confirms operation for every valid category string up to three characters long. That doesn't prove the system will not fail for a four-character-long category.
The only people who regard TDD as some kind of panacea are those with no formal background of any kind, who don't understand the role of testing in the overall software engineering context, and what its limitations are.
As a programmer with a problem, you first instinct is to start at the solution.
TDD pulls you back, and you first have to write the api and decide how you verify it.
If you're doing it well, you make both of those things as simple as possible, first. Reduce dependencies and inputs, etc.
That's actually the most important part, an easily consumable api that's easy to verify, not the implementation, and TDD forces you to do it first.
Also, a pet peeve of mine is seeing a line or more of code that doesn't do anything. With TDD, such detritus is impossible as you can't write production code unless it's making a red test turn green.
TDD has refactoring steps which only have to preserve passing tests; refactoring can easly be the vector that introduces dead code as well as changes behavior for untested input combinations.
I suspect that a lot of code developed TDD is actually deployed on input combinations that are not covered in the TDD test suite.
A string length function developed by TDD will still work on len("supercalifragilisticexpealidocious") even though that exact string never appeared as a test case, and the consumers of that function will use it on all sorts of untested inputs all the time.
If one then goes and tries to write tests for these kind of methods, it feels like superfluous busywork.
TDD makes you define what your input is, and what your output is. What's in the middle your free to choose.
Write examples of how you would like to be able to solve common cases and handle edge cases if you had the ideal interface based on what you know at the time, then use unit tests with stubbed out interfaces as a tool while trying to develop the implementation.
Refine and even rewrite your examples as you learn about what works and what doesn't work.
Recursively break down complex pieces into smaller modules as you learn what can be handled independently and composed with the other pieces. Break off independent examples depicting how you expect those modules to work and be used as new unit test suites for those new modules.
Don't sweat testing the implementation details, especially when prototyping. What you want is to use tests as a way to speed up your development process and improve the quality of your design and architecture.
Try to write tests that show off how nice your code is to use and how few edge cases you need to handle when calling it. It'll raise the bar for how your code interacts with the rest of the application or system and give you smaller pieces to work and focus on.
Finally, for really green field stuff with a lot of unknown unknowns, you often have to write a lot of standalone, bottom-up throw-away prototypes for things. That doesn't invalidate the usefulness of tests, so don't be afraid to do that pretty regularly.
Particularly with exploratory projects, I'd recommend focusing on rapidly iterating to reach the simplest solution (businesswise and implementation-wise).
If in the end non-trivial logic must be specified, write some tests. Your definition of 'non-trivial' will change with experience.
But tests aren't inherently good. Good tests will speed you up, bad ones will slow you down. And it is incredibly hard to write good tests. Might be even harder than writing software that tests supposed to test:
If software works in 80% of cases, it creates 80% of value.
If tests work in 80% of cases, they cause harm in the remaining 20%.
Want to extract some logic into a new interface or introduce a new parameter to a method?
Well, now instead of just doing that, you will also have to update all tests dealing with said logic.
At my previous job I spent much more time updating tests testing trivial shit than actually writing useful code.
Most tests written by people working on the product will of some baseline level of quality and even if they weren't, even if they were "bad tests". It is much easier to fix bad test than it is to deal with an application on a day to day basis which has no tests.
The answer depends on your approach. You can go outside-in (top-down) or inside-out (bottom-up). The one you use is a matter of taste. A lot of people find outside-in easier, but inside-out can be a bit faster if you have a good sense of how your system will fit together.
To take an outside-in approach, start at the top level of your application. Possible just under the UI, because UIs can be hard to test. (If the UI isn’t tested, make it do as little work as possible, and have it just forward to the tested code to do real work.)
Test-drive a module, class, or method in that top layer. The smaller the better. If there’s something complicated that should be handled by another module/class/function/method, make the module/function/etc., but just hardcode one answer. Make a note of the things you hardcode.
When you’ve finished TDDing your first thing, look at your notes and choose something that you hardcoded. TDD that in the same way. Repeat recursively until the application works. Focus on getting to a working but incomplete application as soon as possible, by working depth-first instead of breadth-first.
Once you have a working but incomplete application—a “walking skeleton”—think of a feature you’re missing and add it in the same way. Continue until the application is done. Remember to look for opportunities to improve the design and refactor as you go.
Inside-out is very similar, except that you do the depth-first recursion in your head, mentally, and don’t write any code until you get to the leaves of the recursion tree. You test-drive the leaf, then test-drive its caller, etc., until you get to the top. It takes more experience, more thinking about design during TDD, and more willingness to refactor.
I have videos on my site if you’d like more. Start with this series:
TDD encourages a form of programming where you build from the outside-in, building layer after layer of abstraction, putting off solving the actual problem until you get to the messy gooey centre where it ends up being a kludge. It's also very easy to make flawed assumptions in your original TDD spec which you don't realize until you get to that gooey centre, having wasted N days in the process.
I honestly don't tend to start anything until I have a basic idea of how it's going to work (not a fashionable idea these days), and find that projects end up with a much saner structure when they're built from the inside out.
So my preference is to start TDD once I've got a very basic version of something working, at which point you can start diving in to all the corner cases.
In TDD, all production code must be written against some pre-existing test. Any code which does not meet this criterion is broken.
A story: I used to work with some devs, most of them would finish coding something in 2 weeks and then spend a week debugging it. I'd take 4 weeks to TDD something. My boss was frustrated at my speed until one day a production system which I'd written, stopped working. Several million dollars was stuck in limbo, the backend team responsible for the paperwork unable to do anything. We had a new dev and the bandwagon was "TDD is crap!" was fully rolling that morning. I asked the backend processing person what had changed? The said we had a new client, they had a weird name. Looked into the client log, sure enough. Added a test with the client's name, and ran the tests. It spat out "1 failing test in module X at line Y because 'this'". Modified the code to handle that test, re-ran the tests, 100% pass. Emailed the results output to the team for a manual upload, processing done, committed and pushed to the production branch. The server would update after business hours. Diagnosed and fixed in 5 minutes. I turned to my colleague who I knew was still stepping through his code for 2 days and said (within earshot of the new dev) "How's that bug you've been hunting? Still not found it? This is the power of TDD. I might be slow at the start, but I fix bugs quickly". The dev manager had everyone take a day of TDD training after that, but nobody really understood. The trainer said my approach to TDD is too strict, with my team doing 110 test runs against the next team of 63 test runs. Our solution had only a few lines of code verses the paragraphs of the others though.
This is the material piece of information which lead to the fix. Your TDD simply made it faster to locate the offending line.
Seems like a really poor justification for TDD imo.
The stack trace tells you where the error is occurring, your intuition leads you to the relevant caller function that you should test, stack trace or logs should lead you to the offending parameters (i.e. the weird name).
With these three pieces of information you can write a failing test and implement a patch.
I don't consider this approach to be "TDD", just "software development with good test coverage".
> The dev manager had everyone take a day of TDD training after that, but nobody really understood
That is exactly the problem with TDD or any other methodology that is hard to get right. Methodology will either push to a pit of success, or a pit of despair. If it's hard to get right, pit of despair it is.
Force 100 people to do TDD. 95 of them will be harmed, 5 will succeed. Same if you do waterfall. Same if you do Visual Basic. Same if you don't do code reviews.
TDD is indistinguishable from any objectively malicious practice. Somehow the fact that few folks were able survive TDD, is twisted to present TDD as some kind of holy grail.
Sounds right to me! I usually start projects/features with a test that covers the end-to-end behavior I want to see. When you have no other tests, a smoke test is the highest value test you can add because it measures the thing you care about.
Most experience developers will come up with the architecture/design of what they are working on through a combination of intuition and experience, write tests after and then use TDD to fill the gaps once they have nailed down the architecture.
I do TDD some of the time and my workflow often look like this:
1. Start implementing an object/class, defining APIs of its key methods while having a higher level picture in my head of how it will fit with other components
2. Implement a good chunk if not all of that object (not more than a few 100 lines typically)
3. Write a test to verify that what I did worked.
4. Start commenting out parts of the code I wrote to make sure the applicable test cases fail. Uncomment after verification is done.
5. Use TDD to cover and implement edge case handling and anything leftover
6. Repeat 1-5 for each class/component/module until done
I only get into TDD once the architecture and high level design are finalized, and I have some code written for them and I have some reasonable level of confidence that it is not gonna drastically change. I also generally go "bottom-up" starting from the leafs of the object graph up and use TDD maybe ~50% of the time (just a rough guess).
Trying to force yourself to let "TDD drive the design" will likely result in frustration as you have experienced and slowdown.
The tests usually run lightning quick because the units will be small.
When bugs occur in your codebase it will be at the right level of abstraction to write tests for the bug.
I almost never do strict TDD, I hack away to get something going and use the tests to refactor the working code into clean srp code. The end result is the same. However where I would use strict TDD is when there is massive complexity (like creating a video encoder) or if you are following a published specification.
So ‘just barely working, green, refactor, else red, refactor, green’ is probably closer to my actual day-to-day. The working code is clean, the tests are clean, and the coverage is good. The order doesn't matter.
In a DAW(Digital Audio Workstation), this is like adjusting the knobs, adding effects when you haven't even added any samples, it just doesn't make any sense, stop forcing yourself to do things that doesn't align with your flow, you'll likely get annoyed for no reason.
The way I work is I do the architecture design, I write the code, then I write my test by following the design graph.
If I don't know what the design would look like, I do what I call "code freestyling" to get a basic idea of what the overall design would look like then I repeat the above step.
C4 modeling would give you the abstract design or system architecture graph, you don't have to do all 4 levels diagram, just do the one that makes sense to you.
If you are doing something simple, you might not need to do any of that but as soon as you are building something of scale with optimal security not doing system designing is like building a skyscraper with no design, well, you know what might go wrong :)
1. Talk through the requirements and write down some key bullets
2. Whiteboard some design options. Make sure you have at least 2 interesting ones.
3. Prototype them out. Decide on your favorite.
4. Delete the prototype.
5. Start writing tests now that you know where you want to be, and TDD from there.
The key point is that TDD has to start from a relatively defined sense of what you want. If you don’t have that it’s totally fine explore. Just don’t ship that code.
So what I do is that I first hack out the application before I write any tests. E.g if I code a matrix multiplication routine I'm not going to waste time writing unit tests for it. Cause perhaps a few hours later I might realize that I don't need matrix multiplication after all. An added benefit is that I don't have to constantly "context switch" between test writing mode and implementation writing mode.
I write tests when I have something interesting to test. So if the application unexpectedly crashes when I run it I write a test that triggers the crash. Then I fix the bug which causes the test to pass. I find these types of tests to be much more valuable than the "cookie-cutter" unit tests TDD expects you to write.
What DOES work for me is having a rule that any implementation should be accompanied by tests that prove the implementation works, bundled in the same commit.
I've been doing this habitually for a few years now and it has had a dramatic positive impact on my productivity.
I wrote a bit about how I do this here: https://simonwillison.net/2020/Feb/11/cheating-at-unit-tests...
I’ve found it safer to start with (manual) integration tests where I verify intended behavior from outside in. That avoids building inner layers which turn out to be slightly but annoyingly off compared to what is desired.
Unfortunately, as soon as you jump off the pure OO bits and domain logic, the TDD-ness of your end-to-end program could be all over the place. You’re entirely at the mercy of available tools and your proficiency with them, and the time you can invest in learning those tools.
Now I have a sixth sense for what I should write tests for and when I should write them before vs after the implementation.
Rules are misleading and teach the wrong thing. Instead, here’s one loose question I ask myself: “does the interface inform the implementation or the other way around?”
When the interface is super important and known, I can write a test first. When the implementation will reveal what the interface’s shape might be like, the test might come after.
It defines the information you need to send in and/or receive.
Non functional. Then capture some basic behavior of one of the methods/properties in the tests, then implement the solution. Then start capturing edge-cases the same way.
If you’re asking yourself “how” or “should” I test this, then you’re doing great.
Apply SOLID principles to separate meaningful concerns into separate classes; and iterate there.
Take my viewpoint with a large grain of salt.
How I TDD 1. create a file with a test. 2. Get it running continuously (dotnet watch test or mocha --watch) 3. Write the code in the same file (just to start the design) as the test so I don't have to bounce around 4. Iterate at step 3 for a while to explore the core domain/model/data (in your example, a Transaction has an amount, a payee, and time for instance), just to reach "escape velocity" 5. Ruthlessly push out dependencies that cross boundaries (file system, network, etc), protecting the core 6. Practice Red-Green-Refactor technique (write a failing/not compiling test, make it pass, refactor) 7. Red-Green phase, I just make things work 8. Refactor phase, I design, reflect on object/function communication patterns
Designing architecture is not easy. And keeping it simple, even less. So I've spent time learning about design patterns, boundaries, and communication patterns and implications to improve my design skills.
My personal experience is that i start TDD once my project is "mature" enough that the next upgrade/addition can be put at two different place easily enough and i don't know where is the best place to do it. Basically, on my latest project, i asked myself something like: "is this part of this class method, or should the main loop take care of it?". That's where i started completing tests to 90% coverage (main loop + pagination mocks were skipped, ideally i could've reach 95% but laziness caught up) and started TDD. In the end, i still made the wrong choice and added my change to the class method, only to change it back two weeks ago (or roughly six month later). Here, TDD didn't help me to make the best software architecture choice, but having to write test first for this part of code made me decouple it from the method i originally put it in, and allowed me to made a architecture change in less than an hour.
Another piece of general advise on TDD: Adjust what tests and how many you write based on what your language already gives you. E.g. when I write Ruby or JS, I'll write a ton of unit tests for almost Avery conceivable scenario. When writing Rust or Swift I write mostly unit tests for interesting business logs, but not much else.
If it doesn't fit your mental model of programming, just don't use it.
TDD is kind of a forcing function for modular design (whether OOP or functional programming).
In your scenario, you'd want:
1) a module to read the CSV -> test that it can load a file correctly given a path string, correctly handles invalid paths (format, file does not exist, etc)
2) a module to parse a row -> given a string, test that it can parse a 0 row file, a 1 row file, a 2 row file, and correctly handles an unparseable file (e.g. binary file); test that it correctly handles incorrectly formatted rows
3) a module to categorize -> given a set of rows, test that it can categorize a sample set correctly with expected output
If you try to make 2 depend on 1 or 3 depend on 2 and 1, it's no longer a unit test and becomes more of an integration test.
Take that first module, the CSV loader… if you build it in a truly modular way, it can be hard to stop with the features that just support your end user application.
I mean, it's not hard, you just choose to stop.
All it really takes is an awareness that you're programming to solve a specific limited defined task, not programming just for fun.
Of course, if you're doing it for fun, then fine, don't stop.
But nothing about stopping is hard. It's just a choice you have.
http://www.dubberly.com/articles/managing-complex-design-pro...
Now what does work is if you can decompose the whole idea into parts, each of which is written using TDD. You might still want to outline some key interfaces so you get the shapes of things to fit. With practice, all this extra stuff often happens in your head as you're doing TDD and you make adjustments along the way.
Also, the way you described making bottom-up things that didn't quite work, is a valid and possibly effective way of doing TDD. If you're able to determine that a lower implementation is unsuitable it can be rewritten and if the tests are not too brittle will give you confidence to do the rewrite safely and faster.
I think what would have worked for you is to write a test for the categorization, but feed it pre-parsed data structures in the tests. If the data structure is wrong, you will find out now. After that works, you have decided on a correct data structure, and you now can write code to read and parse CSV into those same data structures.
It’s all a matter of practice. If you do things in the incorrect order, these things can happen.
Then you should find some sort of coverage tool so you can know if you covered most lines or not. I tend to ignore error handling lines but if its easy to get into that case I'll writ a test for it. Then I'll write test that take <1ms to execute. If something needs a network I'll try to separate the logic so it can work on a file OR I'll it connect to something on my local machine.
Generally I try to get 95%+ coverage (including error cases). I have many test and the entire suite run in <100ms. Maybe my project is easier to do TDD on but that's the way I do it
Don't test private functions. Only test the public interface.
Use dependency injection everywhere.
a) TDD primarily refers to unit tests (it does not address other types of testing that you need to take into account when designing a program)
b) you have the wrong idea of what constitutes a 'unit' test versus other kinds of testing. Specifically, as per M. Feathers, anything that touches the filesystem, e.g. to read a file or database (as you do here), is not a unit test.
I would highly recommend Chapter 2 of M. Feathers' "Working effectively with Legacy Code" on this topic.
Now the only problem I have with TDD is time. It takes a lot of time to do it right and usually I don't have that time (in my organisation). So maybe you can tdd some parts of your application, maybe the most critical parts. Even for personal projects you may not have the time: demotivation usually kicks in for personal projects. TDD defers results and the positive feedback needed to keep going. At least in my experience
As long as all the restful resources work properly and get an expected outcome, there's a good chance everything else should work fine too.
However, a big inflection point for me was getting hold of a good/capable IDE (intellij). It removes a lot of the drudgery.
You write test with yet-to-be defined functions. IDE let’s you run the single test (amongst many) with one shortcut. Fails ofc!. Then you use a shortcut to let IDE implement the function (takes you where it needs to be implemented, easy with python modules eg. YMMV). Rinse and repeat.
TDD as it’s supposed to be done according to evangelists doesn’t work. I have the same problem as you every time I try TDD for implementation, and the answer I get is always that I’m doing it wrong somehow. I have better results without TDD in these cases though.
I've found that TDD with a reading of "Testability-Driven Design" was useful for driving adoption of certain architectural patterns that made my software more easily testable. That, in turn, informed how I authored my APIs. And so on.
A practical example is that my team is building a product that has a workflow which scrapes websites and then converts the output to internal models (before more processing occurs). The way we make this TDD is by simply loading static "fixtures" during testing instead of actually reaching out to the website to grab the HTML.
You might make the case "what if the HTML DOM changes at the origin"?
That's not the point of the unit test; the unit test is only testing "given this structure, this is the output I would extract from the HTML". If the HTML structure at the source changes, that's an integration or E2E test.
I couldn't disagree more. Indeed, it's precisely because framework and library black box behavior and upgrades tend to introduce random unforeseeable bugs that TDD is essential.
It's super important to take some known inputs, and after a series of framework/library operations, to check that the outputs are what they're supposed to be, for normal operation and for all the wacko edge cases (zero-length strings, etc.).
A funny pattern I noticed is most TDD debates is that whoever challenges TDD always comes up with concrete examples, and TDD advocates always stay incredibly vague.
There's a good discussion between DHH and Martin Fowler/Kent Beck on TDD: https://martinfowler.com/articles/is-tdd-dead/
You can also look up Ron Jeffries and Peter Norvig attempts at sudoku solvers. Guess who actually solved the problem, and who wrote 5 blogs posts without getting anywhere?
TDD is snake oil.
The majority of developers have less than 5 years experience, and they're teaching new devs. So the skills that we're developed by people who embraced TDD in the early 2000s have been swamped out by new people.
I think the best way learn if you don't have access to a master is probably videos. See if can find kent beck doing some examples. There's subtle knowledge & flows that hard to teach through just text alone.
This is also what I've run into in practice, across numerous systems.
Actually the only cases where I've been able to do TDD effectively (and get >95% test coverage) was when I was in control over the entire codebase and its design from day one, thus being able to enforce whatever modularization was necessary by the tests, otherwise the systems quickly tended to get untestable (at least in regards to being able to start with a test). In addition, I was generally more successful in cases where I was dealing with writing libraries (that are called by other code), as opposed to web applications which need to deal with shuffling around bunches of data.
Let me give you a few examples of what didn't work. Suppose that I'm asked to help out with this pre-existing codebase that's been around for 5 years or so. I look how some request to purchase a product is processed (just a made up example, though along the lines of what I've seen): first you have some server side rendering with the full complexity of PrimeFaces and around 10-20 variables that affect whether certain buttons are visible, possibly there being a modal dialog or redirection in the middle of the process.
Then, you have the view code, which once again contains different methods that interact with services, passing data around and choosing whether to display additional messages or something else. Then, in the service layer, you might have 5-10 services that are involved in retrieving the data necessary for everything, from auditing and e-mail notifications, getting various pieces of information about users/billing/discounts and then checking all of that against what the user actually wants to purchase, perhaps some validation code thrown in along the way.
Validation code which also might need to interact with the database due to incrementally grown business requirements over the years. And from there on out, additional code for returning early in the case of any errors being present, which might need to be further processed for displaying them to the user. Oh, and if the process is successful without any errors, then you might also have some scheduled processes that would be created and would need to be handled. Almost any of these steps could have transaction rollbacks along the way and there might also be a large amount of PrimeFaces/Spring related code and workarounds in there.
And I've seen similar patterns across many projects, almost regardless of technologies used (Java is just a good example): if you give developers the chance to create tightly coupled code, they will jump at it horrifyingly often, especially if they're under time pressure or have a "good enough" mentality. Over time, any sorts of boundaries will dissolve, unless you go for a microservices based approach, but even then you'll still be dealing with "magic" code which will be hard to test (especially if you need some kind of application context to be running during the tests), be it IoC/DI, annotations/decorators, proxies for your code, or even something like ORM interactions.
Speaking of databases, good luck if your code has like 10-20 database calls along the hot path, which you'll need to mock or actually let it hit some database instance, though hopefully one with state that's either reproducible or can be rolled back (both of which will make your CI slow and more error prone). You might go for an in-memory database, but good luck trying to transpose or make Oracle-specific complex queries portable to something like that, given that for many learning JPQL is too much.
In short: there are situations where TDD is simply unsuitable because of how certain systems/domains are architected (I'm yet to see the PrimeFaces layer as something easily testable) and you're better off simply using unit tests where you can, as well as a healthy helping of integration tests (which you may or may not be able to do, depending on how your environments are laid out).
Of course, things might not look so dire for projects that are a bit newer and where you can still alter their structure. For library code, however, personally I'd recommend TDD, especially if you're ever in the need of doing anything security adjacent (e.g. wrapper code around proven security libraries). And for basically all of the cases where you can write mostly functional code with little to no side effects.
Also, here's a not so fun memory: trying to write tests that dealt with loading data from the file system. In Java, you can have a common piece of code for doing that (e.g. Paths.get), though what paths are valid will depend on the actual system and file system that JVM will be running on. In other words, when locally you have Windows and the CI server has a Linux distro or something, your tests will not run consistently. But you don't want to test that bit of code? Better be prepared to add it as an exception to any code coverage quality gate that you've set up. So annoying.
Your program is going to have a flow like this:
$ rcd2-awesome-finance-app september2022.csv
Category Amount
Food $100.00
Clothes $200.00
...
main(string filename)
csv = open(filename)
cat->$ = new dict()
foreach line in csv
transaction = parse(line)
cat = categorize(transaction)
amount = amount(transaction) // may just be t[2] not a function
cat->$[cat] = cat->$[cat].or_default(0) + amount // or other logic for first time seeing it
foreach (cat, amount) in cat->$
print("{cat}\t{amount}\n")
Categorizing is the core, so categorizing is the thing that really needs testing. The rest doesn't even need (substantial) testing anyways, it's a bog standard CSV-data driven tool. Which leads to a point I want to make:
You don't need to do TDD on everything to do TDD. TDD is the "show your steps" of programming (analogous to the high school algebra requirement that you show and name each step in a solution). Many people despise that in their math homework, but it's important for the teacher for the same reason TDD can be important to you, the programmer.
Math teachers (ok, some may, they're poor teachers) don't ask students to show their steps just for grins. They ask it so they can observe the thought process of the student. A student tasked with solving for x with `2x = 4` may come up with `x=2`, but how? Did they divide by 2 or subtract 2 (I've done my share of math tutoring, I have seen students subtract what should've been divided). They got the answer by coincidence. So when they are given another problem where that coincidence doesn't work, they get the wrong answer and in more complex problems how they went wrong is non-obvious ( in 3x=9 => x = 6 it would be glaringly obvious, but with more terms and unobserved steps it's not; it's early the coffee hasn't kicked in yet so I can't think of good example problems to illustrate this).
TDD does the same thing, but it's there for you, not an instructor, to observe what you are thinking along the way. Each test is supposed to be small so that you can easily see that the logic is correct. However, if you can see a larger step to take (like that structure I threw up at the top or parts of it) then go for it. I've written I don't know how many CLI apps and I would never use TDD for anything but that inner categorization logic (or its equivalent for them). It would be silly for me to write a test to see that I was reading the file correctly. I can think of only one reason to do it: You don't know the language and API yet so you want to test your own knowledge, more than the program itself, to ensure you're calling it correctly. Even then, unless it has a weird file reading API I wouldn't bother, 99% of them will be the same or similar enough.