How Not to Encrypt a File – Courtesy of Microsoft (opens in new tab)

I... sort of have mixed feelings on this.

I agree that the article could do far more to explain what's good, both in content (talking about why these things are bad) and in style (defining all terms immediately).

But holy shit, the MSDN article is bad. It's so hideously bad that I think there's nontrivial social value in bashing it extensively to discourage people from writing docs like this without getting them sanity-checked.

In short, I think this article is largely useless to people reading guides and trying to avoid the pitfalls of the original source, but is aimed at people writing crypto guides who have no business doing so.

Explaining to you how to do it the right way is not an obligation of anyone that says X article is wrong?

"This article on global warming could spend less time bashing governments for inaction and more time talking about how I can reduce my emissions."

"This bad restaurant review could spend less time bashing the chef's food and more time telling me where the good restaurants are."

Similarly maybe the author didn't explain what "IV" means because their audience understands that term.

"This article in CACM uses 'NVRAM' in the heading, while the words "non-volatile" don't appear until much later. Non-volatile is pretty obvious, 'NVRAM' isn't."

> I myself don't really understand IVs

Time to drop some knowledge!

IVs are used in a number of places in cryptography, so I'll just pick one (easy) example.

Consider the stream cipher ChaCha20. You can think of ChaCha20 as a black box. You input a key and an IV and out you get a really, really long stream of uniformly random bytes. (This is a simplification but sufficient here). ChaCha20 works in such a way that having any or all of the output stream doesn't help you figure out what the inputs were. It's irreversible. ChaCha20 is also deterministic; the same input will give the same output.

You can then use the output of random bytes to encrypt a message by XORing with your plaintext. To later decrypt, you feed the same key and IV, get the same stream, XOR the ciphertext with it, and by the property of XOR you'll get the plaintext.

Now why is there an IV? Let's consider a ChaCha without an IV. The system works like so:

    R = ChaCha(Key)
    Ciphertext = Message ^ R

So let's encrypt two different messages:

    R = ChaCha(Key)
    Ciphertext1 = Message1 ^ R
    R = ChaCha(Key)
    Ciphertext2 = Message2 ^ R

Notice how R is the same for both messages? Again, ChaCha is deterministic; the output is the same for the same inputs. Since the key is the same, R is the same. Now an attacker, knowing this, can do this

    Q = Ciphertext1 ^ Ciphertext2

What does Q end up being? Let's look:

    Ciphertext1 ^ Ciphertext2
    = Message1 ^ R ^ Message2 ^ R
    = Message1 ^ Message2

So Q ends up being equal to the XOR of the two messages. That's really bad. The xor of two messages might be enough to tell the attacker what the messages are, especially if the messages are predictable (like english text). But maybe that's not scary enough. Well there's another attack. What if you're encrypting a data format with a header. Headers often have the same data in the same places. So the attacker knows part of the message. Uh oh...

    R = Ciphertext1 ^ Message1

If the attacker knows the message (or any parts of it) they can recover the R of those parts. And now, since your key is always the same and your R is always the same, all the other messages you encrypt will have those bytes exposed.

This is where IVs come in:

    R = ChaCha(Key, IV)

IV should be unique per message. That means that every R is different! None of the above attackers work anymore. XORing two ciphertexts together returns gibberish:

    R1 = ChaCha(Key, IV1)
    Ciphertext1 = Message1 ^ R1
    R2 = ChaCha(Key, IV2)
    Ciphertext2 = Message2 ^ R2

    Ciphertext1 ^ Ciphertext2
    = Message1 ^ R1 ^ Message2 ^ R2

And if the attacker knows the message, all they can recover is R1 or R2 (or any R). But that's useless, because since all your IVs are unique that R will never be seen ever again.

That's the point of IVs.

> why the IV isn't required to be able to decrypt the file again.

It is required. Obviously you need all the inputs to ChaCha to get the byte stream again, to decrypt the message.

Now sometimes the IV is known from the protocol. So say you're using ChaCha to encrypt network traffic. You might set the IV equal to the packet number. So both sides already know the packet number.

But you always need the IV to decrypt.

> and from my perspective I'm left with no clearer of anuidea about why IVs shouldn't be considered secret,

Consider again ChaCha20 as a blackbox. Key+IV goes in, stream of bytes comes out. There's no way to reverse that without the key (and IV). Since the attacker doesn't know the Key, they can't reverse it. Knowing the IV doesn't help.

Another way to think about it is that, instead of accepting a 256-bit key and a 64-bit IV, it's really just a 320-bit key. Knowing 64-bits of a 320-bit key doesn't help break a cipher. The cipher is still 256-bits strong. So you can share the IV without affecting security.

BIG NOTE: It's important that an IV is always unique. If an IV is ever re-used, the above attacks become available again because R will be the same for two messages.

Hope that helps. This is only one way that IVs are used. In ChaCha20 it's called a nonce, because ChaCha20 is geared towards usage on network protocols where the above trick of using packet number is applied. For block ciphers there are various cipher modes that get used, and most of them need an IV. The purpose is always the same; to make this "session" of encryption unique.

There's another way to use IVs, and I think they re-affirm the concept of what an IV actually is. Let's say you have a cipher that only accepts a key! No IV (like AES). You still want to make your encryption sessions unique. A way to do that is this:

    TempKey = HMAC (IV, Key)

And then use TempKey. HMAC is a form of hash. In this case it lets us combine a Key and IV in an irreversible way, yielding a new key. TempKey will be the right size key for the cipher (say, 256-bits). What this is doing is giving us a unique key for every encryption session. And that's the heart of IVs. And in many ways, ChaCha20 is doing exactly that. It's hashing together Key and IV and using the output hash to generate a long stream of random data that can't be reversed back to the key+IV.

(and in case you're wonder, yes, you can use a cryptographically secure hash function alone to build a stream cipher like ChaCha. It'll just be _really_ _really_ slow, because hash functions are really, really slow compared to ChaCha.)

In addition to fpgaminer's excellent explanation, I highly recommend the book "Cryptography Engineering: Design Principles and Practical Applications" by Niels Ferguson, Bruce Schneier, and Tadayoshi Kohno. It's an excellent overview of how to use crypto primitives and why to use them that way.

I know you're not just looking for answers but a pointer to some better documentation, and I can't provide you with those, but:

> why IVs shouldn't be considered secret

The least is considered secret, the least can be leaked and cause problems.

> why the IV isn't required to be able to decrypt the file again

The IV is required to decrypt the file again. In the linked document's design the IV is actually the encryption key, which means it is known by the receiver, which is why it's not included. But that is just a special case that should never be reproduced.

Agreed. Where is the pointer to the correct article to use when encrypting a file in C#.

Luckily we have a snapshot: https://web.archive.org/web/20170327154501/https://support.m...

Here's the original Microsoft article:

https://gist.github.com/mikemaccana/badf6c16f203e05c02b42f93...

(disabled JS in DevTools, caught it from archive.org before JS to wipe it kicked in)

This benchmark [1] gives 196.2 GH/s for DES using 8 Nvidia GTX 1080 Ti and Hashcat 3.5. So while your average computer is probably not quite sufficient it is certainly in reach.

[1] https://gist.github.com/epixoip/ace60d09981be09544fdd3500505...

Here's a project that did 1.4G/s on a single GPU five years ago:

https://www.reddit.com/r/crypto/comments/162ufx/research_pro...

Stick multiple modern GPUs in a machine and single digit days seems feasible.

Yes: http://www.h-online.com/security/features/A-death-blow-for-P...

This does not contain a MAC though, does it? Also why CBC? Why not CTR/GCM instead? And why AES256 instead of Chacha20-Poly1305 or some other modern AEAD?

Oh, come on: whatever Microsoft's faults might be they have a very long track record, stretching back decades, of providing overall high quality documentation for developers.

Yes, there are errors. Yes, sometimes there is deeply misguided advice. But, on the whole, MSDN and its ilk has helped me far more often than it's hurt me.

Key point: compared with much other vendor and OSS documentation, Microsoft are absolutely streets ahead.

But that's in the codecs library, not a cryptography library.

To be fair that's not a tutorial on how to encrypt and decrypt a file, it's a reference on the possible encodings you can use for a string.