The Trouble with crypto libraries

Cryptography is hard. Hard to design, hard to implement, hard to use, and hard to get right. 

Modern and well-studied ciphers rely on intractable problems and are believed to have a very high security margin, even in a post-quantum world.

But the cipher is rarely the weakest link in an application using cryptography. No matter how secure a function is, its security can be totally destroyed by a tiny weakness in its implementation or by using it incorrectly. And an application using cryptography has to do way more than picking a decently secure set of primitives.

The well-known OpenSSL library provides a wide range of cryptographic primitives and related helpers. Unfortunately, developers are likely to be confused by its overly-complex API. And sadly, the W3C crypto API is currently taking the very same path.

Is encryption all it takes to provide integrity? Is the key length the best indicator of how secure a cipher is? How should random numbers be generated? What are these “modes”? Is opting for the fastest one, like ECB, the best way to go? Why not just use SHA1(secret | message) instead of a HMAC?

Frequently, developers make arbitrary choices in response to the above questions, leading to security disasters. Worse, this happens most frequently when the intent was to achieve a very common operation.

In order to address this, Daniel J. Bernstein released NaCl.

NaCl is a new and very opiniated cryptographic library.

From a user perspective, it exposes a very simple, high-level API, with a tiny set of functions for each operation. Under the hood, it uses high-speed, highly-secure primitives and constructions, implemented with extreme care to avoid side-channel attacks. NaCl has no low-security options and makes very conservative choices of cryptographic primitives, while remaining exceptionally fast.

NaCl is an awesome toolkit for developers willing to add cryptography to their applications, that drastically reduces the risk of building insecure constructions. In fact, last year OpenDNS released DNSCrypt, an open-source tool I built for securing DNS communications, that leverages the NaCl crypto library.

But NaCl didn’t get much adoption, for several important reasons:

  • While each primitive comes with one or more portable implementations, NaCl itself is not portable. In particular, it can only be compiled on some Unix flavors.
  • NaCl ships with constructions that were just prototypes, and that shouldn’t be used any more.
  • The compiled code is only guaranteed to work on the machine it was compiled on.
  • NaCl is not a shared library, making it difficult to use in other programming languages, and it hasn’t been designed to be installed system-wide.
  • Packaging NaCl for different operating systems is tough due to its specific build system and compile-time requirements.

Well, NaCL just got a lot better. Today, I’m announcing Sodium, a portable, cross-compilable, installable, packageable, API-compatible version of NaCl. 

Sodium uses the same implementations of crypto primitives as NaCl, but is known to work on all the platforms supported by DNSCrypt, including Bitrig, OpenBSD, Dragonfly BSD, NetBSD, FreeBSD, SmartOS, OSX, Linux, Windows, iOS and Android.

Sodium installs a shared library and a standard set of headers, supports parallel compilation and testing, and uses a portable implementation of each primitive.

It also includes convenience functions for generating secure random numbers, and adds additional primitives and helpers to address common needs.

What Sodium does for you

  • Authenticated public-key and authenticated shared-key encryption
    This operation ensures that a message remains secret and can’t be modified by an attacker.
  • Public-key and shared-key signatures
    This operations allows the recipient of a message to verify that it actually comes from the expected sender, and that it hasn’t been modified.
  • Hashing
    This operation compresses a message to a short, fixed-size output from which the original message can’t be computed.
  • Keyed hashes for short messages
    A lot of applications and programming language implementations have been recently found to be vulnerable to denial-of-service attacks when a hash function with weak security guarantees, like Murmurhash 3, was used to construct a hash table.
    In order to address this, Sodium provides the “shorthash” function, currently implemented using SipHash-2-4. This very fast hash function outputs short, but unpredictable (without knowing the secret key) values suitable for picking a list in a hash table for a given key.
  • Secure pseudo-random numbers generation
    This generates pseudo-random numbers suitable for cryptographic purposes. Implementations can be dynamically chosen at run-time. On Windows platforms, it uses the Cryptographic Services Provider by default. Sodium also provides a secure, chroot()-resistant drop-in replacement for the arc4random() function family, including the ability to generate random numbers within a given interval with a nearly random distribution.

Why I built Sodium

One of the best things about working at OpenDNS is that the company is committed to the larger goal of making the Internet safer, and being transparent in our efforts. We build products that secure businesses against malware, but we also take time to build tools that can serve the greater needs of the security community. For example, earlier this year the Umbrella Security Labs announced theUmbrella Security Graph, a tool that helps researchers predict unknown threats. Before that, OpenDNS released DNSCrypt, an open-source tool I built for securing DNS communications, that leverages the NaCl crypto library. The company has been vocal on privacy, too (see

OpenDNS’s open letter to congress on SOPAand anti-censorship policy). As an engineer and security advocate, I see cryptography as a critical building block to achieving privacy and security. I wanted to give back to the security community in a way that could make real impact. I hope you’ll find Sodium to be a valuable resource in your cryptography toolbox, and in your efforts to secure communications.

Installing Sodium

People quickly adopted Sodium and built packages for different operating systems.

The source code can also be downloaded here: Sodium source code or, for the bleeding-edge version, clone the Sodium Git repository

 

Using Sodium

While the Sodium library provides an API for the C language, bindings for other languages, currently Ruby and Python, are also available:

Python example

Installing the PyNaCl package

$ git clone git@github.com:dstufft/pynacl.git
$ cd pynacl
$ python setup.py install

Public-key signature

import nacl
key_seed = nacl.random(32)
private_key = nacl.signing.SigningKey(key_seed)

# this preprends a 512-bit signature to the message
signed_message = private_key.sign('message')

public_key = private_key.verify_key
# signing_key has to remain secret
# public_key should be sent to the recipient

Verifying the signature of a signed message

try:
message = public_key.verify(signed_message)
print(message)
except BadSignatureError:
sys.stderr.write("Incorrect signaturen")

Ruby example

Installing the RbNaCl gem

$ git clone git://github.com/cryptosphere/rbnacl.git
$ cd rbnacl
$ gem build rbnacl.gemspec
$ gem install rbnacl-*.gem

Public-key authenticated encryption

# Alice's key pair
alice_private_key = Crypto::PrivateKey.generate
alice_public_key = alice_private_key.public_key

# Bob's key pair
bob_private_key = Crypto::PrivateKey.generate
bob_public_key = bob_private_key.public_key

# Alice's "box" to encrypt/decrypt messages for/from Bob
alice_box = Crypto::Box.new(bob_public_key, alice_private_key)

# Bob's "box" to encrypt/decrypt messages for/from Alice
bob_box = Crypto::Box.new(alice_public_key, bob_private_key)

# Alice encrypts and signs a message for Bob.
# This operation needs a nonce for each message: a unique 192-bit value that
# should never ever be reused with the same key.
nonce = Crypto::Random.random_bytes(24)
ciphertext = alice_box.box(nonce, 'message')

# Bob verifies and decrypts Alice's message
begin
message = bob_box.open(nonce, ciphertext)
puts(message)
rescue Crypto::CryptoError
STDERR.puts("Ciphertext failed verification")
end

Privacy and security are important concerns, and cryptography is a critical building block to achieve these.

 

  • Zooko Wilcox-OHearn

    Congratulations! I’m happy to see this development.

  • Terry Smith

    Awesome thank you so much!

  • Fellow Traveler

    Great work!

  • Sean McGregor

    Great stuff. I spent this morning looking into building crypto libraries for Google’s NaCl (Native Client)… Naming confusions aside, how difficult would it be to build Sodium for Native Client? It looks like the build systems might fight each other, but Sodium could potentially serve several powerful use cases on the platform. Regardless, thanks for releasing this!

  • Jeff D

    How does this compare to OpenSSL?

  • Stickplayer

    Confused – you mentions specific support for Windows, yet you don’t list Windows at all under “Installing Sodium”.

    Is there, in fact, a Windows install? Or a Visual Studio compatible project?

    Thanks!

  • http://blog.est.im/ est

    n00b here, is soldium enough to 1. decrypt SSL/TLS 2. Write pure python openssh client? 3. implement simple openvpn protocol?

  • http://www.facebook.com/people/Dan-Kaminsky/515164691 Dan Kaminsky

    I shall be subscribing to this newsletter.

  • http://twitter.com/7fttallrussian Dmitriy

    Bernstein used some crazy speed optimizations in NaCl, that drove their build system almost unusable. Did you just drop them?

  • http://www.facebook.com/magicseth Seth Raphael

    Having done unholy things to compile NaCl on iPhones and androids, this is such a welcome addition!

  • Transonic

    Frank – great to see this undertaking!

    2 questions:

    1 -
    “Why not just use SHA1(secret | message) instead of a HMAC?” – I’m not a
    crypto guy, so I’ve often wondered the same thing. I am 100% sure
    there is a good reason, but is it something you could take a shot at
    explaining? Thanks!

    2 – I’ve been intrigued by NaCl for a while,
    but I was put off by its requirement of an underlying Unix OS. I work
    on 32-bit MIPS processors for cable modems. Your work is an important
    step towards making NaCl even more platform independent, but it seems
    like its still targeted to “desktop” platforms. I’ve never understood
    why something that’s basically just moving bits around isn’t one of the
    *most* portable pieces of software. I understand there are things like
    underlying random number generators, but I don’t see any reason why that
    can’t be abstracted away into an required function of the port (e.g.
    “Provide a function “uint32_t NextRand(void) that returns the next 32
    bit unsigned integer from your platforms PRNG” or whatever (I’m sure
    I’ve got the API wrong, that’s not the point)

    I guess what I’m asking is – how hard would be it / what needs to change to port NaCl to a “bare metal” system like 32-bit MIPS, PIC, ARM Cortex M3 or something? No RTOS, no /dev/urandom, just the processor & memory, and say “You write these routines that bolt into the hardware” or whatever.

    Thanks!

  • http://twitter.com/jedisct1 Frank Denis

    Answering questions through comments on this blog is not very convenient, so feel free to join the Sodium mailing list (see the web site for more info on it) to ask more questions and discuss about the library.

    Jeff: see http://cr.yp.to/highspeed/coolnacl-20120725.pdf

    Stickplayer: sample build scripts for Windows, iOS and Android are available here: https://github.com/jedisct1/libsodium/tree/master/dist-build

    Sodium can be built on Windows using MingW. I’m currently developing it under the MSYS environment. What you get is a native DLL compatible with Visual Studio and anything that can load a dynamic library. It doesn’t have any dependencies on any other library.

    Est: TLS, OpenSSH and OpenVPN are totally different protocols. Sodium doesn’t implement any of these. However, if you’re looking for a super fast and simple alternative to OpenVPN and IPSec based on NaCl, take a look at SigmaVPN: https://play.google.com/store/apps/details?id=com.frozenriver.sigmavpn&hl=en – This includes an excellent client for Android.

    Dmitriy: Sodium uses autotools.

    Transonic:

    1) Just like MD5, the SHA family function is vulnerable to hash length extension attacks: http://en.wikipedia.org/wiki/Length_extension_attack

    It allows an attacker to append extra data to the message, and compute a valid tag for the new input, even without knowing the original message or the key. This is a simple attack, yet it can be a devastating against web applications if such a scheme is used to verify queries or cookies.

    One way to prevent length-extension attacks is to hash the output of the function itself: h(h(m)). A HMAC works the same way, plus proper padding, to build a keyed hash function out of deterministic functions.

    Sodium provides the crypto_auth() function to do so, currently implemented as HMAC-SHA-512-256.

    2) The default PRG uses /dev/urandom, but Sodium has been designed to be as portable as possible and the PRG can be switched at runtime. See the randombytes.h header. The default PRG implementation is defined in randombytes_sysrandom, which uses /dev/urandom on Unix and the CSP on Windows.

    A sample alternative implementation is shipped with Sodium, namely randombytes_salsa20_random. With that one, /dev/urandom is only used once, in order to derive a key, and the PRG output then comes from a salsa20 stream cipher.

    You can use these implementations as examples to write a new implementation specific to your hardware. Sodium doesn’t need any changes to support your new implementation, just call randombytes_set_implementation() at startup time, and this will be used for all operations requiring random numbers.

    • Transonic

      Frank – thanks for taking the time to answer my questions in great detail, I really appreciate it. Sorry for taking so long to acknowledge it! After reading your answer about NaCl & portability, I’m motivated to try it on my embedded platform.