End-to-end encryption between Node.js and webcrypto

Written by Jeroen van Veen on 30th April 2016

Creating a Javascript p2p web framework has been a personal project, almost an obsession, of mine for almost two years now. Two months ago I hit a brick wall when I was trying to implement end-to-end encryption using the webcrypto API and Node.js crypto. Here is a short explanation of what I am trying to accomplish, and what progress I made so far.

p2p

Background

The open source project I work on is called High5. Its purpose is to allow creation of an overlay network of universal/isomorphic Javascript peers (nodes), that are interconnected by using suitable transports. This could be a websocket transport, a memory transport (to provide an HTML snapshot state to other nodes) or a webRTC transport (browser to browser, not implemented yet). Each node reasons about the network and routing using a graph. The figure above represents an imaginary overlay network, where the yellow nodes are running inside Node.js and the blue nodes are running in Chromium/Chrome. A simple preliminary protocol allows nodes to route JSON messages to each other.

If Node C wants to send a chat message to node F, then it has to route all the way through node A and B (or through A-D-G-B-F) in order to arrive at node F. The node figures out the route by using Dijkstra’s shortest path algorithm. These routing messages can be used to exchange peer-to-peer web application data with, and are also useful to setup direct webRTC transports between browser nodes with. It then uses the overlay network as a decentralized signalling channel for webRTC SDP messages. Take for example the webRTC transport between node D and G. By using route D-A-B-G as a signalling channel, it’s quite straightforward to setup a connection between these browsers, which would otherwise not be aware of each other.

Asymmetric encryption

Now this is all fine and dandy, but node C doesn’t want node A and B to read a message that’s meant for node F! Then how do we get the message from C to F without A and B being able to read it? An option would be to use an encryption method like AES to scramble the message with a common secret key; the session key. AES is a fast and effective way to encrypt and decrypt larger messages with. But how do we get the session key to node F without node A and B learning about it? If A or B knows the session key, then encrypting the text wouldn’t make much sense anymore, because anybody with the session key can decrypt the message! As it turns out, this problem was already solved in the early 1970’s with the invention of asymmetric cryptography methods like RSA and Diffie Hellman key exchange.

Asymmetric cryptography uses two keys; a private key and a public key. A message from Alice to Bob can be encrypted by Alice using Bob’s public key. Bob receives the encrypted message from Alice and is the only one being able to decrypt it using his private key. Asymmetric cryptography is rather slow and has limitations on the encrypted message size, but is very useful to exchange a common secret, the AES session key, between Alice and Bob without having to fear that an MITM can read it as well.

RSA encryption

After learning a bit more about cryptography methods, I initially thought I needed to create this RSA key exchange myself. On the Node.js side I initially used node-rsa to generate a key pair in Node.js. Using some webcrypto examples, I managed to encrypt some text using RSA-OAEP-256 (OAEP padding with SHA-256 hashing) and the Node.js generated public key. At that point, I thought I would be able to decrypt it in Node.js using the private key. Wrong! I couldn’t get it to work, because the node-rsa decrypt method kept on complaining about invalid OAEP padding. I tried other libraries like ursa to see if that would fix the problem, but it didn’t. After countless hours of messing around, not really knowing what I was doing, I finally found the reason. The used OAEP padding mechanism in node-rsa and in ursa’s OpenSSL bindings is hardcoded to use SHA1 hashing! Knowing this, I finally managed to decrypt the message using node-forge, which allowed me to use SHA-256 hashing for OAEP.

Diffie Hellman key exchange

Since then, a lot of people who know a thing or two about cryptography advised me not to ‘roll-my-own’ key exchange mechanism. So I continued reading and learned a bit about Diffie Hellman key exchange. DH is probably the right solution to safely exchange a common session key. With DH, Alice and Bob still exchange two public keys, just like with RSA. The difference is that with DH, Alice can derive the session key directly from her private key and Bob’s public key. Bob can do the same with his private key and Alice’s public key. They both end up with the same secret, without an MITM to be able to learn about it. This is better than ‘rolling-my-own’, since it doesn’t require to send the session key (encrypted) over the wire at all. Alice and Bob can from then on use the negotiated AES session key to encrypt follow-up messages. There is also the topic of message authentication, but I’ll leave that outside the scope of this article. Webcrypto allows using DH by using ECDH. Now I’m back at where I began! Generating an ECDH key pair with webcrypto is quite straightforward, but how to do the same thing in Node.js, and properly generate the session key with the right parameters?

Node.js Subtle

Node.js built-in crypto library doesn’t resemble the webcrypto API at all. There are some third-party libs, but none of them explain how to interop with webcrypto. Happily I found the Subtle project, which aims to be a ‘WebCrypto Polyfill for node/browsers’. Awesome! This would make life on the Node.js side a lot easier, because then we can stick to the webcrypto API and perform the same operations in the browser and on Node.js. Bummer! It failed to compile on Node.js 5.x. Subtle depends on a compiled elliptic curve extension called ecc, which in turn depends on an older version of nan. After searching around for some hours about nan, I managed to get ecc working with nan 2.1.0. With Subtle up and running, I tested it to do the same RSA key exchange as described at the beginning of this post. Importing and exporting the keys was a breeze using spki/pkcs8, but now I got the same OAEP padding error again! (Error: Invalid RSAES-OAEP padding.]). This may have to do with the SHA1 hashing issue again. Finally, I tried ECDH key exchange between webcrypto and Node.js Subtle, but sadly this failed as well using my example code. Back to the drawing board…

The way to go?

I still have to figure out how to properly apply RSA encryption and ECDH key exchange using the Node.js Subtle library, as well do some more research on message authentication (digital signatures) and ephemeral ECDH. Overall, I feel that this Subtle library may be on the right track to make end-to-end encryption accessible for Node.js web developers without having to know all the ins and outs of cryptography. Another interesting direction to this may be the TweetNaCl-js project, but I didn’t try this yet. This is it for now! I need to read a lot more about the subject first, and then I’ll be able to write the next post, in which I will describe how to implement a complete end-to-end message encryption method.

Background information

If you enjoy this subject and want to know more, I found these resources very helpful when trying to learn a bit more about cryptography:

Do you know other resources which could be useful or have any other tips to help me on my quest, let me know in the comments below!

Your thoughts

  • Written by Shyam on 2nd August 2016

    Hi Jeroen,

    Nicely summed up. Did you manage to get the inter-operability between node crypto and WebCrypto? I am stuck at the same point as you are.

    Thanks,
    Shyam

  • Written by Jeroen van Veen on 17th August 2016

    Hi Shyam,

    Yes, I got that somewhat functional in this subtle fork (https://github.com/garage11/subtle/commits/master). This is very hacky and worked for me, but I would first investigate this webcrypto polyfill(https://github.com/PeculiarVentures/node-webcrypto-ossl). Also please notice https://github.com/garage11/subtle/commit/b69391bca4a92a215ceb6b39ae926f1b1770861c#diff-80190a42addc00707da1550249e75407R10

  • Written by Alex on 7th October 2016

    This is awesome story, and something I want to get into.

    It would be cool to hear more, since I am looking for end-to-end encrypted app for exchange of data, over WebRTC, potentially in Electron app.

    THanks
    Alex

  • Written by Jeroen van Veen on 24th October 2016

    Hi Alex,

    Thanks, I would like to write another article, but I need to do some background research on the topic first. The current status is that I replaced Ecc in subtle with Node.js/openssl function calls and that I have a working mvp that does ECDH key exchange and AES GCM using this Subtle fork. I still need to figure out how to properly apply AES GCM and I would like to see if i can replace subtle with node-webcrypto-ossl.

    As soon as I get to it, I’ll continue this article with a follow-up. It may take a while though.

  • Written by John on 9th March 2017

    Hi,

    First, sorry for my English, so hard for me.

    Can you share your code for encryption web-node? I am trying to use in progresive web app for encrypt indexedDB and after send to server secure.

    Regards,
    John

  • Written by Yury Strozhevsky on 5th April 2018

    Main aim of the node-webcrypto-ossl is supporting for PKIjs library (https://github.com/PeculiarVentures/PKI.js). So, in fact you PKIjs could work on both sides (in browser and in Node environment). As for encryption – in PKIjs we made all kinds of encryption (except for key agreement using ECDH – because ECDH is not officially supported in WebCrypto API). You could find all examples in the “examples” subfolder. Live examples could be found here – http://pkijs.org/#examples. As for Node examples – you could check CircleCI and check that all the same tests run in Node (https://circleci.com/gh/PeculiarVentures/PKI.js).

  • Written by Jeroen van Veen on 5th April 2018

    Hi Yury,

    Thank you for the links! PKI.js looks very interesting. I was reading something lately about establishing trust between peers using web of trust mechanisms. I’m wondering what other methods could be applied when it comes to establishing trust in a public key. Could PKI.js be used for something like that? What other usecases are there? I don’t have any crypto background. I just wrote this article to document some of my findings and curiosity while trying to secure messages on-transit in a hobby prototype application. Could you elaborate what PKI.js can be used for in less technical terms?

  • Written by Robin Lunde on 29th April 2018

    Hey!
    Maybe have a look at this module:
    https://github.com/anvilresearch/webcrypto ?
    Seems to have a lot of great functionality for inter-operability between nodejs and webcrypto API!

  • Written by Yury Strozhevsky on 4th May 2018

    PKIjs has a really FULL set of classes to implement ALL kinds of PKI-related services/data. PKIjs could be suitable for making Certificate Authority servers (CA), OCSP servers, TimeStamp services and many more. Anything you could imagine related to PKI could be implemented using PKIjs. On top of this library already exists CAdES.js (library for making CMS Advanced Signatures) and PAdES.js (library for making and verifying signatures in any PDF files). Both libs are private at the moment, but CAdES.js would be published within May 2018 and PAdES.js would be published later this year. There are number of enterprise-level solutions using PKIjs, but information about such services is not sufficient for making good description of them. I am the author of the PKIjs and developing all these libs for years. Any questions welcome yury@strozhevsky.com

  • Written by Akshith on 26th May 2018

    Hey Jeroen,
    Thank you for the summary.
    Now that its been about 2 years since the post, is there any update you can give us on the state of crypto?

    Best

  • Written by Jeroen van Veen on 18th June 2018

    Hi Akshith,

    To be honest, this personal project (High5, later called Garage11) died silently without much traction, and so did my research on applying webcrypto. I’m planning to reintroduce the concept of an overlay network (and further R&D concerning webcrypto) in the somewhat distant future with a more promising company project I’m involved in (https://github.com/voipgrid/vialer-js). It is an opensource softphone using WebRTC and SIP-over-websockets to place calls with, with the purpose to become a unified communication tool supporting multiple protocols. This is where an overlay network may be an interesting option. It’s so far using webcrypto to encrypt application state with using PBKDF2, but it will take considerable time and resources to get at the point where an overlay network could be introduced. It also needs some more work to make it generic for non-platform users, but I’m quite optimistic about its progress so far. Will write a new article by then.

Devhouse Spindle