Hex to Base64 Encoding in Go

Intro

I’ve recently started a new job as a security engineer, and in preparation I’ve been going through the Cryptopals Crytpo challenges. In the first challenge, we are given a hex encoded string:

49276d206b696c6c696e6720796f757220627261696e206c696b65206120706f69736f6e6f7573206d757368726f6f6d

which we need to decode, and then encode to base64, which should look like this:

SSdtIGtpbGxpbmcgeW91ciBicmFpbiBsaWtlIGEgcG9pc29ub3VzIG11c2hyb29t

We can see what we’re starting with and what we’re supposed to end with, but how do we get there?

What the Hex!?

The first part of our problem is decoding our input string that’s been hex encoded, so what does it mean for a string to be hex encoded? To hex encode a string, we break each byte of the string into two 4-bit chunks(also called nibbles) and then we convert each of those chunks into their hexadecimal representation. That sounds really complicated, so let’s break it down.

First, if you’re not familiar with binary and hexadecimal, do a quick read up here.

A string is really just a collection of bytes. In Go, we can take a look at these bytes pretty easily:

import "fmt"
func main() {
    b := []byte("Hello, playground")
    fmt.Println(b)
}

This will print out our byte values (in base10) for the string Hello, playground

[72 101 108 108 111 44 32 112 108 97 121 103 114 111 117 110 100]

From our example above, we can see that H maps to 72. If we were to write 72 in binary, it would look like this:

01001000

We are almost there! Now, we are going to split that byte into two halves. Each half is composed of 4 bits, giving us 0100 and 1000 .

We now are going to represent each of those 4 bit chunks with their hexadecimal value. Here’s a brief guide on converting binary to hex.

0100 in hexadecimal is represented as 4
1000 in hexadecimal is represented as 8
So if we were to Hex Encode H it would be 48

Aren’t we supposed to be decoding Hex?

Let’s take a look at the string we’re given in the challenge again:

49276d206b696c6c696e6720796f757220627261696e206c696b65206120706f69736f6e6f7573206d757368726f6f6d

Let’s see if we can reverse engineer what we just did so we can decode the first character of our string. Remember, 2 hex characters decode to 1 byte, so the first byte of our decoded string will be represented by 4 and 9 from input.

4 in binary is 0100
9 in binary is 1001
So our first byte is 01001001
In decimal that is 73, which in ASCII maps to I .

We did it! We decoded our first character!

This is going to take forever!

Now that we know what’s going on under the hood, let’s see if we can’t leverage some Go code to do this for us. Lucky for us, Go has some really handy built in functions around hex encoding. Here is what I came up with:

package main
import (
 "encoding/hex"
 "fmt"
)
func main() {
    trial :=           []byte("49276d206b696c6c696e6720796f757220627261696e206c696b65206120706f69736f6e6f7573206d757368726f6f6d")
     dec, err := decodeHex(trial)
     if err != nil {
         fmt.Printf("failed to decode hex: %s", err)
         return
     }
    fmt.Printf("%s", dec)
}
func decodeHex(input []byte) ([]byte, error) {
    dst := make([]byte, hex.DecodedLen(len(input)))
    _, err := hex.Decode(dst, input)
    if err != nil {
        return nil, err
    }
    return dst, nil
}

The code is pretty straight forward, so I’m not going to dive too deep into it. All we are doing is making an empty byte slice and using Go’s hex.Decode function to decode our input string to our newly created slice. After running the program, you should see this:

I'm killing your brain like a poisonous mushroom

That’s not even it’s final form!

Now we have finished decoding our message, we have Base64 encode it. You can read up on Base64 Encoding a little more in depth here, but I’ll give the quick rundown.

To base64 encode a string, we’re going to break it down into bytes again. We group three bytes together (giving us 24 bits), then split that into four 6-bit chunks. Then we’ll cross reference each of the 6-bit chunks with this encoding table:

Let’s take a look at the first three characters of the string we’re encoding: I'm

I'm is represented by the bytes 73 39 109, which in binary is:

01001001 00100111 01101101

To Base64 encode this, we will put all those bits together, and divide them into four 6-bit groups. What we’ll end up with is this:

Binary:  010010 010010 011101 101101  
Decimal: 18     18     29     45
Base64:  S      S      d      t

Hey, that looks like it matches up with our desired output string, so we must be doing something right!

There’s a couple more things to keep in mind when base64 encoding. If you get to the end of your string, and the bits don’t divide nicely by 6, add 0’s until you fill your last 6-bit chunk. For example, if instead of encoding I'm, we were encoding just the string I', then it would look like this:

Original: 01001001 00100111
Sextets:  010010 010010 011100  -notice we added two zeroes on
Base64:   S      S      c       =

So our encoded string would be SSc=

You might be wondering where the = came from. When base64 encoding, you always have to end up with sections of 4 bytes. If you don’t, you have to add padding at the end of your encoded string, and the padding character is =.

Just get to the code already!

Once again, Go has pretty nice built in functions for most of this, so our actual code is going to be very straightforward.

We’re going to add this function to our original code:

func base64Encode(input []byte) ([]byte, error) {
    eb := make([]byte, base64.StdEncoding.EncodedLen(len(input)))
    base64.StdEncoding.Encode(eb, input)
 
    return eb
}

Now all that’s left to do is update our main function to call this after our hex decoding function, and we should see our final string.

package main
import (
 "encoding/hex"
 "encoding/base64"
 "fmt"
)
func main() {
    trial := []byte("49276d206b696c6c696e6720796f757220627261696e206c696b65206120706f69736f6e6f7573206d757368726f6f6d")
    db, err := decodeHex(trial)
    if err != nil {
        fmt.Printf("failed to decode hex: %s", err)
        return
    }
 
    f := base64Encode(db)
    fmt.Printf("%s", f)
}
func decodeHex(input []byte) ([]byte, error) {
    db := make([]byte, hex.DecodedLen(len(input)))
    _, err := hex.Decode(db, input)
    if err != nil {
        return nil, err
    }
    return db, nil
}
func base64Encode(input []byte) ([]byte) {
    eb := make([]byte, base64.StdEncoding.EncodedLen(len(input)))
    base64.StdEncoding.Encode(eb, input)
 
    return eb
}

And there we have it, we’ve solved the first problem of Set 1 of Cryptopals. Feel free to play with this on Go Playground here.

If you want to check out my solution on Github, where the code is a little cleaner, there are tests to run, a CLI to play with, and you can peek ahead to future problems I’ve solved, check it out here: https://github.com/juggler434/crypto