If you are as old as me, you probably remember the sound of a modem and the awesome ASCII drawings that welcomed you when you logged into a BBS.

  _  __           _                          _               
 | |/ /          | |                        | |              
 | ' /  ___    __| |  ___  _ __ ___    __ _ | | __ ___  _ __ 
 |  <  / _ \  / _` | / _ \| '_ ` _ \  / _` || |/ // _ \| '__|
 | . \| (_) || (_| ||  __/| | | | | || (_| ||   <|  __/| |   
 |_|\_\\___/  \__,_| \___||_| |_| |_| \__,_||_|\_\\___||_|

Usually, this was handcrafted art that probably took hours to create. We have neither the time nor the skills for that, so let's ask the computer for some help instead. What does it take to generate ASCII from an image in the browser?

First, an image

The first thing we need to do is to be able to read color information from an image. The <canvas> element can help us with that. The Canvas API gives us access to each individual pixel and their colors. We load an image in some way (there are several ways to do this), and draw it to the canvas.

It looks like this:

// Load an image that is embedded in the page.
// Alternatively, you can ask the user to upload an image
const rawImageElement = document.getElementById("sourceImage");
const canvas = document.getElementById('theCanvas');
const context = canvas.getContext('2d');
context.drawImage(rawImageElement, 0, 0, canvas.width, canvas.height)

Now we're ready to extract the colors of each pixel. We do this via an ImageData object. You might expect that the API would give you access to each pixel in the form of a two-dimensional matrix. Conceptually, what we want is a getPixelAt(x, y) function that returns a representation of the pixel at that point.

It could, for example, look like this:

{ red: 1, green: 2, blue: 3, alpha: 0.5 }

We're not that lucky, we'll have to create such a function ourselves. We'll come back to that.

ImageData has a data property which is a one-dimensional Uint8ClampedArray.

const imageData = context.getImageData(0, 0, canvas.width, canvas.height)

What on earth does that mean?

• Uint8 means that each value is an unsigned integer of 8 bits, i.e., 2^8 possible values. This means you can represent integers between 0 and 255. This is the same range that RGB colors are defined in on the web.
• Clamped hints that the values cannot under or overflow. This works well with image processing, e.g., if you want to increase the brightness in an image. The maximum brightness of a color is 255, so if we add 1, we get 255 and not 0.

From one dimension to another

This array is in one dimension, but an image has two dimensions. How can we extract the color of a pixel at position [x, y]?

Each pixel is represented by four elements in the array. It looks like this:

[
    x0y0Red, x0y0Green, x0y0Blue, x0y0Alpha, 
    x1y0Red, x1y0Green, x1y0Blue, x1y0Alpha,
    x2y0Red, x2y0Green, x2y0Blue, x2y0Alpha,
    ...
]

To extract a given pixel, we can do something like this:

const getPixelAt = (imageData, x, y) => {
    const redIdx = y * (imageData.width * 4) + x * 4;
    return {
        red: imageData.data[redIdx],
        green: imageData.data[redIdx + 1],
        blue: imageData.data[redIdx + 2],
        alpha: imageData.data[redIdx + 3]
    }
}

Convert to grayscale

Now we have the colors of each pixel readily available, and can start the process of converting them to ASCII characters. Since we're going to display the ASCII characters on this website, we can choose to retain the color information in the image by applying color to each ASCII character. Traditionally, ASCII art doesn't have any color, so we'll stick with using grayscale.

There are several ways to convert colors to grayscale, but the simplest is to take the average of the RGB values.

So if the color is {red: 100, green: 10, blue: 10}, the grayscale value becomes:

(100 + 10 + 10) / 3 = 40

The example below shows how to convert a grayscale value to an ASCII character.

$@B%8&WM#*ohkbdqwmO0QCJYXzvunrjf/|()1}[]?-+~<>i!lI;:,"^`'.

Here we can see that the ASCII character $ represents the darkest grayscale tone (0), while a period represents the lightest tone (255). The idea is to create a list of characters that gradually fill their "square" with fewer and fewer pixels.

The code to convert a grayscale value to an ASCII character then becomes:

const brightnessToChar = (darkToBrightArray, brightness) => {
    const charIdx = Math.floor(((darkToBrightArray.length-1) / 255) * brightness)
    const character = darkToBrightArray[charIdx];
    // Force the web page to actually render a space character
    return character === " " ? " ": character;
}

We can call this function like this:

const DARK_TO_BRIGHT_ASCII = "@#$&%*o+i;:,.'` "
brightnessToChar(DARK_TO_BRIGHT_ASCII, 0) // returns @
brightnessToChar(DARK_TO_BRIGHT_ASCII, 255) // returns  

Here we can experiment with different variants to see which lists of ASCII characters work best. In the following, we use a shorter variant that gives a less detailed expression.

@#$&%*o+i;:,.'`

Converting an image

Here I've taken a picture of a jumping happy dog, let's try to convert it to ASCII. I've removed the background beforehand to get the best possible result. You can of course remove the background automatically with code, but that will be another blog post.

To render each cell with ASCII characters, we simply convert each pixel to a <div>, and then put it in a CSS grid.

const addToDom = (imageData, parentDomElement) => {
    const container = document.createElement("div");
    container.style = "font-family: monospace; display:grid; grid-template-columns: repeat(" + imageData.width  +", 1rem)"

    for (let i=0; i < imageData.data.length; i += 4) {
        const x = (i / 4) % imageData.width;
        const y = Math.floor((i / 4) / imageData.width);

        const cell = document.createElement("div")
        cell.innerHTML = brightnessToChar(brightnessAt(x,y, imageData));
        container.appendChild(cell)
    }
    parentDomElement.appendChild(container);
}

I see your ASCII, and raise you Emoji

That was all well and good, but we can do even better. What if we replace ASCII with emoji?

First, we need to know which emojis are supported on the web, which we can find here.

A possible mapping from dark to light could be:

🖤 🥷 🦍 🦓 👣 👻 💀 👀 🦴 🤍 💬 🗯

Everything is a remix

The result is as follows

Since this is on the web, we can easily set aside the grayscale and go for colors. We just need a mapping from color + intensity to emoji characters. Creating the mapping by hand sounds boring, but we can easily create a program that automates it for us.

Part 2

The next thing I'm going to do is connect this to the webcam API. Seeing yourself as a stream of emojis, what could be better than that?

Here is the blog post about videomoji

For those especially interested, you can read the source code here.