Caching data can significantly improve application performance, but how do you ensure your cached data remains valid during testing?
This article demonstrates an effective caching testing strategy using Redis, NodeJS, Docker, Jest, and Testcontainers. You'll learn how to spin up a Dockerized Redis cache in tests, write automated integration tests to verify caching behavior, and invalidate stale cached data.
Following a clear step-by-step guide, you'll gain the knowledge to implement robust caching with Redis while verifying its functionality through automated testing. The result is faster tests, confidence in your cache, and reduced debugging time.
Understanding the Technologies 🔍
Optimizing application performance is crucial for delivering a seamless user experience. One way to achieve this is by leveraging caching technologies, and Redis stands out as a prominent solution for in-memory data storage.
Pairing Redis with NodeJS, a popular JavaScript runtime, can supercharge your app's performance. To top it off, we'll incorporate Docker and Testcontainers, enabling efficient and reliable integration testing. Below is a detailed overview of each of the technologies.
Overview of NodeJS 📶
At the core of modern JavaScript application development, NodeJS has gained massive popularity due to its asynchronous, event-driven architecture. It allows developers to use JavaScript on the server-side to build fast and scalable network applications.
Integrating Redis caching into a NodeJS application is straightforward thanks to the well-supported node_redis client. This further enhances the application's performance and responsiveness.
In-Memory Data Storage with Redis 📝
Redis, short for Remote Dictionary Server, is a versatile and high-performance open-source data structure store. As an in-memory database, it stores data in RAM rather than on disk, providing lightning-fast access times for frequently accessed information.
Beyond acting as a database, Redis is a powerful cache and message broker, making it an ideal choice for improving application responsiveness and scalability.
Docker for Containerization 🫙
Building, deploying, and managing applications can become a cumbersome task when confronted with different environments and dependencies. Docker swoops in to save the day by introducing the concept of containers.
These lightweight, isolated units bundle applications, libraries, and all necessary dependencies, ensuring consistent behavior across various systems. Docker containers guarantee portability, enabling smooth deployment and execution on any platform that supports Docker.
Testing with Jest 🆑
Unit testing is the foundation of a robust application, and Jest has emerged as a popular testing framework for JavaScript developers. It simplifies test creation through its auto-mocking capabilities and runs tests in parallel for improved performance.
With Jest, you can craft test suites to validate the caching behavior of your NodeJS application, ensuring flawless cached data handling.
Testcontainers for Dynamic Integration Testing 🪜
Integration testing is crucial in ensuring that different components of an application work harmoniously. Testcontainers brings a game-changing dynamic to integration testing. It allows developers to create disposable instances of various databases, web browsers, or any application that can run in a Docker container.
This means your integration tests can execute in an isolated, controlled environment, free from interference with other systems. Testcontainers' simplicity and versatility make it an ideal companion for testing Redis-based caching in a NodeJS application.
In the upcoming section, we will seamlessly combine these technologies to test our cached data systematically.
Guide: Building a Node.js App with Redis Caching
Prerequisites 👀
Make sure you have the following installed on your system :
- NPM (Node Package Manager)
- Node.js (v14 or later)
- Docker (for running Redis in a container)
1. Set up the Project 👷
Create a new directory for your project and initialize a new Node.js project using npm :
# create, cd and initialize the project
mkdir redis-cache-test
cd redis-cache-test
npm init -y2. Install Dependencies ⬇️
Install the required dependencies for the project :
# install redis, node and test container libraries
npm install express ioredis supertest testcontainers --save
# install types and test based libraries
npm install @types/express @types/ioredis @types/supertest jest ts-jest --save-dev3. Configure TypeScript 🗂️
Create a tsconfig.json file in the project root to configure TypeScript :
{
"compilerOptions": {
"target": "ES6",
"module": "CommonJS",
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true
},
"include": ["src"]
}4. Set up Express and Redis Client 🌱
Create an app.ts file in the src folder with the following code :
import express from 'express';
import IORedis from 'ioredis';
const app = express();
const redisClient = new IORedis({
host: process.env.REDIS_HOST || 'localhost',
port: Number(process.env.REDIS_PORT) || 6379,
});
app.get('/data', async (req, res) => {
try {
// check if the data is in the cache
const data = await redisClient.get('data');
if (data) {
console.log('Data retrieved from cache.');
res.send(JSON.parse(data));
} else {
console.log('Data not found in cache. Fetching from the database.');
// simulate fetching data from the database
const newData = { message: 'Hello, World!' };
// store the fetched data in the cache for future use
await redisClient.set('data', JSON.stringify(newData));
res.send(newData);
}
} catch (error: unknown) {
console.error('Error occurred:', (error as Error).message);
res.status(500).send('Internal Server Error');
}
});
export { app, redisClient };This code sets up an Express app with a route /data. It checks if data is cached in Redis, retrieves it if present, or fetches it from the database if not.
The fetched data is then stored in Redis for future use. Errors are logged and a 500 status is sent for any exceptions.
5. Writing Tests 🧪
Create a test-setup.ts file in the src folder to set up the Redis container for testing :
// test-setup.ts
import { GenericContainer, StartedTestContainer } from 'testcontainers';
// define a type for the Redis container
export type RedisContainer = StartedTestContainer;
// define a function to start the Redis container
export const startRedisContainer = async (): Promise<RedisContainer> => {
const container = await new GenericContainer('redis')
.withExposedPorts(6379)
.start();
// get the host and port of the running Redis container
const host = container.getHost();
const port = container.getMappedPort(6379);
// set the environment variables for the Redis client
process.env.REDIS_HOST = host;
process.env.REDIS_PORT = port.toString();
// return the container object to stop it after all tests are completed
return container;
};
// define a function to stop the Redis container after all tests are completed
export const stopRedisContainer = async (container: RedisContainer): Promise<void> => {
await container.stop();
};This code defines functions to manage a Redis container using Testcontainers.
startRedisContainer starts the container, sets environment variables, and returns the container object. stopRedisContainer stops the container.
Create a app.test.ts file in the __tests__ folder to write test cases :
// app.test.ts
import supertest, { Response } from 'supertest';
import { app } from '../src/app';
import { RedisContainer, startRedisContainer, stopRedisContainer } from '../src/test-setup';
jest.setTimeout(20000); // set the Jest timeout to 20 seconds to allow the Redis container to start
describe('Redis Cache Testing', () => {
// variable to store the Redis container object
let container: RedisContainer;
// use Jest's "beforeAll" hook to start the Redis container before all tests
beforeAll(async () => {
container = await startRedisContainer();
}, 30000); // Add a timeout of 30 seconds to the beforeAll hook to allow the container to start
// use Jest's "afterAll" hook to stop the Redis container after all tests
afterAll(async () => {
await stopRedisContainer(container);
});
it('should fetch data from the cache', async () => {
// use Response type
const response1: Response = await supertest(app).get('/data');
expect(response1.status).toBe(200);
expect(response1.body).toEqual({ message: 'Hello, World!' });
// use Response type
const response2: Response = await supertest(app).get('/data');
expect(response2.status).toBe(200);
expect(response2.body).toEqual({ message: 'Hello, World!' });
});
});This code conducts integration tests for caching with Redis. It starts a Redis container, fetches data from the cache, and checks the response status and body for correctness.
6. Run the Tests 🤩
Add the following scripts to the package.json file :
"scripts": {
"test": "jest",
"test:docker": "jest --setupFiles ./src/test-setup.ts --detectOpenHandles"
}Now, you can run the tests using the following command :
npm run test:dockerIf your setup is successful, you should see testcontainer and redis image running on Docker Desktop as shown below :
When running the tests, if you provide the correct parameters, you should see an output message similar to the following image :
The project can be downloaded below :
Conclusion 🔥
Testing cached data with Redis, NodeJS, Docker, and Jest using Testcontainers provides a robust and isolated testing environment.
By spinning up a Docker container running Redis for each test, the tests have a clean cache to work with. Jest enables unit testing of the caching logic and integration with Redis.
Overall, this stack provides a comprehensive testing approach to validate caching implementations, ensuring the application handles cached data properly across environments. Leveraging these tools together facilitates effective testing of real-world caching scenarios.