Dustin W. Carr
April 10, 2024
Summary:
Delve into the advantages of using Redis as a primary data store, emphasizing its simplicity and flexibility which empower developers to tailor database interactions specifically for their applications.
redis
database
data management
The reasons to use Redis as your primary data store are plentiful, and that is discussed in detail by many other articles on the web, such as this one . However, one of the most compelling reasons to use Redis is its simplicity and flexibility. What this enables is for you to truly get to know your data, so that you can understand just what it is that you need from your data store. What you may fi\nd is that the indexes and queries that SQL brings only amount to a few extra lines of code. The process of writing that code feels powerful, because there are no hidden interactions, and SQL is replaced with an extremly simple database interface that specializes in YOUR needs for YOUR app, not the general needs of every db programmer in the world.
In this article, I will walk you through the process of writing a simple data interface for Redis, and how to use it to build a simple chat application.
This is not an introduction to redis, or to typescript, or zod. Though it is a basic implementation that uses all of these. Just ask Chat for clarification on any of these topics. The important part is how to get applications written once you have these pieces in place.
The Data Interface
Any database definition requires a schema. In this case, we will use zod to define the schema. Zod is a typescript library that allows you to define schemas for your data. This is useful for many reasons, but in this case, it allows us to define the shape of our data, and to validate that the data we are storing is correct.
Let's demonstrate this with a simple user schema:
import { z } from "zod";
export enum Role {
ADMIN = "admin",
USER = "user",
}
export const createUserSchema = z.object({
id: z.string().default(() => uuidv4()),
name: z.string(),
email: z.string(),
password: z.string().optional(),
created: z.string().default(() => new Date().toISOString()),
updated: z.string().default(() => new Date().toISOString()),
lastLogin: z.string().default(() => new Date().toISOString()),
imageId: z.string().optional(),
imageUrl: z.string().optional(),
role: z.nativeEnum(Role).default(Role.USER),
});
export type CreateUserInput = z.infer<typeof createUserSchema>;
export type User = z.infer<typeof createUserSchema>;
This schema defines a user object, with a few fields. The `createUserSchema` is the schema that we will use to validate the data that we are storing. The `CreateUserInput` type is the type that we will use to create new users. The `User` type is the type that we will use to read users from the database. The ease with which we can generate complex inferred types with zod is one of the benefits of this entire approach. In fact, often developers have to write two schemas, one for the database definition and one for the application. With zod, we can write one schema and use it for both.
Also with zod we can introduce transforms and defaults that provide a great deal of extensibility. We use this to automatically set the timestamps, as well as provide default values for the role and id fields. This is a great way to ensure that the data is always in the correct shape. These are features that people rely upon with ORMs, but with zod, you can have them without the complexity of an ORM, but still have all the type safety.
We will want to have zod schemas for any sort of data query that we do. So for the UserSchema, we will also want to update the user with an arbitrary set of the User fields, but with the Id required.
export const updateUserSchema = createUserSchema
.omit({ id: true })
.partial()
.extend({
id: z.string(),
updated: z.string().default(() => new Date().toISOString()),
});
export type UpdateUserInput = z.infer<typeof updateUserSchema>;To create a database of users, we will be using io-redis, a redis client for node. We will also be using the redis hash data structure to store our users. The hash data structure is a key-value store, where the key is a string, and the value is a map of strings to strings. This is a perfect fit for our user schema.
import Redis from "ioredis";
import { remember } from "@epic-web/remember";
// assumes the environment variable REDIS_URL is set
// family: 6 means we are using ipv6, required for upstash on fly.io
export const redis = remember("redis", () => {
return new Redis(process.env.REDIS_URL as string, { family: 6 });
});Now that we have a redis client, we can create a data interface for our users. We will create a new file, `user.ts`, and add the following code:
import { redis } from "./redis";
import {
createUserSchema,
CreateUserInput,
User,
updateUserSchema,
UpdateUserInput,
} from "./schema";
const userKey = (userId: string) => `user:${userId}`;
const userIdKey = (userString: string) => `userId:${userString}`;
export async function createUser(
userInfo: CreateUserInput
): Promise<UserInfo | null> {
const nameExists = await db.exists(userIdKey(userInfo.name));
if (nameExists) return null;
const emailExists = await db.exists(userIdKey(userInfo.email));
if (emailExists) return null;
const _userInfo = createUserSchema.parse(userInfo);
const userId = _userInfo.id;
await db.hset(userKey(userId), _userInfo);
await db.set(userIdKey(userInfo.name), userId);
await db.set(userIdKey(userInfo.email), userId);
const user = await getUser(userId);
console.log("created User", user?.id);
return user;
}This function creates a new user in the database. It first checks if the user already exists by checking if the name and email are already in use. If the user does not exist, it creates a new user with the provided information. It then returns the user. The reason why we are able to see if the user exists is because we have created an index for the user by name and email. And we did that with just a couple lines of code. We first define a key for the user, and then we define a key for the index. We then use the `hset` command to store the user in the database, and the `set` command to store the index.
Our getUser function is similarly simple:
// we don't want to return the password with any user queries.
export async function getUser(userId: string): Promise<UserInfo | null> {
const userInfo = (await db.hgetall(userKey(userId))) as UserInfo | null;
delete userInfo?.password;
return userInfo;
}And we can also easily get the user by name or email:
export async function getUserByName(name: string): Promise<User | null> {
const userId = await db.get(userIdKey(name));
if (!userId) return null;
return getUser(userId);
}
export async function getUserByEmail(email: string): Promise<User | null> {
const userId = await db.get(userIdKey(email));
if (!userId) return null;
return getUser(userId);
}
export const userExists = async (userId: string) => {
return await db.hexists(userKey(userId), "name");We can also update the user, though this requires a little more bookkeeping:
export async function updateUser(
userInfo: UpdateUserInput
): Promise<UpdateUserOutput> {
const updateUserInfo = updateUserSchema.parse(userInfo);
const userId = updateUserInfo.id;
const existingUser = await getUser(userId);
if (!existingUser) return { data: null, error: "user not found" };
if (userInfo.name && userInfo.name !== existingUser.name) {
if (await db.exists(userIdKey(userInfo.name)))
return { data: null, error: "username or email already in use" };
}
if (userInfo.email && userInfo.email !== existingUser.email) {
if (await db.exists(userIdKey(userInfo.email)))
return { data: null, error: "username or email already in use" };
}
if (userInfo.name) {
db.del(userIdKey(existingUser.name));
db.set(userIdKey(userInfo.name), userId);
}
if (userInfo.email) {
db.del(userIdKey(existingUser.email));
db.set(userIdKey(userInfo.email), userId);
}
await db.hset(userKey(userId), updateUserInfo);
return { data: await getUser(userId), error: null };
}This function updates a user in the database. It first checks if the user exists, and then checks if the new name or email is already in use. If the new name or email is not in use, it updates the user in the database, and updates the index. It then returns the updated user. Additionally, if the username or email changes, we have to remove the old index entries and add the new ones. This is a little more complex, but still quite simple.
The return type pattern is designed to be optimal for interacting with the front end. It is a common pattern to return an object with a data field and an error field. This allows the front end to easily check for errors and display them to the user.
This shows the basic concept of using redis as a primary data store. Next we will show how to connect this user to other tables in an efficient manner.
Next Part: The Chat Application
Coming Soon!
Dustin W. Carr
April 10, 2024
redis
database
data management
Summary:
Delve into the advantages of using Redis as a primary data store, emphasizing its simplicity and flexibility which empower developers to tailor database interactions specifically for their applications.
The reasons to use Redis as your primary data store are plentiful, and that is discussed in detail by many other articles on the web, such as this one . However, one of the most compelling reasons to use Redis is its simplicity and flexibility. What this enables is for you to truly get to know your data, so that you can understand just what it is that you need from your data store. What you may fi\nd is that the indexes and queries that SQL brings only amount to a few extra lines of code. The process of writing that code feels powerful, because there are no hidden interactions, and SQL is replaced with an extremly simple database interface that specializes in YOUR needs for YOUR app, not the general needs of every db programmer in the world.
In this article, I will walk you through the process of writing a simple data interface for Redis, and how to use it to build a simple chat application.
This is not an introduction to redis, or to typescript, or zod. Though it is a basic implementation that uses all of these. Just ask Chat for clarification on any of these topics. The important part is how to get applications written once you have these pieces in place.
The Data Interface
Any database definition requires a schema. In this case, we will use zod to define the schema. Zod is a typescript library that allows you to define schemas for your data. This is useful for many reasons, but in this case, it allows us to define the shape of our data, and to validate that the data we are storing is correct.
Let's demonstrate this with a simple user schema:
import { z } from "zod";
export enum Role {
ADMIN = "admin",
USER = "user",
}
export const createUserSchema = z.object({
id: z.string().default(() => uuidv4()),
name: z.string(),
email: z.string(),
password: z.string().optional(),
created: z.string().default(() => new Date().toISOString()),
updated: z.string().default(() => new Date().toISOString()),
lastLogin: z.string().default(() => new Date().toISOString()),
imageId: z.string().optional(),
imageUrl: z.string().optional(),
role: z.nativeEnum(Role).default(Role.USER),
});
export type CreateUserInput = z.infer<typeof createUserSchema>;
export type User = z.infer<typeof createUserSchema>;
This schema defines a user object, with a few fields. The `createUserSchema` is the schema that we will use to validate the data that we are storing. The `CreateUserInput` type is the type that we will use to create new users. The `User` type is the type that we will use to read users from the database. The ease with which we can generate complex inferred types with zod is one of the benefits of this entire approach. In fact, often developers have to write two schemas, one for the database definition and one for the application. With zod, we can write one schema and use it for both.
Also with zod we can introduce transforms and defaults that provide a great deal of extensibility. We use this to automatically set the timestamps, as well as provide default values for the role and id fields. This is a great way to ensure that the data is always in the correct shape. These are features that people rely upon with ORMs, but with zod, you can have them without the complexity of an ORM, but still have all the type safety.
We will want to have zod schemas for any sort of data query that we do. So for the UserSchema, we will also want to update the user with an arbitrary set of the User fields, but with the Id required.
export const updateUserSchema = createUserSchema
.omit({ id: true })
.partial()
.extend({
id: z.string(),
updated: z.string().default(() => new Date().toISOString()),
});
export type UpdateUserInput = z.infer<typeof updateUserSchema>;To create a database of users, we will be using io-redis, a redis client for node. We will also be using the redis hash data structure to store our users. The hash data structure is a key-value store, where the key is a string, and the value is a map of strings to strings. This is a perfect fit for our user schema.
import Redis from "ioredis";
import { remember } from "@epic-web/remember";
// assumes the environment variable REDIS_URL is set
// family: 6 means we are using ipv6, required for upstash on fly.io
export const redis = remember("redis", () => {
return new Redis(process.env.REDIS_URL as string, { family: 6 });
});Now that we have a redis client, we can create a data interface for our users. We will create a new file, `user.ts`, and add the following code:
import { redis } from "./redis";
import {
createUserSchema,
CreateUserInput,
User,
updateUserSchema,
UpdateUserInput,
} from "./schema";
const userKey = (userId: string) => `user:${userId}`;
const userIdKey = (userString: string) => `userId:${userString}`;
export async function createUser(
userInfo: CreateUserInput
): Promise<UserInfo | null> {
const nameExists = await db.exists(userIdKey(userInfo.name));
if (nameExists) return null;
const emailExists = await db.exists(userIdKey(userInfo.email));
if (emailExists) return null;
const _userInfo = createUserSchema.parse(userInfo);
const userId = _userInfo.id;
await db.hset(userKey(userId), _userInfo);
await db.set(userIdKey(userInfo.name), userId);
await db.set(userIdKey(userInfo.email), userId);
const user = await getUser(userId);
console.log("created User", user?.id);
return user;
}This function creates a new user in the database. It first checks if the user already exists by checking if the name and email are already in use. If the user does not exist, it creates a new user with the provided information. It then returns the user. The reason why we are able to see if the user exists is because we have created an index for the user by name and email. And we did that with just a couple lines of code. We first define a key for the user, and then we define a key for the index. We then use the `hset` command to store the user in the database, and the `set` command to store the index.
Our getUser function is similarly simple:
// we don't want to return the password with any user queries.
export async function getUser(userId: string): Promise<UserInfo | null> {
const userInfo = (await db.hgetall(userKey(userId))) as UserInfo | null;
delete userInfo?.password;
return userInfo;
}And we can also easily get the user by name or email:
export async function getUserByName(name: string): Promise<User | null> {
const userId = await db.get(userIdKey(name));
if (!userId) return null;
return getUser(userId);
}
export async function getUserByEmail(email: string): Promise<User | null> {
const userId = await db.get(userIdKey(email));
if (!userId) return null;
return getUser(userId);
}
export const userExists = async (userId: string) => {
return await db.hexists(userKey(userId), "name");We can also update the user, though this requires a little more bookkeeping:
export async function updateUser(
userInfo: UpdateUserInput
): Promise<UpdateUserOutput> {
const updateUserInfo = updateUserSchema.parse(userInfo);
const userId = updateUserInfo.id;
const existingUser = await getUser(userId);
if (!existingUser) return { data: null, error: "user not found" };
if (userInfo.name && userInfo.name !== existingUser.name) {
if (await db.exists(userIdKey(userInfo.name)))
return { data: null, error: "username or email already in use" };
}
if (userInfo.email && userInfo.email !== existingUser.email) {
if (await db.exists(userIdKey(userInfo.email)))
return { data: null, error: "username or email already in use" };
}
if (userInfo.name) {
db.del(userIdKey(existingUser.name));
db.set(userIdKey(userInfo.name), userId);
}
if (userInfo.email) {
db.del(userIdKey(existingUser.email));
db.set(userIdKey(userInfo.email), userId);
}
await db.hset(userKey(userId), updateUserInfo);
return { data: await getUser(userId), error: null };
}This function updates a user in the database. It first checks if the user exists, and then checks if the new name or email is already in use. If the new name or email is not in use, it updates the user in the database, and updates the index. It then returns the updated user. Additionally, if the username or email changes, we have to remove the old index entries and add the new ones. This is a little more complex, but still quite simple.
The return type pattern is designed to be optimal for interacting with the front end. It is a common pattern to return an object with a data field and an error field. This allows the front end to easily check for errors and display them to the user.
This shows the basic concept of using redis as a primary data store. Next we will show how to connect this user to other tables in an efficient manner.
Next Part: The Chat Application
Coming Soon!
© 2024 DarkViolet.ai All Rights Reserved