Feb 06, 2023
I dedicate my weekends to discovering new technologies and expanding my skill set. This curiosity led me to explore Cloudflare Workers and their recently launched D1 service - a serverless SQLite database. D1 is still in its alpha phase and documentation is sparse and frequently updating. Nonetheless, I believe that Cloudflare Workers and D1 have the potential to play a role the coming cloud-hosted, multi-tenant version of the Raytha platform. That's why I believe it's important to develop a solid understanding of how these technologies work.
As always, I tie the exploration of a product or service with a fun one-day build. This time I planned to build an API that allows bots to post tweets, but makes it extremely inconvenient for a human to do the same. This would be a twitter, but for bots only.
As always, I tie the exploration of a product or service with a fun one-day build. This time I planned to build an API that allows bots to post tweets, but makes it extremely inconvenient for a human to do the same. This would be a twitter, but for bots only.
Botter
We will call this little project "Botter", and it has two requirements:
- Bots must compete with each other.
- There must be some mathematical operation that a human would struggle with to compute quickly.
I decided to go with one tweet posted every 10 seconds and bots must compete to be able to post that tweet. And for the second requirement, I have chosen to go with prime factorization. A bot must generate the prime factors of a challenge posed by the platform.
API Endpoints
- [GET] https://botter.raytha.app
- [GET] https://botter.raytha.app/latest
- [PUT] https://botter.raytha.app/:id
The way it works is that you can make a GET request to the base route to see the 25 most recent posts by bots. The most recent post, which can also be grabbed by doing a GET request to /latest will be the challenge to solve. If the `completed_at` property is null, then you know that the challenge has not yet been solved. Grab the `id` property and `challenge` property.
Compute the prime factors of the `challenge` property and make a PUT request to /:id where :id is the `id` property from the previous GET request. The JSON payload in your PUT request should resemble:
{ "text": "your tweet message", "handle": "your_twitter_handle", "primes": [1, 2, 3] }The `primes` property should be an array of the prime factors from the challenge.
If you win the challenge, your tweet will appear at the top of https://botter.raytha.app. Otherwise, you will have to wait until the next challenge is posted.
Can you write a script to do this? I include a python script below, but see if you can solve it on your own first.
Cloudflare Worker and D1
Cloudflare Worker API side was achieved with itty-router-extras, which allows you to have multiple routes on a single Cloudflare Worker. Incredibly convenient. It also provides helper utilities for returning JSON and reading the body of request content.
It occurred to me that I did not want to verify that every value submitted by the consumer bots is a valid prime number. So I cheated and limited to the first 25 prime numbers in our numerical system.
The Cloudflare Worker script is as follows:
It occurred to me that I did not want to verify that every value submitted by the consumer bots is a valid prime number. So I cheated and limited to the first 25 prime numbers in our numerical system.
The Cloudflare Worker script is as follows:
import {
json,
missing,
error,
status,
withContent,
withParams,
ThrowableRouter,
} from 'itty-router-extras'
const primes = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
// create an error-safe itty router
const router = ThrowableRouter({ base: '/' })
router.get('/latest', async ({ }, env) => {
const stmt = env.DB.prepare(`SELECT * FROM posts ORDER BY created_at DESC LIMIT 1;`)
const item = await stmt.first();
return json(item)
})
router.get('/', async ({ }, env) => {
const stmt = env.DB.prepare(`SELECT * FROM posts ORDER BY created_at DESC LIMIT 25;`)
const items = await stmt.all();
return json(items.results)
})
router.put('/:id', withParams, withContent, async ({ id, content }, env) => {
console.log(content)
if (!content) {
return error(400, { "error": "Malformed request, check your JSON body content."})
}
if (!('primes' in content)) {
return error(400, { "error": "'primes' property missing from payload."})
}
if (!('text' in content)) {
return error(400, { "error": "'text' property missing from payload."})
}
if (!('handle' in content)) {
return error(400, { "error": "'handle' property missing from payload."})
}
if (!Array.isArray(content.primes)) {
return error(400, { "error": "'primes' property must be an array of prime numbers" })
}
if (content.primes.length < 1 || content.primes.length > 10) {
return error(400, { "error": "'primes' property must be an array of length between 1 and 10 inclusive" })
}
let isValid = true
content.primes.forEach(element => {
if (!primes.includes(element)) {
isValid = false
}
})
if (!isValid) {
return error(400, { "error": "'primes' is not a valid collection of numbers" })
}
const stmt = env.DB.prepare(`SELECT * FROM posts WHERE id = ?1 LIMIT 1;`).bind(id)
const item = await stmt.first();
if (item.completed_at != undefined) {
return error(403, { "error": "Someone beat you to the punch on this one. Try on the next one." })
}
let selectedValue = content.primes.reduce((a, b) => a * b)
if (selectedValue == item.challenge) {
const info = await env.DB.prepare('UPDATE posts SET text = ?1, completed_at = ?2, handle = ?3 WHERE id = ?4;')
.bind(content.text, Math.round(Date.now() / 1000), content.handle, id)
.run()
}
else {
return error(403, { "error": `Your computed primes came to: ${selectedValue} which is not equal to ${item.challenge}` })
}
return json({ success: true, message: "Contrats! You won."})
})
// POST to the collection
router.post('/', async (request, env) => {
if (!request.headers.has("X-API-KEY") || request.headers.get("X-API-KEY") != env.SECRET_KEY) {
return error(403, { "error": "Unauthorized access" })
}
const stmt = env.DB.prepare(`SELECT * FROM posts ORDER BY created_at DESC LIMIT 1;`)
const item = await stmt.first();
if (item != undefined && item.completed_at == undefined) {
return error(401, { "error": "Nobody solved the most recent one yet." })
}
let numberOfPrimesToUse = Math.floor(Math.random() * 10)
if (numberOfPrimesToUse < 2) {
numberOfPrimesToUse = 2
}
let primesToUse = []
for (let i = 0; i < numberOfPrimesToUse; i++) {
let index = 1 + Math.floor(Math.random() * primes.length-1)
primesToUse.push(primes[index])
}
let selectedValue = primesToUse.reduce((a, b) => a * b)
const info = await env.DB.prepare('INSERT INTO posts (challenge, created_at) VALUES (?1, ?2)')
.bind(selectedValue, Math.round(Date.now() / 1000))
.run()
return info.success
? status(204) // send a 204 no-content response
: error(400, info)
})
// Clear everything
router.post('/clear', async (request, env) => {
if (!request.headers.has("X-API-KEY") || request.headers.get("X-API-KEY") != env.SECRET_KEY) {
return error(403, { "error": "Unauthorized access" })
}
const info = await env.DB.prepare('DELETE FROM posts;').run()
return info.success
? status(200)
: error(400, info)
})
// 404 for everything else
router.all('*', () => missing('Are you sure about that?'))
// attach the router "handle" to the event handler
export default {
fetch: router.handle
}and the Cloudflare D1 .sql script to create the table is:
CREATE TABLE posts (
id INTEGER NOT NULL PRIMARY KEY,
challenge INTEGER NOT NULL,
handle TEXT,
created_at INTEGER NOT NULL,
completed_at INTEGER,
text TEXT
)Bot Script in Python
The project would not be complete if it did not include a script to test the API's functionality.
import requests, time, json
BASE_URL = "https://botter.raytha.app"
def get_latest():
response = requests.get(BASE_URL + "/latest")
item = response.json()
return item
def post_challenge(id, prime_factors, text, handle):
response = requests.put(BASE_URL + "/" + str(id), json={ "text": text, "handle": handle, "primes": prime_factors })
item = response.json()
print(item)
return response.status_code == 200
def compute_prime_factors(n):
i = 2
factors = []
while i * i <= n:
if n % i:
i += 1
else:
n //= i
factors.append(i)
if n > 1:
factors.append(n)
return factors
def main(handle, text):
num_runs = 0
num_wins = 0
while True:
latest_challenge = get_latest()
if not latest_challenge['completed_at']:
prime_factors = compute_prime_factors(latest_challenge['challenge'])
win_status = post_challenge(latest_challenge['id'], prime_factors, text, handle)
if win_status:
num_wins += 1
time.sleep(1)
num_runs += 1
if num_runs % 10 == 0:
print("Number of times run: " + str(num_runs))
print("Number of wins: " + str(num_wins))
if __name__ == '__main__':
handle = "raythahq"
text = "I am the prime factor man"
main(handle, text)This script was pretty simple, but in the main() method, get the latest challenge and see if the completed_at attribute has a value. If not, compute the primes, and then make a PUT request to the API and hope for the best. Run it in a loop and sleep every second.
Conclusion
The overall developer experience for Cloudflare Workers and D1 was surprisingly decent. I was impressed by the speed at which I could make changes to the script and see the results almost instantly. Although documentation could be improved, I understand that D1 is a new product and improvements in this area will likely come with time.
In terms of the potential role that Cloudflare Workers and D1 could play in the Raytha platform, one possibility is to use them to retrieve customer information associated with a requested domain before routing the request to the Azure Kubernetes Cluster. This lookup could be performed using either D1 or the Workers KV product, and would allow the headers of the request to be updated with customer information before it is passed on to the hosted application. The exact approach has yet to be determined.
Zack Schwartz @apexdodge
ENTREPRENEUR & SOFTWARE ENGINEER, AUSTIN, TX
I enjoy tackling a wide array of business challenges ranging from front line product support and operations to sales and marketing efforts. My core expertise is in software development building enterprise level web applications.
I enjoy tackling a wide array of business challenges ranging from front line product support and operations to sales and marketing efforts. My core expertise is in software development building enterprise level web applications.