The start of building a better system

I have been heads down and knee deep in my work for the past three months since joining DEV (now Forem) and what you see here is the just the start of us building a better system for running Forem. Like any new project, it's been a classic Yak Shave.

I wanted to drop a quick note to highlight some of the things we have done to make forem.dev a thing and to kickstart the conversation about the design and operation of Forem from a systems perspective.

So let's get to it!

Forem Systems Engineering has some high level principles that guide our work:

• Open Source
• Reproducible
• Secure
• Persistent
• Portable
• Available

We are trying to meet these principles every day when building the systems that will power Forem. Building a better system starts from the the top down. Each principle builds off of the other. In order to provide any kind of available service on the Internet you need to have a portable, persistent, secure, and reproducible setup built on Free Open Source Software. I will make another post that expand on these on these principles down the road.

To quickly go over the current forem.dev stack, it is running on AWS right now and not Heroku which still powers dev.to. We are using Terraform and Ansible to manage the cloud infrastructure and Fedora CoreOS for the host VM which is using Podman to run the Forem container stack. You can see some of the Forem specific containers that we are building on our Quay.io account.

We have two data stores, S3 for images and a Postgresql RDS managed by AWS. Since we are focused on building a minimal and viable stack for Forem users, we are not using any kind of high availability strategies for our setup right now. Most users that run Forem on their own won't need those features just yet anyways. The goal here is to get this off the ground and iterate towards a better system.

In the future, we want to expand to other cloud providers such as Google Compute, and Digital Ocean. We even want to build our systems with self-hosters in mind. This would enable you to easily run Forem in your basement on that old computer that is mostly used for BitTorrents and side projects or on your own cloud account. We also want to build system specific features so you can export your data for safekeeping or run in a highly available setup.

There are a lot of things that are inflight and changing every week. To be honest, things are pretty unstable right now. I expect that forem.dev will encounter downtime and issues as we hack on our systems goals.

We'd rather embrace these challenges in the open vs building in a bubble of closed source. Once we have things a bit more stable, we want to open source this work, so you can host your own Forem easily and help us build the systems that power Forem together.