Forem Creators and Builders 🌱

Joe Doss
Joe Doss

Posted on • Updated on

The start of building a better system

I have been heads down and knee deep in my work for the past three months since joining DEV (now Forem) and what you see here is the just the start of us building a better system for running Forem. Like any new project, it's been a classic Yak Shave.

Molly: Hey Joe you got that Vault server online yet??

I wanted to drop a quick note to highlight some of the things we have done to make forem.dev a thing and to kickstart the conversation about the design and operation of Forem from a systems perspective.

So let's get to it!

Forem Systems Engineering has some high level principles that guide our work:

  • Open Source
  • Reproducible
  • Secure
  • Persistent
  • Portable
  • Available

We are trying to meet these principles every day when building the systems that will power Forem. Building a better system starts from the the top down. Each principle builds off of the other. In order to provide any kind of available service on the Internet you need to have a portable, persistent, secure, and reproducible setup built on Free Open Source Software. I will make another post that expand on these on these principles down the road.

To quickly go over the current forem.dev stack, it is running on AWS right now and not Heroku which still powers dev.to. We are using Terraform and Ansible to manage the cloud infrastructure and Fedora CoreOS for the host VM which is using Podman to run the Forem container stack. You can see some of the Forem specific containers that we are building on our Quay.io account.

We have two data stores, S3 for images and a Postgresql RDS managed by AWS. Since we are focused on building a minimal and viable stack for Forem users, we are not using any kind of high availability strategies for our setup right now. Most users that run Forem on their own won't need those features just yet anyways. The goal here is to get this off the ground and iterate towards a better system.

In the future, we want to expand to other cloud providers such as Google Compute, and Digital Ocean. We even want to build our systems with self-hosters in mind. This would enable you to easily run Forem in your basement on that old computer that is mostly used for BitTorrents and side projects or on your own cloud account. We also want to build system specific features so you can export your data for safekeeping or run in a highly available setup.

There are a lot of things that are inflight and changing every week. To be honest, things are pretty unstable right now. I expect that forem.dev will encounter downtime and issues as we hack on our systems goals.

This is fine...

We'd rather embrace these challenges in the open vs building in a bubble of closed source. Once we have things a bit more stable, we want to open source this work, so you can host your own Forem easily and help us build the systems that power Forem together.

Top comments (11)

Collapse
 
vaidehijoshi profile image
Vaidehi Joshi

I'm excited to see this come together!! It has been really fun to be a (small) part of building this architecture from the ground up :)

Collapse
 
lee profile image
Lee

Brilliant. Lot’s to learn just from this one post 😎

Collapse
 
lee profile image
Lee

Hey Joe. Is this available to deploy to AWS now?

Collapse
 
jdoss profile image
Joe Doss

Not yet Lee. Once we have all the kinks worked out we'll release the automation for launching Forems on AWS.

Collapse
 
lee profile image
Lee

Thanks Joe, it sounds amazing actually. Do you have a timeline?

Thread Thread
 
jdoss profile image
Joe Doss

I don't have an official timeline as of now. There is still a lot of work to be done before it is ready to be opensourced.

Soon...

Thread Thread
 
lee profile image
Lee

πŸ’ͺ🏽

Collapse
 
volanar profile image
volanar

The idea of forem is to combine many small communities into one global community, or does each instance of forem work independently?

Collapse
 
jdoss profile image
Joe Doss

From my perspective in Systems each Forem works independently right now. We have other initiatives app side that will help be a directory of communities through forem.com

Collapse
 
volanar profile image
volanar

The business model of monetization will be similar to the forum discourse? Do you have plans to launch a cloud solution for customers?

Thread Thread
 
jdoss profile image
Joe Doss

I can't comment on the business model myself. Maybe @ben or @peter can expand on that more for you. I am currently working on a cloud solution or customers as my primary role at Forem.