Forem Creators and Builders

loading...

How stop, start, restart or get the status of various services for your Forem server

Andrew Brown
・6 min read

TL;DR: You probably never need to touch systemd services and you should really use the ForemCTL instead

When you have a program or application you want to run on Linux, its a common practice to create a systems service which is a fancy script that allows you to start, stop, restart or get the status of a service.

A service just means a program or app that runs in the background awaiting to accept work to do.

Selfhost uses Fedora CoreOS (FCOS) and FCOS uses SystemD.

What is SystemD?

SystemD is a System and service manager. You can define a service with a .service file and then can perform tasks like:

sudo systemctl restart httpd.service
sudo systemctl stop httpd.service
sudo systemctl start httpd.service
sudo systemctl status httpd.service
Enter fullscreen mode Exit fullscreen mode

Where are the SystemD services defined?

Within selfhost if you open up the buntane file at playbooks/templates/forem.yml.j2

You can see the SystemD services defined:

systemd:
  units:
  - name: forem-pod.service
    enabled: true
    contents: |
      [Unit]
      Description=Forem pod service
      Wants=network.target
      After=network-online.target forem-container.service
      Wants=forem-postgresql.service forem-redis.service forem.service forem-traefik.service
      Before=forem-postgresql.service forem-redis.service forem-openresty.service forem.service

      [Service]
      LimitNOFILE=1048576
      Environment=PODMAN_SYSTEMD_UNIT=%n
      Restart=on-failure
      ExecStartPre=-/usr/bin/podman pod create \
        --infra-conmon-pidfile %T/%N.pid \
        --name %N \
        -p 443:443 -p 80:80
      ExecStart=/usr/bin/podman pod start %N
      ExecStop=/usr/bin/podman pod stop -t 10 %N
      ExecStopPost=/usr/bin/podman pod stop -t 10 %N
      ExecStopPost=/usr/bin/podman pod rm %N
      PIDFile=%T/%N.pid
      KillMode=none
      Type=forking
      SyslogIdentifier=%N

      [Install]
      WantedBy=multi-user.target default.target
Enter fullscreen mode Exit fullscreen mode

How to view all services on your Forem server

There can be many services on your Forem server, and easy way to see a full of them is the following command:

list-units --type=service
Enter fullscreen mode Exit fullscreen mode
UNIT                                                        LOAD   ACTIVE SUB     DESCRIPTION
  chronyd.service                                             loaded active running NTP client/server
  console-login-helper-messages-gensnippet-os-release.service loaded active exited  Generate os release snippet for display via console-login-helper-messages
  console-login-helper-messages-gensnippet-ssh-keys.service   loaded active exited  Generate SSH keys snippet for display via console-login-helper-messages
  dbus-broker.service                                         loaded active running D-Bus System Message Bus
  dracut-shutdown.service                                     loaded active exited  Restore /run/initramfs on shutdown
  fcos-python-3.service                                       loaded active exited  FCOS Python 3 service
  fedora-coreos-pinger.service                                loaded active exited  Telemetry service for Fedora CoreOS
  forem-imgproxy.service                                      loaded active running Forem Imgproxy Service
  forem-openresty.service                                     loaded active running Forem OpenResty Service
  forem-pod.service                                           loaded active running Forem pod service
  forem-postgresql.service                                    loaded active running Forem Postgresql Service
  forem-rails.service                                         loaded active running Forem Rails Service
  forem-redis.service                                         loaded active running Forem Redis Service
  forem-traefik.service                                       loaded active running Forem Traefik Service
  forem-worker.service                                        loaded active running Forem Worker Service
  forem.service                                               loaded active exited  Forem Service
  getty@tty1.service                                          loaded active running Getty on tty1
  irqbalance.service                                          loaded active running irqbalance daemon
  kmod-static-nodes.service                                   loaded active exited  Create List of Static Device Nodes
  ldconfig.service                                            loaded active exited  Rebuild Dynamic Linker Cache
  lvm2-monitor.service                                        loaded active exited  Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling
  NetworkManager-wait-online.service                          loaded active exited  Network Manager Wait Online
  NetworkManager.service                                      loaded active running Network Manager
  ostree-remount.service                                      loaded active exited  OSTree Remount OS/ Bind Mounts
  polkit.service                                              loaded active running Authorization Manager
  serial-getty@ttyS0.service                                  loaded active running Serial Getty on ttyS0
  sshd.service                                                loaded active running OpenSSH server daemon
  sssd.service                                                loaded active running System Security Services Daemon
  systemd-fsck@dev-disk-by\x2dlabel-boot.service              loaded active exited  File System Check on /dev/disk/by-label/boot
  systemd-homed-activate.service                              loaded active exited  Home Area Activation
  systemd-homed.service                                       loaded active running Home Area Manager
  systemd-hwdb-update.service                                 loaded active exited  Rebuild Hardware Database
  systemd-journal-catalog-update.service                      loaded active exited  Rebuild Journal Catalog
  systemd-journal-flush.service                               loaded active exited  Flush Journal to Persistent Storage
  systemd-journald.service                                    loaded active running Journal Service
Enter fullscreen mode Exit fullscreen mode

You can see there are alot of services. Most of these came with Fedora CoreOS.

The ones that were setup by forem all start with forem- so we can use grep to just see these services:

systemctl list-units --type=service | grep forem
Enter fullscreen mode Exit fullscreen mode

which will produce:

  forem-imgproxy.service                                      loaded active running Forem Imgproxy Service
  forem-openresty.service                                     loaded active running Forem OpenResty Service
  forem-pod.service                                           loaded active running Forem pod service
  forem-postgresql.service                                    loaded active running Forem Postgresql Service
  forem-rails.service                                         loaded active running Forem Rails Service
  forem-redis.service                                         loaded active running Forem Redis Service
  forem-traefik.service                                       loaded active running Forem Traefik Service
  forem-worker.service                                        loaded active running Forem Worker Service
  forem.service                                               loaded active exited  Forem Service
Enter fullscreen mode Exit fullscreen mode

When would we want to mess with SystemD services?

There are rare cases where you need to go in and restart a service, but generally starting, stoping and restarting services would be if you had a deployment pipeline.

If you need to do a sanity check to make sure your Rails, Postgres, Redis or Sidekiq service is running you would check its status:

sudo systemctl status forem-rails
Enter fullscreen mode Exit fullscreen mode
● forem-rails.service - Forem Rails Service
     Loaded: loaded (/etc/systemd/system/forem-rails.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-07-22 19:48:40 UTC; 38min ago
    Process: 2955 ExecStartPre=/usr/bin/podman image exists localhost/forem/forem:current (code=exited, status=0/SUCCESS)
    Process: 2985 ExecStartPre=/usr/bin/podman run --name forem-rails-bootstrap --pod forem-pod --log-driver journald --replace --rm --sdnotify ignore --env-file /opt/forem/env>
   Main PID: 3132 (conmon)
      Tasks: 2 (limit: 2239)
     Memory: 52.6M
        CPU: 3.637s
     CGroup: /machine.slice/machine-forem.slice/machine-forem-pod.slice/forem-rails.service
             └─3132 /usr/bin/conmon --api-version 1 -c d7bdd1f6baae5ee03b2798c91ccf9795d04e51a53bf0293968461143fa470253 -u d7bdd1f6baae5ee03b2798c91ccf9795d04e51a53bf0293968461>

Jul 22 20:07:10 www.thedev.cloud conmon[3132]: I, [2021-07-22T20:07:10.242930 #7]  INFO -- : [191d47a8-969e-4031-830a-a635506fbc38]   Parameters: {"locale"=>nil}
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: I, [2021-07-22T20:07:10.292204 #7]  INFO -- : [191d47a8-969e-4031-830a-a635506fbc38]   Rendered devise/registrations/new.html.erb>
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: I, [2021-07-22T20:07:10.564313 #7]  INFO -- : [191d47a8-969e-4031-830a-a635506fbc38]   Rendered layout layouts/application.html.e>
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: I, [2021-07-22T20:07:10.564739 #7]  INFO -- : [191d47a8-969e-4031-830a-a635506fbc38] Filter chain halted as :verify_private_forem>
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: I, [2021-07-22T20:07:10.565142 #7]  INFO -- : [191d47a8-969e-4031-830a-a635506fbc38] Completed 200 OK in 322ms (Views: 295.1ms | >
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: I, [2021-07-22T20:07:10.597743 #8]  INFO -- : [ca2ed591-22bf-4516-91f8-1d4040e16114] Started GET "/sitemaps/sitemap.xml.gz" for 6>
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: F, [2021-07-22T20:07:10.640000 #8] FATAL -- : [ca2ed591-22bf-4516-91f8-1d4040e16114]
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: [ca2ed591-22bf-4516-91f8-1d4040e16114] ActionController::RoutingError (No route matches [GET] "/sitemaps/sitemap.xml.gz"):
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: [ca2ed591-22bf-4516-91f8-1d4040e16114]
Jul 22 20:07:10 www.thedev.cloud conmon[3132]: E, [2021-07-2
Enter fullscreen mode Exit fullscreen mode

Forem Control is way easier to use

The most common tasks related to services would be using foremctl

$ foremctl help
Usage: foremctl {console|deploy|help|rake|restart|start|stat|status|stop|update|version}

console         Open a Rails console
deploy          Updates and deploy the most current version of Forem
help            Show this message
rake            Run a rake task
restart         Restart Forem
start           Start Forem
stats           Show CPU, RAM, Disk IO usage of the Forem containers
status          Show the current running Forem containers
stop            Stop Forem
update          Updates Forem to the lastest container
version         Shows information on the current running version of Forem
Enter fullscreen mode Exit fullscreen mode

To get a bunch of stats of current services

$ foremctl stats
ID            NAME                CPU %   MEM USAGE / LIMIT  MEM %   NET IO             BLOCK IO           PIDS
29a1cb7fe232  forem-imgproxy      --      42.04MB / 2.039GB  2.06%   77.78kB / 125.2kB  40.1MB / 0B        8
2fa68d8d5c2b  forem-openresty     --      11.33MB / 2.039GB  0.56%   77.78kB / 125.2kB  8.413MB / 12.29kB  4
4e2db20b2690  forem-traefik       --      16.8MB / 2.039GB   0.82%   77.78kB / 125.2kB  8.192kB / 20.48kB  10
94977d75c1ce  forem-postgresql    --      85.6MB / 2.039GB   4.20%   77.78kB / 125.2kB  36.28MB / 68.75MB  8
9bc4592eeb54  forem-redis         0.23%   11.11MB / 2.039GB  0.54%   77.78kB / 125.2kB  11.38MB / 8.914MB  6
d7bdd1f6baae  forem-rails         0.01%   411.5MB / 2.039GB  20.18%  77.78kB / 125.2kB  29.72MB / 9.077MB  29
e5712942bcb9  ff711bf9eef1-infra  --      749.6kB / 2.039GB  0.04%   77.78kB / 125.2kB  675.8kB / 0B       1
e990ee4624ea  forem-worker        7.22%   345.8MB / 2.039GB  16.96%  77.78kB / 125.2kB  4.289MB / 7.807MB  10
Enter fullscreen mode Exit fullscreen mode

Conclusion

You probably never want to touch these, in case of rare debugging cases.

Discussion (0)