Forem Creators and Builders

Shreehari
Shreehari

Posted on

Selfhost forem

Hi, im trying to selfhost forem on AWS.
After configuring everything, im trying to run ansible-playbook -i inventory/forem/setup.yml playbooks/providers/aws.yml, but its giving me the following error:

❯ ansible-playbook -i inventory/forem/setup.yml playbooks/providers/aws.yml

ERROR! couldn't resolve module/action 'amazon.aws.ec2'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/home/sln/Downloads/selfhost/playbooks/providers/aws.yml': line 170, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


    - name: "Launch Forem instance for {{ app_domain }}"
      ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"
Enter fullscreen mode Exit fullscreen mode

The aws credentials are good. Not really sure what the issue.
In one of the issue i found that passing the vpc id might fix the issue and i tried. Still the same error occurs(My AWS account has only one VPC i.e, the default VPC). The IAM aswell has the fullaccess permissions - AmazonEC2FullAccess, AmazonS3FullAccess, AmazonVPCFullAccess.

Any help or sugggestions are appreciated.
Thank you

Discussion (9)

The discussion has been locked. New comments can't be added.
Collapse
ce7in profile image
Muhammed Cetin • Edited on

The values of the vault secrets in the setup.yml inventory must have indentations like the commented ones. Can you try after checking these values again.

Example:

An example vault secret

Collapse
gary profile image
Shreehari Author

Yes, i have added indentations.

Collapse
ce7in profile image
Muhammed Cetin • Edited on

However, the error persists.

The error message says the error appears to be in '/home/sln/Downloads/selfhost/playbooks/providers/aws.yml': line 170, column 7. Have you modified this file?

If you cannot find the error in this file, can you share the file with me?

gary profile image
Shreehari Author • Edited on

Yes. I modified this file and modified the following keys:

    fcos_aws_region: eu-west-1
    fcos_aws_size: t3.small
Enter fullscreen mode Exit fullscreen mode

This is file /home/sln/Downloads/selfhost/playbooks/providers/aws.yml

---
- name: Deploy Forem to AWS
  hosts: all
  become: false

  collections:
    - amazon.aws
    - community.aws
    - community.general

  vars:
    fcos_arch: x86_64
    fcos_platform: aws
    fcos_format: vmdk.xz
    fcos_stream: stable
    fcos_aws_region: eu-west-1
    fcos_aws_size: t3.small
    fcos_aws_ebs_size: 100
    fcos_aws_profile: forem-selfhost
    butane_cleanup: true
    ssh_key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"

  roles:
    - preflight

  tasks:
  - name: Get FCOS facts
    include_role:
      name: fcos
      tasks_from: facts

  - name: Convert butane file to an Ignition file
    include_role:
      name: butane
      tasks_from: butane
    vars:
      butane_input_template: "../templates/forem.yml.j2"
      butane_aws_s3: true
      butane_aws_s3_url: "https://forem-selfhost-{{ app_domain |replace('.', '-') }}-ign.s3.{{ fcos_aws_region }}.amazonaws.com/forem.ign"

  - amazon.aws.ec2_vpc_net_info:
      filters:
        "isDefault": "true"
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
    register: forem_vpc_info

  - name: Set forem_vpc_id fact
    ansible.builtin.set_fact:
      forem_vpc_id: "{{ forem_vpc_info['vpcs'][0]['vpc_id'] }}"

  - name: Gather info about VPC subnets
    amazon.aws.ec2_vpc_subnet_info:
      filters:
        vpc-id: "{{ forem_vpc_id }}"
        availability-zone: "{{ fcos_aws_region }}a"
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
    register: forem_subnet_info

  - name: Gather info about VPC AZs
    amazon.aws.aws_az_info:
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
    register: forem_az_info

  - name: "Get route table facts for {{ forem_vpc_id }}"
    community.aws.ec2_vpc_route_table_info:
      region: "{{ fcos_aws_region }}"
      filters:
        vpc-id: "{{ forem_vpc_id }}"
      profile: "{{ fcos_aws_profile }}"
    register: forem_vpc_route_table

  - name: "Generate list of route tables for {{ forem_vpc_id }}"
    set_fact:
      forem_vpcd_route_table_ids: "{{ forem_vpc_route_table.route_tables|map(attribute='id')|list }}"

  - name: "Create S3 VPC endpoint in {{ forem_vpc_id }}"
    community.aws.ec2_vpc_endpoint:
      state: present
      region: "{{ fcos_aws_region }}"
      vpc_id: "{{ forem_vpc_id }}"
      service: "com.amazonaws.{{ fcos_aws_region }}.s3"
      route_table_ids: "{{ forem_vpcd_route_table_ids }}"
      profile: "{{ fcos_aws_profile }}"
    register: forem_vpc_s3_endpoint

  - name: Set forem_vpc_s3_endpoint_id fact
    set_fact:
      forem_vpc_s3_endpoint_id: "{{ forem_vpc_s3_endpoint.result.vpc_endpoint_id }}"

  - name: Wait for S3 VPC Endpoint
    pause:
      seconds: 30

  - name: Create FCOS ignition bucket
    amazon.aws.s3_bucket:
      name: "forem-selfhost-{{ app_domain |replace('.', '-') }}-ign"
      state: present
      encryption: "AES256"
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
      policy: |
        {
          "Version": "2012-10-17",
          "Id": "VPCEaccesstoignitionbucket",
          "Statement": [
            {
              "Sid": "VPCE-access-to-ign-bucket",
              "Principal": "*",
              "Action": "s3:GetObject",
              "Effect": "Allow",
              "Resource": ["arn:aws:s3:::forem-selfhost-{{ app_domain |replace(".", "-") }}-ign/*"],
              "Condition": {
                "StringEquals": {
                  "aws:sourceVpce": "{{ forem_vpc_s3_endpoint_id }}"
                }
              }
            }
          ]
        }

  - name: "Upload butane_ignition_stdout to forem-selfhost-{{ app_domain |replace('.', '-') }}-ign"
    amazon.aws.aws_s3:
      bucket: "forem-selfhost-{{ app_domain |replace('.', '-') }}-ign"
      object: "/forem.ign"
      content: "{{ butane_ignition_stdout | to_json | string }}"
      mode: put
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
    register: forem_ign_s3

  - name: Create Forem SSH key
    amazon.aws.ec2_key:
      name: "forem-{{ app_domain }}"
      key_material: "{{ ssh_key }}"
      profile: "{{ fcos_aws_profile }}"
      region: "{{ fcos_aws_region }}"

  - name: "Create Forem security group for {{ app_domain }}"
    amazon.aws.ec2_group:
      name: "forem-{{ app_domain }}"
      description: "Forem security group for {{ app_domain }}"
      vpc_id: "{{ forem_vpc_id }}"
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
      tags:
        "Name": "forem-{{ app_domain }}"
      rules:
        - proto: tcp
          ports:
            - 22
          cidr_ip: "{{ local_wan_ip_address }}/32"
          rule_desc: "Allow SSH access from {{ local_wan_ip_address }}"
        - proto: tcp
          ports:
            - 80
            - 443
          rule_desc: "Allow HTTP and HTTPS access from 0.0.0.0/0"
          cidr_ip: 0.0.0.0/0
      rules_egress:
        - proto: "all"
          from_port: 0
          to_port: 65535
          cidr_ip: "0.0.0.0/0"
          rule_desc: "Allow outbound access to 0.0.0.0/0"
    register: forem_security_group

  - name: "Launch Forem instance for {{ app_domain }}"
    amazon.aws.ec2:
      key_name: "forem-{{ app_domain }}"
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"
      group: "forem-{{ app_domain }}"
      instance_type: "{{ fcos_aws_size }}"
      image: "{{ fcos_aws_image }}"
      wait: yes
      wait_timeout: 500
      vpc_subnet_id: "{{ forem_subnet_info.subnets | map(attribute='id') | list | first }}"
      volumes:
        - device_name: /dev/xvda
          volume_type: gp2
          volume_size: "{{ fcos_aws_ebs_size }}"
          encrypted: yes
          delete_on_termination: no
      monitoring: yes
      assign_public_ip: yes
      user_data: "{{ butane_boot_ignition_stdout | to_json | string }}"
      instance_tags:
        App: "forem"
        Domain: "{{ app_domain }}"
        Name: "forem-{{ app_domain }}"
      count_tag:
        App: "forem"
        Domain: "{{ app_domain }}"
        Name: "forem-{{ app_domain }}"
      exact_count: 1
    register: forem_ec2_instance

  - name: Wait 300 seconds for port 22 to become open
    wait_for:
      port: 22
      host: "{{ forem_ec2_instance.tagged_instances | map(attribute='public_ip') | list | first }}"
      delay: 10
    connection: local

  - name: "Delete object forem-selfhost-{{ app_domain |replace('.', '-') }}-ign/forem.ign from S3"
    amazon.aws.aws_s3:
      bucket: "forem-selfhost-{{ app_domain |replace('.', '-') }}-ign"
      object: "/forem.ign"
      mode: delobj
      region: "{{ fcos_aws_region }}"
      profile: "{{ fcos_aws_profile }}"

  - name: Output EC2 setup message
    ansible.builtin.debug:
      msg:
      - "The public IPv4 IP Address for {{ app_domain }} is {{ forem_ec2_instance.tagged_instances | map(attribute='public_ip') | list | first }}"
      - "Please add an A entry for {{ app_domain }} that points to {{ forem_ec2_instance.tagged_instances | map(attribute='public_ip') | list | first }}"
      - "Example:"
      - "    {{ app_domain }} IN A {{ forem_ec2_instance.tagged_instances | map(attribute='public_ip') | list | first }}"
      - "Once you have DNS resolving to this EC2 instance please read the Forem Admin Docs: https://admin.forem.com/"
Enter fullscreen mode Exit fullscreen mode
gary profile image
Shreehari Author

This is the output of the ansible-playbook command when run in verbose mode -

❯ ansible-playbook -i inventory/forem/setup.yml playbooks/providers/aws.yml -vvvvv

ansible-playbook [core 2.11.0] 
  config file = /home/sln/Downloads/selfhost/ansible.cfg
  configured module search path = ['/home/sln/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/sln/Downloads/venv-forem/lib/python3.8/site-packages/ansible
  ansible collection location = /home/sln/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/sln/Downloads/venv-forem/bin/ansible-playbook
  python version = 3.8.13 (default, Apr 19 2022, 02:32:06) [GCC 11.2.0]
  jinja version = 3.1.2
  libyaml = True
Using /home/sln/Downloads/selfhost/ansible.cfg as config file
Reading vault password file: /home/sln/.config/forem/selfhost_ansible_vault_password
setting up inventory plugins
Loading collection amazon.aws from /home/sln/.ansible/collections/ansible_collections/amazon/aws
Loading collection community.digitalocean from /home/sln/.ansible/collections/ansible_collections/community/digitalocean
redirecting (type: inventory) ansible.builtin.gcp_compute to google.cloud.gcp_compute
Loading collection google.cloud from /home/sln/.ansible/collections/ansible_collections/google/cloud
ansible_collections.amazon.aws.plugins.inventory.aws_ec2 declined parsing /home/sln/Downloads/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
Skipping due to inventory source file name mismatch. The file name has to end with one of the following: do_hosts.yaml, do_hosts.yml digitalocean.yaml, digitalocean.yml, digital_ocean.yaml, digital_ocean.yml.
ansible_collections.community.digitalocean.plugins.inventory.digitalocean declined parsing /home/sln/Downloads/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
ansible_collections.google.cloud.plugins.inventory.gcp_compute declined parsing /home/sln/Downloads/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
host_list declined parsing /home/sln/Downloads/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
script declined parsing /home/sln/Downloads/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
Skipping empty key (hosts) in group (all)
Parsed /home/sln/Downloads/selfhost/inventory/forem/setup.yml inventory source with yaml plugin
Loading collection community.aws from /home/sln/.ansible/collections/ansible_collections/community/aws
Loading collection community.general from /home/sln/Downloads/venv-forem/lib/python3.8/site-packages/ansible_collections/community/general
redirecting (type: modules) community.aws.ec2_vpc_route_table_info to amazon.aws.ec2_vpc_route_table_info
redirecting (type: modules) community.aws.ec2_vpc_endpoint to amazon.aws.ec2_vpc_endpoint
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
ERROR! couldn't resolve module/action 'amazon.aws.ec2'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/home/sln/Downloads/selfhost/playbooks/providers/aws.yml': line 170, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


  - name: "Launch Forem instance for {{ app_domain }}"
    ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"

Enter fullscreen mode Exit fullscreen mode
ce7in profile image
Muhammed Cetin

I couldn't find any problem in your aws.yml file. However, the problem might be in the setup.yml file. There should be a misspelling.

You can follow this path to catch the error:

  1. Use the original aws.yml file first. If everything is OK, the error is in this file.
  2. Otherwise, the problem is probably in the setup.yml file; examine it. Vault secrets, domain names, or other settings might be wrong.
  3. If you wouldn't be able to find any problem within setup.yml, think if you have changed anything else. Follow the changes.
gary profile image
Shreehari Author

Thanks for the help. Will try those. Since im trying forem.dev for the first time, i am facing such issues.

gary profile image
Shreehari Author

Hi Muhammed Cetin
I tried the used original aws.yml file but still getting the same error.

I am using ubuntu to deploy forem. According the the docs, i need to install butane, but not able to install butane in ubuntu. Is there any way to install it or other ways to delploy with production settings?

I tried the development version of the forem by installing ruby and other packages locally, it worked fine and was able to run it successfully.

Please do advice the process to deploy forem from ubuntu is different or do i need to stick to the developer documentation.

gary profile image
Shreehari Author

Was able to fix those errors after updating the aws.yml and referred this while updating - docs.ansible.com/ansible/latest/co...