Hey all! I'm trying to get started with Fleet, and...
# fleet
r
Hey all! I'm trying to get started with Fleet, and I'm following the instructions here: https://fleetdm.com/docs/deploy/deploy-on-aws-with-terraform However, this doesn't work:
Copy code
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

module "fleet" {
  source = "<http://github.com/fleetdm/fleet//terraform?ref=main|github.com/fleetdm/fleet//terraform?ref=main>"
}
Because the "fleet" module requires
certificate_arn
, which then leads to something more like this:
Copy code
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.39"
    }
  }
}

locals {
  fleet_domain_name = "<http://fleet.ryancammer.com|fleet.ryancammer.com>"
  fleet_image = "fleetdm/fleet:v4.46.1"
}

module "acm" {
  source  = "terraform-aws-modules/acm/aws"
  version = "~> 5.0"

  domain_name = local.fleet_domain_name
  zone_id     = data.aws_route53_zone.main.id

  wait_for_validation = true
}

module "fleet" {
  source = "<http://github.com/fleetdm/fleet//terraform?ref=main|github.com/fleetdm/fleet//terraform?ref=main>"
  certificate_arn = module.acm.acm_certificate_arn
}

data "aws_route53_zone" "main" {
  name         = local.fleet_domain_name
  private_zone = false
}
And that results in this error:
```Plan: 79 to add, 0 to change, 0 to destroy.
│ Error: Invalid for_each argument
│ on .terraform/modules/fleet/terraform/byo-vpc/main.tf line 69, in data "aws_subnet" "redis":
│ 69: for_each = toset(var.redis_config.subnets)
│ ├────────────────
│ │ var.redis_config.subnets is list of string with 3 elements
│ The "for_each" set includes values derived from resource attributes that cannot be determined until apply, and so
│ Terraform cannot determine the full set of keys that will identify the instances of this resource.
│ When working with unknown values in for_each, it's better to use a map value where the keys are defined statically in your
│ configuration and where only the values contain apply-time results.
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends
│ on, and then apply a second time to fully converge.```
I fought with it for a bit, defining a VPC, and even setting the values for the redis subnets, to no avail. 1. Is this the right channel for this? 2. Does anyone have a working terraform file to get started with? Thank you in advance! I appreciate the help.
b
hey Ryan. I have seen this error if you attempt to apply terraform before the underlying resources are created. It's kinda goofy, but terraform is trying its best to give you a hint at what to do:
Copy code
Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends
Try running this in the following order:
Copy code
terraform apply -target module.fleet.module.vpc
terraform apply -target module.fleet
terraform apply
this will build out the VPC, then the other resources like RDS MySQL & Redis, then finally the application stack.
it does seem like the guide is missing the middle apply, I'll submit a PR to amend that issue.
r
All right! That definitely worked. Now I just need to figure out why it's 503ing 504ing now no 503ing again on me...
Ah! This is why. I'm figuring this out now.
b
Try using the migrations module mentioned in the guide:
Copy code
module "migrations" {
  source                   = "<http://github.com/fleetdm/fleet//terraform/addons/migrations?ref=main|github.com/fleetdm/fleet//terraform/addons/migrations?ref=main>"
  ecs_cluster              = module.fleet.byo-vpc.byo-db.byo-ecs.service.cluster
  task_definition          = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.family
  task_definition_revision = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.revision
  subnets                  = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].subnets
  security_groups          = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].security_groups
}
r
I did, and I'm running into issues. The first is that these arguments appear to be required:
Copy code
module "migrations" {
  source                   = "github.com/fleetdm/fleet//terraform/addons/migrations?ref=main"
  ecs_cluster              = module.fleet.byo-vpc.byo-db.byo-ecs.service.cluster
  task_definition          = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.family
  task_definition_revision = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.revision
  subnets                  = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].subnets
  security_groups          = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].security_groups
  desired_count            = 1
  ecs_service              = "dummy_service"
  min_capacity             = 1
}
Copy code
desired_count            = 1
  ecs_service              = "dummy_service"
  min_capacity             = 1
That causes this error:
Copy code
module.migrations.null_resource.main: Provisioning with 'local-exec'...
module.migrations.null_resource.main (local-exec): Executing: ["/bin/sh" "-c" "/bin/bash .terraform/modules/migrations/terraform/addons/migrations/migrate.sh REGION=us-east-2 ECS_CLUSTER=fleet TASK_DEFINITION=fleet TASK_DEFINITION_REVISION=1 SUBNETS=[\"subnet-0959f4f9dc4ad5ffa\",\"subnet-09ae5664431607fdc\",\"subnet-0ec3b786fc5fee397\"] SECURITY_GROUPS=[\"sg-0fb3d165c137ac2e0\"] ECS_SERVICE=dummy_service MIN_CAPACITY=1 DESIRED_COUNT=1"]

module.migrations.null_resource.main (local-exec): An error occurred (ValidationException) when calling the RegisterScalableTarget operation: Maximum capacity must be specified
╷
│ Error: local-exec provisioner error
│
│   with module.migrations.null_resource.main,
│   on .terraform/modules/migrations/terraform/addons/migrations/main.tf line 7, in resource "null_resource" "main":
│    7:   provisioner "local-exec" {
│
│ Error running command '/bin/bash .terraform/modules/migrations/terraform/addons/migrations/migrate.sh REGION=us-east-2
│ ECS_CLUSTER=fleet TASK_DEFINITION=fleet TASK_DEFINITION_REVISION=1
│ SUBNETS=["subnet-0959f4f9dc4ad5ffa","subnet-09ae5664431607fdc","subnet-0ec3b786fc5fee397"]
│ SECURITY_GROUPS=["sg-0fb3d165c137ac2e0"] ECS_SERVICE=dummy_service MIN_CAPACITY=1 DESIRED_COUNT=1': exit status 254.
│ Output:
│ An error occurred (ValidationException) when calling the RegisterScalableTarget operation: Maximum capacity must be
│ specified
│
Screenshot 2024-03-01 at 6.37.28 PM.png
I thought maybe by specifying 0 for
desired_count
and `min_capacity`:
Copy code
module "migrations" {
  source                   = "<http://github.com/fleetdm/fleet//terraform/addons/migrations?ref=main|github.com/fleetdm/fleet//terraform/addons/migrations?ref=main>"
  ecs_cluster              = module.fleet.byo-vpc.byo-db.byo-ecs.service.cluster
  task_definition          = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.family
  task_definition_revision = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.revision
  subnets                  = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].subnets
  security_groups          = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].security_groups
  desired_count            = 0
  ecs_service              = "dummy_service"
  min_capacity             = 0
}
But nope:
I'm modifying the script locally and I pointed at my local directory:
Copy code
module "migrations" {
  source                   = "../../fleet/terraform/addons/migrations"
Okay, this worked:
Copy code
module "migrations" {
  source                   = "../../fleet/terraform/addons/migrations"
  ecs_cluster              = module.fleet.byo-vpc.byo-db.byo-ecs.service.cluster
  task_definition          = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.family
  task_definition_revision = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.revision
  subnets                  = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].subnets
  security_groups          = module.fleet.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].security_groups
  desired_count            = 0
  ecs_service              = "fleet"
  min_capacity             = 0
}
with this modification in `migrate.sh`:
Copy code
MAX_CAPACITY=1
	aws application-autoscaling register-scalable-target --region "${REGION:?}" --service-namespace ecs --resource-id "service/${ECS_CLUSTER:?}/${ECS_SERVICE:?}" --scalable-dimension "ecs:service:DesiredCount" --min-capacity "${CAPACITY:?}" --max-capacity "${MAX_CAPACITY:?}"
Okay that worked. Thanks for your help @Benjamin Edwards!