Osquery uses basic SQL commands to leverage a relational data-model to describe a device.

osquery

Hey gang, trying to update from `4.64.1` to `4.64.2` and I seem to have stuck my instance. Keep getting `Waiter ServicesStable failed: Max attempts exceeded` when trying to deploy to AWS and my whole instance is down. Any chance someone could help?

<@U06M4STFXJ6> looking for a resource - stand by

Are you using our terraform? AS far as I know the process should be just updating the version string &amp; `terraform apply`

hello! yes I am using the example code from here: <https://github.com/fleetdm/fleet-terraform/tree/main/example>

I guess I don't know if the timeout you got is permanent... Does it let you try again after a cooldown period?

I can try again as often as I can but I always get the max retries exceeded message

also I can provide the full error output if you would like to take a look

error.txt

Nope! We've had this instance running for about 5-6 months now with plenty of updates in that time

Is ECS just not having enough time to reach a stable state?

`wait_for_steady_state_timeout = 600` increase the timeout?

I am thinking we should try to like restart something other than just Fleet services if possible.

Although 10m seems like a long time out...

Is there a restart for the EC2 instance itself, not just Fleet or whatever is running on it?

Do you have access the aws management console? Are there any other ECS service events in the console?

also how do I set that `wait_for_steady_state_timeout` variable?

image.png

Should just be able to drop that into the `resource "aws_ecs_service"` block. However, I think that might be a last resort as <@U03LYLUQ5DF> is right.. that's a pretty long time to wait for the service to reach a stable state

so maybe the task is failing for some reason? Can you look at the task logs?

```Stopped reason
CannotPullContainerError: pull image manifest has been retried 5 time(s): httpReadSeeker: failed open: unexpected status code <https://registry-1.docker.io/v2/fleetdm/fleet/manifests/sha256:9c02cc1121e6b3dbfc24382517c6cea08da7c66248386c35841e2fbd59c998f9>: 429 Too Many Requests - Server message: toomanyrequests: You have reached your unauthenticated pull rate limit. <https://www.docker.com/increase-rate-limit>```

```# To avoid pull-rate limiting from dockerhub, consider using our <http://quay.io|quay.io> mirror
# for the Fleet image. e.g. "<http://quay.io/fleetdm/fleet:v4.47.1|quay.io/fleetdm/fleet:v4.47.1>"```
Don't tell me it was right in front of me this whole time! :sob:

Just use the right version. :slightly_smiling_face:

Sorry about that gang, got too lost in the sauce. Let me see if this fixes it

Not a problem! Let us know if you still need help. :slightly_smiling_face: