Small Focused Unit/Integration Tests

Don’t try and test everything in a single test. Each test should have a single reason to fail.
If you are using infrastructure in your integration tests it is worth having some simple tests that check that it is present.
These are simple to write and quick to run and will make the rest of the test suit far more stable.

Assertions about infrastructure is about is that S3 bucket present. Have I created that database. Is the database at least this version. These are assertions that would be far too unstable to have in a distributed manner. In fact these should explain how to setup or configure the resource when they fail. Having a test framework with multiline strings can give you a way to tell future people (including you) how to fix something.