Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV-2459: Review and Improvements #643

Merged
merged 30 commits into from
Aug 9, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7239efd
review and improvements to learning journey
milldr Aug 7, 2024
797dcc1
review and improvements to learning journey
milldr Aug 7, 2024
13265f1
Merge branch 'staging' into DEV-2459/total-review
milldr Aug 7, 2024
e817669
review and improvements to learning journey
milldr Aug 7, 2024
f6332f5
review and improvements to learning journey
milldr Aug 8, 2024
558853c
Apply suggestions from code review
milldr Aug 8, 2024
343f6af
review and improvements to learning journey
milldr Aug 8, 2024
4179a14
Merge branch 'DEV-2459/total-review' of github.com:cloudposse/docs in…
milldr Aug 8, 2024
ef4c2f7
review and improvements to learning journey
milldr Aug 8, 2024
aa1f101
review and improvements to learning journey
milldr Aug 8, 2024
7bccacc
review and improvements to learning journey
milldr Aug 8, 2024
9bdebfd
review and improvements to learning journey
milldr Aug 8, 2024
0d6723f
review and improvements to learning journey
milldr Aug 8, 2024
adfd582
review and improvements to learning journey
milldr Aug 8, 2024
806f3e4
review and improvements to learning journey
milldr Aug 8, 2024
0b0e41a
Add ECS FAQ documentation
milldr Aug 8, 2024
f979342
Update eks GitHub Actions controller deployment steps
milldr Aug 9, 2024
56ec84e
Merge branch 'staging' into DEV-2459/total-review
osterman Aug 9, 2024
5bbc2ec
Update deployment steps for the Actions Runner Controller
milldr Aug 9, 2024
66bce59
Merge branch 'DEV-2459/total-review' of github.com:cloudposse/docs in…
milldr Aug 9, 2024
0f78a1e
Add integration instructions for JumpCloud with AWS SSO
milldr Aug 9, 2024
98360fe
note case
milldr Aug 9, 2024
1d0f38e
Apply suggestions from code review
milldr Aug 9, 2024
d6e51d4
review and improvements to learning journey
milldr Aug 9, 2024
407ae86
review and improvements to learning journey
milldr Aug 9, 2024
c503459
review and improvements to learning journey
milldr Aug 9, 2024
d591be8
Update layers with additional database options
milldr Aug 9, 2024
3a35378
fixed steps
milldr Aug 9, 2024
cafa4f1
Update docs/layers/network/connect-network.mdx
milldr Aug 9, 2024
fa69a18
Merge branch 'staging' into DEV-2459/total-review
milldr Aug 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions docs/jumpstart/action-items.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,7 @@ Before we can get started, here's the minimum information we need from you.

Please also provision a single test user in your IdP for Cloud Posse to use for testing and add those user credentials to 1Password.

- [GSuite](https://aws.amazon.com/blogs/security/how-to-use-g-suite-as-external-identity-provider-aws-sso/): Follow the "Google Workspace SAML application setup" steps only. Cloud Posse will handle the rest.
- [Office 365](/layers/identity/aws-sso/office365/how-to-setup-office-365-aws-sso)
- [JumpCloud](https://jumpcloud.com/support/integrate-with-aws-iam-identity-center)
- [Other AWS supported IdPs: Azure AD, CyberArk, Okta, OneLogin, Ping Identity](https://docs.aws.amazon.com/singlesignon/latest/userguide/supported-idps.html)
- [AWS Identity Center (SSO) ClickOps](/layers/identity/aws-sso/)

<Admonition type="caution">
- GSuite does not automatically sync Users and Groups with AWS Identity Center without additional configuration! If using GSuite as an IdP, considering deploying the [ssosync tool](https://github.com/awslabs/ssosync).
Expand All @@ -80,7 +77,7 @@ Before we can get started, here's the minimum information we need from you.
If deploying AWS SAML as an alternative to AWS SSO, we will need a separate configuration and metadata file. Again, please refer to the relevant linked guide.

- [GSuite](https://aws.amazon.com/blogs/desktop-and-application-streaming/setting-up-g-suite-saml-2-0-federation-with-amazon-appstream-2-0/): Follow Steps 1 through 7. This document refers to Appstream, but the process will be the same for AWS.
- [Office 365](/layers/identity/aws-saml/how-to-setup-saml-login-to-aws-from-office-365)
- [Office 365](/layers/identity/tutorials/how-to-setup-saml-login-to-aws-from-office-365)
- [JumpCloud](https://support.jumpcloud.com/support/s/article/getting-started-applications-saml-sso2)
- [Okta](https://help.okta.com/en-us/Content/Topics/DeploymentGuides/AWS/aws-configure-identity-provider.htm)
</Step>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Deploy CloudTrail and ECR"
sidebar_label: "Deploy CloudTrail"
sidebar_label: "Deploy Account Baseline"
sidebar_position: 4
---
import Intro from '@site/src/components/Intro';
Expand Down
2 changes: 1 addition & 1 deletion docs/layers/accounts/accounts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Cloud Posse has developed a set of components to provision, configure, and manag
### Using an Organization

Leveraging multiple AWS accounts within an AWS Organization is the only way to satisfy these requirements. Guard rails
can be be in place to restrict what can happen in an account and by whom.
can be created to restrict what can happen in an account and by whom.

We then further organize the flat account structure into organizational units. Organizational units (OUs) can then
leverage things like Service Control Policies to restrict what can happen inside the accounts.
Expand Down
80 changes: 51 additions & 29 deletions docs/layers/accounts/deploy-accounts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,35 +50,31 @@ This step-by-step process outlines how to deploy AWS accounts using `atmos` work
<Step>
## <StepNumber/> Configure Root Account as Organization

Before performing the "Deploying Accounts" step, the root account needs to be configured as an AWS Organization.
Before performing the "Deploy Accounts" step, the root account needs to be configured as an AWS Organization.

To deploy all accounts, we need to request an increase of the Account Quota from AWS support, which requires an AWS Organization to be created first.

This process also enables [AWS RAM for Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_enable-ram.html) via a CLI command, which is required for connecting the Organization.
This process also enables [AWS RAM for Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_enable-ram.html) via a CLI command, which is required for connecting the Organization.
</Step>

<Step>
## <StepNumber/> Raise Account Limits

Complete the root account setup with the following ClickOps steps.

From the `root` account (not `SuperAdmin`):
To deploy all accounts, we need to request an increase of the Account Quota from AWS support, which requires an AWS Organization to be created first.

1. Increase the [account quota to 20+](https://us-east-1.console.aws.amazon.com/servicequotas/home/services/organizations/quotas) for the Cloud Posse reference architecture, or more depending on your business use-case
From the `root` account (not `SuperAdmin`), increase the [account quota to 20+](https://us-east-1.console.aws.amazon.com/servicequotas/home/services/organizations/quotas) for the Cloud Posse reference architecture, or more depending on your business use-case

</Step>

<Step>
## <StepNumber/> Deploying Accounts
## <StepNumber/> Deploy Accounts

<Note title="IMPORTANT">
<Note title="Important">
With the addition of support for dynamic Terraform roles, our `baseline` cold start refarch layer now depends
on/requires that we have `aws-teams` and `aws-team-roles` stacks configured. This is because `account-map` uses those
stacks to determine which IAM role to assume when performing Terraform in the account, and almost every other component
uses `account-map` (indirectly) to chose the role to assume.
uses `account-map` (indirectly) to chose the role to assume. However, these components do _not_ need to be deployed yet.
</Note>

Again verify the "account" configuration in `stacks/catalog/account.yaml`.
Again verify the "account" configuration in `stacks/catalog/account.yaml`. In the next step, we will create and configure all accounts in the AWS Organization using the configuration in that stack file.

Once confident, begin the accounts deployment:

Expand All @@ -88,8 +84,11 @@ This step-by-step process outlines how to deploy AWS accounts using `atmos` work

These deployments will create all AWS member accounts and store relevant account metadata as "mappings" in the Terraform
outputs of the `account-map` component. Rather than querying this `account` component each time we need an Account ID or
role, we provision a static component `account-map`. **IMPORTANT:** Always run
`atmos terraform apply account-map -s core-gbl-root` after provisioning accounts.
role, we provision a static component `account-map`.

<Note title="Important">
Always run `atmos terraform apply account-map -s core-gbl-root` after provisioning accounts.
</Note>

Once you've created the accounts, you'll need to provision the baseline configuration within the accounts themselves.
This is accomplished by running `atmos workflow deploy/account-settings -f accounts`.
Expand All @@ -101,21 +100,44 @@ This step-by-step process outlines how to deploy AWS accounts using `atmos` work
<Step>
### <StepNumber/> ClickOps to Complete Account Setup

For each new account:

1. Perform a password reset by attempting to log in to the AWS console as a "root user", using that account's email
address, and then clicking the "Forgot password?" link. You will receive a password reset link via email, which
should be forwarded to the shared Slack channel for automated messages. Click the link and enter a new password. (Use
1Password or Random.org to create a password 26-38 characters long, including at least 3 of each class of character:
lower case, uppercase, digit, and symbol. You may need to manually combine or add to the generated password to ensure
3 symbols and digits are present.) Save the email address and generated password as web login credentials in
1Password. While you are at it, save the account number in a separate field.
1. Log in using the new password, choose "My Security Credentials" from the account dropdown menu and set up
Multi-Factor Authentication (MFA) to use a Virtual MFA device. Save the MFA TOTP key in 1Password by using
1Password's TOTP field and built-in screen scanner. Also, save the Virtual MFA ARN (sometimes shown as "serial
number").
1. While logged in, enable optional regions as described in the next step, if needed.
1. (Optional, but highly recommended): Unsubscribe the account's email address from all marketing emails.
For each new account, you will need to perform some manual configurations via ClickOps. These configurations include setting up the root account credentials, enabling MFA, and unsubscribing the account's email address from all marketing emails.

<Steps>
1. Reset the root user password:
<Steps>
1. Perform a password reset by attempting to log in to the AWS console as a "root user" using that account's email address
1. Click the "Forgot password?" link
1. You will receive a password reset link via email, which should be forwarded to the shared Slack channel for automated messages. Click the link
1. Enter a new password

<Note title="Tip">
Use 1Password to create a password 26-38 characters long, including at least 3 of each class of character: lower case, uppercase, digit, and symbol
</Note>

1. Save the email address and generated password as web login credentials in 1Password
1. Finally, record the account number in a separate field of the 1Password item, and save it. This is optional but recommended.
</Steps>

1. Configure MFA:
<Steps>
1. Log in to the AWS console using the new password
1. Choose "My Security Credentials" from the account dropdown menu
1. Set up Multi-Factor Authentication (MFA) to use a Virtual MFA device
1. Save the MFA TOTP key in 1Password with 1Password's "One-Time Password" field
1. Enter the generated MFA codes in AWS to verify the MFA device
1. Save the Virtual MFA ARN in the same 1Password entry. We will need this to set up the MFA device for `SuperAdmin` in Leapp
</Steps>

1. Enable any necessary optional regions

1. Optional, but highly recommended - unsubscribe the account's email address from all marketing emails
<Steps>
1. Go to [AWS Marketing Preferences](https://pages.awscloud.com/communication-preferences.html)
1. Click "Unsubscribe from Email"
1. Enter the account's email address
1. Check "Unsubscribe from all AWS marketing emails"
</Steps>
</Steps>

For more details, review
[the detailed "AWS Cold Start" documentation](/layers/accounts/tutorials/manual-configuration/#configure-root-account-credentials-for-each-account).
Expand Down
15 changes: 8 additions & 7 deletions docs/layers/accounts/prepare-aws-organization.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,7 @@ import Intro from '@site/src/components/Intro';
import KeyPoints from '@site/src/components/KeyPoints';
import Steps from '@site/src/components/Steps';

:::tip Cold Start

The set up process for the "baseline" or "account" tier is commonly referred to as the Cold Start.

:::

The Cold Start is the most manually involved tier. Read through the following steps and see [the detailed documentation](/layers/accounts/tutorials/manual-configuration/) for edge cases.
The Cold Start involves more manual steps than layers. Read through the following steps and see [the detailed documentation](/layers/accounts/tutorials/manual-configuration/) for edge cases.

In short, the steps are...

Expand All @@ -24,6 +18,13 @@ In short, the steps are...
| Vendor components | `atmos workflow vendor -f baseline` |
| Configure root SuperAdmin | Click Ops |

:::tip Cold Start

The set up process for the "baseline" or "account" tier is commonly referred to as the Cold Start.

:::


## Prerequisites

Follow the [prerequisites steps in the How-to Get Started guide](/layers/project/#0-prerequisites).
Expand Down
61 changes: 59 additions & 2 deletions docs/layers/data/data.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,64 @@
---
title: Provision Databases
sidebar_class_name: hidden
---

# Data
import Intro from '@site/src/components/Intro';
import PrimaryCTA from '@site/src/components/PrimaryCTA';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

TODO
<Intro>
There are many options for provisioning databases in AWS. The reference architecture provides support for a variety of databases, including Aurora PostgreSQL, Aurora MySQL, DynamoDB, RDS, ElastiCache, DocumentDB, and more.

Each database has its own unique features and benefits, so you can choose the one that best fits your needs or deploy multiple databases to support different use cases.
</Intro>

### SQL Database Options

The reference architecture supports several SQL database options, including Aurora PostgreSQL, Aurora MySQL, RDS, and more.

<Tabs queryString="sql">
<TabItem value="aurora-postgres" label="Aurora PostgreSQL">
Aurora PostgreSQL is a fully managed, PostgreSQL-compatible relational database service that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. Aurora PostgreSQL is a good choice for applications that require the features and capabilities of PostgreSQL with the scalability, performance, and reliability of a managed service.

<PrimaryCTA to="/components/library/aws/aurora-postgres/">Get Started</PrimaryCTA>
</TabItem>

<TabItem value="aurora-mysql" label="Aurora MySQL">
Aurora MySQL is a fully managed, MySQL-compatible relational database service that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. Aurora MySQL is a good choice for applications that require the features and capabilities of MySQL with the scalability, performance, and reliability of a managed service.

<PrimaryCTA to="/components/library/aws/aurora-mysql">Get Started</PrimaryCTA>
</TabItem>

<TabItem value="rds" label="RDS">
Amazon RDS is a managed relational database service that supports multiple database engines, including MySQL, PostgreSQL, MariaDB, Oracle, and SQL Server. RDS is a good choice for applications that require the features and capabilities of a relational database with the scalability, performance, and reliability of a managed service.

<PrimaryCTA to="/components/library/aws/rds">Get Started</PrimaryCTA>
</TabItem>

<TabItem value="redshift" label="RedShift">
Amazon Redshift is a fully managed, petabyte-scale data warehouse service that provides fast query performance using SQL and business intelligence tools. Redshift is a good choice for applications that require high-performance analytics and data warehousing with built-in security, backup, and restore capabilities.

<PrimaryCTA to="/components/library/aws/redshift">Get Started</PrimaryCTA>
</TabItem>
</Tabs>

### NoSQL Database Options

The reference architecture also supports several NoSQL database options, including DynamoDB, DocumentDB, and more. SQL and/or NoSQL databases can always be deployed side by side to support different use cases.

<Tabs queryString="nosql">
<TabItem value="dynamodb" label="DynamoDB">
DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. DynamoDB is a good choice for applications that require low-latency, high-throughput access to data with built-in security, backup, and restore capabilities.

<PrimaryCTA to="/components/library/aws/dynamodb">Get Started</PrimaryCTA>
</TabItem>

<TabItem value="documentdb" label="DocumentDB">
Amazon DocumentDB is a fully managed, MongoDB-compatible document database service that provides fast and scalable performance with built-in security, backup, and restore capabilities. DocumentDB is a good choice for applications that require the features and capabilities of MongoDB with the scalability, performance, and reliability of a managed service.

<PrimaryCTA to="/components/library/aws/documentdb">Get Started</PrimaryCTA>
</TabItem>

</Tabs>
19 changes: 6 additions & 13 deletions docs/layers/data/setup.mdx
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
---
title: Provision Databases
sidebar_label: Provision Databases
title: Setup Databases
sidebar_label: Setup
sidebar_position: 2
description: Provision databases in AWS using Atmos
---
import Intro from '@site/src/components/Intro';
import KeyPoints from '@site/src/components/KeyPoints';
import Steps from '@site/src/components/Steps'
import Note from '@site/src/components/Note'

## Quick Start

Expand All @@ -33,26 +34,18 @@ All deployment steps below assume that the environment has been successfully set
At the moment we have support for:

- [Aurora PostgreSQL](/components/library/aws/aurora-postgres/)

- [Aurora PostgreSQL Resources](/components/library/aws/aurora-postgres-resources/)

- [Aurora MySQL](/components/library/aws/aurora-mysql/)

- [Aurora MySQL Resources](/components/library/aws/aurora-mysql-resources/)

- [AWS Backup](/components/library/aws/aws-backup/)

- [DocumentDB](/components/library/aws/documentdb/)

- [DynamoDB](/components/library/aws/dynamodb/)

- [Elasticsearch Cluster](/components/library/aws/elasticsearch/)

- [RDS](/components/library/aws/rds/)

- [RedShift](/components/library/aws/redshift/)
- [ElastiCache Redis](/components/library/aws/elasticache-redis/)

- [ElastiCache Redis](/components/library/aws/elasticache-redis/) ### Vendor
### Vendor

Vendor all data components with the following workflow:

Expand All @@ -63,7 +56,7 @@ atmos workflow vendor -f data
These run several vendor commands for each included component. You can always run these commands individually to update
any single component. For example:

> Note: We're using `aurora-postgres` for this example. Your database selection may differ.
<Note title="Note">We're using `aurora-postgres` for this example. Your database selection may differ.</Note>

```bash
atmos vendor pull -c aurora-postgres
Expand Down
7 changes: 3 additions & 4 deletions docs/layers/data/tutorials/how-to-migrate-rds-snapshots.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,13 @@ sidebar_position: 100
import Intro from '@site/src/components/Intro';
import KeyPoints from '@site/src/components/KeyPoints';
import Steps from '@site/src/components/Steps'
import Note from '@site/src/components/Note'

## Context

:::info Namespace Abbreviations

<Note title="Note">
In this document we will refer to the Legacy Organization as `legacy` and refer to the new Organization as `acme`.

:::
</Note>

At this point we have a populated database in the `legacy-prod` account that we want to migrate to the new organization.
This database is encrypted with the Amazon Managed KMS key, `aws/rds`. We have already deployed an empty database to the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Should logs be sent to datadog or log only to cloudwatch logs?

## Considered Options

<Tabs>
<Tabs queryString="option">

<TabItem label="Option 1" value="option-1">
### Option 1. Fluent Bit sidecar is required for ECS Fargate
Expand Down
30 changes: 0 additions & 30 deletions docs/layers/ecs/ecs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,33 +67,3 @@ By default we also support datadog logging and metrics. This can be disabled by
```yaml
datadog_agent_sidecar_enabled: false
```

## FAQ

### How do I add a new service to an existing ECS cluster?

Add a new instance of the `ecs-service` component to your stack configuration. The component will automatically detect the ECS cluster and add the service to it.

### How can I add AWS Policies to my ECS Tasks?

Use the `task_policy_arns` to attach policies to individual tasks, this allows those tasks to access AWS resources.

```yaml
task_policy_arns:
- arn:aws:iam::aws:policy/AmazonS3FullAccess
```

### How can I inject secrets into my ECS Service?

Use the `map_secrets` variable which maps a environment variable key to an SSM param store key. This will inject the value of the SSM param store key into the environment variable.

```yaml
map_secrets:
SECRET_KEY: /my/secret/key
```

### How can we create Self Hosted Runners for GitHub with ECS?

If we are not deploying EKS for our platform, it doesn't make much sense to configure EKS solely for self-hosted runners. Instead, we deploy the [Philips Labs Action Runners](/layers/github-actions/philips-labs-github-runners) and connect those instances to GitHub.

For more on self-hosted GitHub Runners, see [`github-runners`](/layers/github-actions).
Loading