Questions about AWS Transfer

There are a few things to consider, including:

• How to design and configure your S3 bucket(s).
• Choosing the protocol(s) to use - FTP, SFTP or FTPS, and authentication methods (eg. whether to hook into external services like Active Directory).
• Network design - whether this is an internet-facing or internal service, whether you need a fixed IP, DNS configuration, any ingress network security rules, or running on a non-standard port (eg. SFTP listening on something other than port 22).
• Designing the IAM role setup, and the mapping of FTP users to home directories, to control the S3 paths that users have access to.

I'm primarily interested in configuring a public-facing SFTP service, so will focus on that.

This depends on the type of "endpoint" chosen for your transfer service. There are two endpoint types: public and VPC. Public endpoints are only accessible over the internet. They have fewer network configuration choices to think about, but come with limitations. One of these limitations is that IPs are assigned by AWS and subject to change. This makes public endpoints unsuitable for cases where the FTP client is on a network with IP-based firewall rules.

If you want to assign a static IP to the FTP server, you must choose the VPC endpoint type, which means access to the server will be controlled by VPC security groups.

AWS publishes a list of IP ranges used by its services and regions. It's not immediately clear to me which "service" the Transfer product corresponds to - I guess S3. These ranges can change, so if you want to make use of this information you have to be able to handle updates. AWS publishes updates as an SNS topic, which enables people to do this programmatically. I'm not sure how often these updates are published. If your only use-case is setting up an FTP service, it will be significantly simpler to use a static IP for the service than to worry about IP ranges.

There are a few differences. For example, public endpoints must use the SFTP protocol, and have no way to configure an allow-list of client IPs, or to change the port used for the FTP service.

AWS recommends using the VPC endpoint type as it provides more security features. I haven't seen any features that you get with the public endpoint type that can't be replicated with the VPC endpoint type. The AWS docs include a comparison of the different endpoint types.

Virtual Private Cloud is Amazon's "virtual network" abstraction - it's what provides your networking configuration. It's a logically-isolated network that you define with traditional network features configuration features like subnets, routing tables and internet gateways.

VPC isn't a feature that you can opt out of - certain AWS resources (eg. EC2 instances) must be launched into this network. If you don't specifically configure a VPC, AWS will initialise a "default" VPC when you first provision EC2 resources.

Security groups (which control the traffic that is allowed in/out of an EC2 instance) and network ACLs (which control traffic entering and exiting a subnet) are both features of the VPC.

VPC-hosted transfer servers can either be internal to the VPC, or internet-facing. To be internet-facing, they must be allocated an Elastic IP.

Elastic IPs are public IPv4 addresses that you can associate with resources in your AWS account. You have the option of importing your own address range, or receiving an address from Amazon's IPv4 pool. There are limits on the number of IPs you can request from Amazon (the default limit is 5).

If you're using the AWS console to create your transfer server, the endpoint details are pretty self-explanatory - there are dropdown fields to select a VPC, subnet and Elastic IP.

If you're using Terraform's aws_transfer_server to create the transfer server, you have to provide the VPC details as endpoint_details. This gives you the option of specifying the VPC ID, subnet, and a list of address_allocation_ids, which point to Elastic IPs. Allocations IDs are essentially just IDs used to identify an Elastic IP - you can see them clearly in the Elastic IP section of the console.

If you're using Route53 to manage DNS for your domain, you can configure an "alias" record, which is a Route53-specific feature to automatically route traffic to an AWS resource.

In the AWS console, if you select to use "other DNS" instead of Route53, the transfer server form still contains a field for filling in the hostname - eg. I can enter ftp.mattduck.com. This confused me initially as I wasn't sure the purpose of entering a hostname if your DNS is configured externally. It turned out that this will create a new public zone in Route53 for the domain that you enter (mattduck.com), with a CNAME record (ftp) pointing to the auto-generated AWS hostname of the server.

Although this is a public zone, Route53 isn't the authoritative nameserver for mattduck.com, so it has no affect if I try to access ftp.mattduck.com on my laptop. I believe EC2 images are configured by default to use AWS nameservers, and that the CNAME entry would be resolved on an EC2 box within the VPC (subject to some VPC settings like enableDnsSupport).

You just create a CNAME record where the value is the public hostname of the transfer server. I'm not aware of any approaches other than this.

This is very surface-level, but there are three kinds of server-side encryption in S3:

• SSE-S3: this is where AWS encrypts each object with a unique key, which it stores to automatically decrypt the object on read. Each key is itself encrypted with a master key, for which AWS manages rotation. If your bucket/objects use SSE-S3 encryption, then the only consideration is making sure the policy associated with the FTP user has the appropriate S3 read/write permissions - there's nothing extra to worry about.
• SSE-KMS: KMS stands for Key Management Service. Instead of Amazon's master keys, encryption uses Customer Master Keys, which are managed in your account, and which have some additional features like being limited via policies and having audit trails for access. For buckets/objects that are encrypted with KMS, the policy associated with the FTP user must have some additional permissions for the relevant key - kms:Decrypt, kms:ReEncrypt, kms:GenerateDataKey and kms:DescribeDataKey.
• SSE-C: in this option, the user must provide an encryption key when uploading an object to S3, and then that same key must be provided in subsequent requests to retrieve the object. This is not supported in AWS Transfer - which makes sense because FTP doesn't provide a way to specify this key.

This is pretty simple, particularly using Terraform - you create an aws_transfer_user, assign them a role with the appropriate policy, and then create an aws_transfer_ssh_key. The user can then authenticate with the SFTP server and perform the actions granted by the policy.

This is probably the most involved / fiddly part to get right (at least if you want to support multiple isolated users with their own directories). There are a two types of home directory - PATH and LOGICAL. With the path-type, you're dependent on IAM policies to control bucket access. With the logical-type, you can create a map of the user's FTP directories to particular S3 targets.

The IAM policies support some user-relative variables like ${Transfer:UserName} and ${Transfer:HomeDirectory}, to make it easier to reuse policies. I haven't dived into this but I think there are some limitations on where they can be used - eg. only on roles but not users (or vice-versa).

Yes. I was curious about this. The docs state that the public key has a maximum length of 2048. I wondered if this meant the key size when using ssh-keygen would be limited to 2048. In fact, the concept of ssh key sizes (when specifying the number of bytes via eg. ssh-keygen -b 3072) refers to the length of the modulus value used to compute the RSA key pair - it's not directly related to the size of the files. When decoded, the files themselves contain the modulus, respective public/private values and some other data.

I think the AWS public key length just refers to the number of characters in the public key. When I use default ssh-keygen options with -b 4096 this comes out at 735.