131 lines
5.4 KiB
Markdown
131 lines
5.4 KiB
Markdown
# AWS SDK for Zig
|
|
|
|
[](https://drone.lerch.org/lobo/aws-sdk-for-zig)
|
|
|
|
This SDK currently supports all AWS services except EC2 and S3. These two
|
|
services only support XML, and zig 0.8.0 and master both trigger compile
|
|
errors while incorporating the XML parser. S3 also requires some plumbing
|
|
tweaks in the signature calculation, which is planned for a zig version
|
|
(probably self-hosted 0.9.0) that no longer has an error triggered. Examples
|
|
of usage are in src/main.zig.
|
|
|
|
This is my first serious zig effort, so please issue a PR if the code isn't
|
|
"ziggy" or if there's a better way.
|
|
|
|
This is designed to be built statically using the `aws_c_*` libraries, so
|
|
we inherit a lot of the goodness of the work going on there. Current
|
|
executable size is 9.7M, about half of which is due to the SSL library.
|
|
Running strip on the executable after compilation (it seems zig strip
|
|
only goes so far), reduces this to 4.3M. This is for x86_linux,
|
|
(which is all that's tested at the moment).
|
|
|
|
# 2021-10-28 SDK Update
|
|
|
|
To get smaller executable size and better portability with faster compilation,
|
|
my intent is to rework the http communications using
|
|
[requestz](https://github.com/ducdetronquito/requestz). This relies on a
|
|
couple other projects, and will require the creation of a zig implementation
|
|
for [Signature Version 4](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html),
|
|
along with support for some S3 quirks, etc. It will also reduce compatibility
|
|
with some edge cases. Long term I think this is a better approach, however and
|
|
will remove (or replace) a ton of the C dependencies as well as avoid a bunch
|
|
of complexity such as the event loop C implementation found in the AWS
|
|
libraries, which eventually could be replaced by zig async. I will create a new branch
|
|
for this work as master is currently working fairly well. My hope is to
|
|
converge on Zig 0.9.0 so this SDK doesn't have to keep up with changes going on
|
|
in zig master. I'd also love to have an official package manager in zig, but I
|
|
know this is high on the priority list for the foundation. My goal is to **not** require
|
|
[gyro](https://github.com/mattnite/gyro) as requestz does, but we'll see how
|
|
that develops.
|
|
|
|
## Building
|
|
|
|
I am assuming here that if you're playing with zig, you pretty much know
|
|
what you're doing, so I will stay brief.
|
|
|
|
First, the dependencies are required. Use the Dockerfile to build these.
|
|
a `docker build` will do, but be prepared for it to run a while. The
|
|
Dockerfile has been tested on x86_64 linux, but I do hope to get arm64
|
|
supported as well.
|
|
|
|
Once that's done, you'll have an alpine image with all dependencies ready
|
|
to go and zig master installed. There are some build-related things still
|
|
broken in 0.8.0 and hopefully 0.8.1 will address those and we can be on
|
|
a standard release.
|
|
|
|
* `zig build` should work. It will build the code generation project, run
|
|
the code generation, then build the main project with the generated code.
|
|
There is also a Makefile included, but this hasn't been used in a while
|
|
and I'm not sure that works at the moment.
|
|
|
|
## Running
|
|
|
|
This library uses the aws c libraries for it's work, so it operates like most
|
|
other 'AWS things'. Note that I tested by setting the appropriate environment
|
|
variables, so config files haven't gotten a run through.
|
|
main.zig gives you a handful of examples for working with services.
|
|
For local testing or alternative endpoints, there's no real standard, so
|
|
there is code to look for `AWS_ENDPOINT_URL` environment variable that will
|
|
supercede all other configuration.
|
|
|
|
## Dependencies
|
|
|
|
|
|
Full dependency tree:
|
|
aws-c-auth
|
|
* s2n
|
|
* aws-lc
|
|
* aws-c-common
|
|
* aws-c-compression
|
|
* aws-c-common
|
|
* aws-c-http
|
|
* s2n
|
|
* aws-c-common
|
|
* aws-c-io
|
|
* aws-c-common
|
|
* s2n
|
|
* aws-lc
|
|
* aws-c-cal
|
|
* aws-c-common
|
|
* aws-lc
|
|
* aws-c-compression
|
|
* aws-c-common
|
|
* aws-c-cal
|
|
* aws-c-common
|
|
* aws-lc
|
|
|
|
Build order based on above:
|
|
|
|
1. aws-c-common
|
|
1. aws-lc
|
|
2. s2n
|
|
2. aws-c-cal
|
|
2. aws-c-compression
|
|
3. aws-c-io
|
|
4. aws-c-http
|
|
5. aws-c-auth
|
|
|
|
Dockerfile in this repo will manage this
|
|
|
|
TODO List:
|
|
|
|
* Implement jitter/exponential backoff. This appears to be configuration of
|
|
`aws_c_io` and should therefore be trivial
|
|
* Implement timeouts and other TODO's in the code
|
|
* Switch to aws-c-cal upstream once [PR for full static musl build support is merged](https://github.com/awslabs/aws-c-cal/pull/89)
|
|
(see Dockerfile)
|
|
* Move to compiler on tagged release (hopefully 0.8.1)
|
|
(new 2021-05-29. I will proceed in this order unless I get other requests)
|
|
* Implement [AWS restXml protocol](https://awslabs.github.io/smithy/1.0/spec/aws/aws-restxml-protocol.html).
|
|
Includes S3. Total service count 4. This may be blocked due to the same issue as EC2.
|
|
* Implement [AWS EC2 query protocol](https://awslabs.github.io/smithy/1.0/spec/aws/aws-ec2-query-protocol.html).
|
|
Includes EC2. Total service count 1. This is currently blocked, probably on
|
|
self-hosted compiler coming in zig 0.9.0 (January 2022) due to compiler bug
|
|
discovered. More details and llvm ir log can be found in the
|
|
[XML branch](https://git.lerch.org/lobo/aws-sdk-for-zig/src/branch/xml).
|
|
|
|
Compiler wishlist/watchlist:
|
|
|
|
* [Merge PR to allow stripping -static](https://github.com/ziglang/zig/pull/8248)
|
|
* [comptime allocations](https://github.com/ziglang/zig/issues/1291) so we can read files, etc (or is there another way)
|