lobo/aws-sdk-for-zig
1
0
Fork 0
Code Issues Pull requests Projects 1 Releases Packages 1 Wiki Activity Actions

Merge branch 'master' into zig-develop

Browse source
This commit is contained in:
Emil Lerch 2025-03-23 16:24:49 -07:00
commit aac7b03c2e
Signed by: lobo
GPG key ID: A7B62D657EF764F8
5 changed files with 115 additions and 85 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitea/workflows/zig-previous.yaml
View file

@ -1,5 +1,6 @@
name: AWS-Zig Build
on:
workflow_dispatch:
push:
branches:
- 'zig-0.13'

61
README.md
View file

@ -1,11 +1,11 @@
AWS SDK for Zig
===============
[Zig 0.13](https://ziglang.org/download/#release-0.13.0):
[Zig 0.14](https://ziglang.org/download/#release-0.14.0):
[![Build Status: Zig 0.13.0](https://git.lerch.org/lobo/aws-sdk-for-zig/actions/workflows/build.yaml/badge.svg)](https://git.lerch.org/lobo/aws-sdk-for-zig/actions?workflow=build.yaml&state=closed)
[![Build Status: Zig 0.14.0](https://git.lerch.org/lobo/aws-sdk-for-zig/actions/workflows/build.yaml/badge.svg)](https://git.lerch.org/lobo/aws-sdk-for-zig/actions?workflow=build.yaml&state=closed)
[Last Mach Nominated Zig Version](https://machengine.org/about/nominated-zig/):
[Last Mach Nominated Zig Version](https://machengine.org/docs/nominated-zig/):
[![Build Status: Mach nominated](https://git.lerch.org/lobo/aws-sdk-for-zig/actions/workflows/zig-mach.yaml/badge.svg)](https://git.lerch.org/lobo/aws-sdk-for-zig/actions?workflow=zig-mach.yaml&state=closed)
@ -13,12 +13,13 @@ AWS SDK for Zig
[![Build Status: Zig Nightly](https://git.lerch.org/lobo/aws-sdk-for-zig/actions/workflows/zig-nightly.yaml/badge.svg)](https://git.lerch.org/lobo/aws-sdk-for-zig/actions?workflow=zig-nightly.yaml&state=closed)
**NOTE ON BUILD STATUS**: The nightly/mach nominated version of this currently
panics under CI, but I have not yet reproduced this panic. Running manually on
multiple machines appears to be working properly
[Zig 0.13](https://ziglang.org/download/#release-0.13.0):
[![Build Status: Zig 0.13.0](https://git.lerch.org/lobo/aws-sdk-for-zig/actions/workflows/zig-previous.yaml/badge.svg)](https://git.lerch.org/lobo/aws-sdk-for-zig/actions?workflow=zig-previous.yaml&state=closed)
Current executable size for the demo is 980k after compiling with -Doptimize=ReleaseSmall
in x86_linux, and will vary based on services used. Tested targets:
in x86_64-linux, and will vary based on services used. Tested targets:
* x86_64-linux
* riscv64-linux
@ -30,22 +31,38 @@ in x86_linux, and will vary based on services used. Tested targets:
Tested targets are built, but not continuously tested, by CI.
Zig-Develop Branch
------------------
Branches
--------
This branch is intended for use with the in-development version of Zig. This
starts with 0.12.0-dev.3180+83e578a18. This is aligned with [Mach Engine's Nominated
Zig Versions](https://machengine.org/about/nominated-zig/). Nightly zig versions
are difficult to keep up with and there is no special effort made there, build
status is FYI (and used as a canary for nominated zig versions).
* **master**: This branch tracks the latest released zig version
* **zig-0.13**: This branch tracks the previous released zig version (0.13 currently).
Support for the previous version is best effort, generally
degrading over time. Fixes will generally appear in master, then
backported into the previous version.
* **zig-mach**: This branch tracks the latest mach nominated version. A separate
branch is necessary as mach nominated is usually, but not always,
more recent than the latest production zig. Support for the mach
version is best effort.
* **zig-develop**: This branch tracks zig nightly, and is used mainly as a canary
for breaking changes that will need to be dealt with when
a new mach nominated version or new zig release appear.
Expect significant delays in any build failures.
Other branches/tags exist but are unsupported
Building
--------
`zig build` should work. It will build the code generation project, fetch model
files from upstream AWS Go SDK v2, run the code generation, then build the main
project with the generated code. Testing can be done with `zig test`.
project with the generated code. Testing can be done with `zig build test`. Note that
this command tests on all supported architectures, so for a faster testing
process, use `zig build smoke-test` instead.
To make development even faster, a build option is provided to avoid the use of
LLVM. To use this, use the command `zig build -Dno-llvm smoke-test`. This
can reduce build/test time 300%. Note, however, native code generation in zig
is not yet complete, so you may see errors.
Using
-----
@ -53,7 +70,8 @@ Using
This is designed for use with the Zig package manager, and exposes a module
called "aws". Set up `build.zig.zon` and add the dependency/module to your project
as normal and the package manager should do its thing. A full example can be found
in [/example](example/README.md).
in [/example](example/build.zig.zon). This can also be used at build time in
a downstream project's `build.zig`.
Configuring the module and/or Running the demo
----------------------------------------------
@ -61,8 +79,8 @@ Configuring the module and/or Running the demo
This library mimics the aws c libraries for it's work, so it operates like most
other 'AWS things'. [/src/main.zig](src/main.zig) gives you a handful of examples
for working with services. For local testing or alternative endpoints, there's
no real standard, so there is code to look for `AWS_ENDPOINT_URL` environment
variable that will supersede all other configuration.
no real standard, so there is code to look for an environment variable
`AWS_ENDPOINT_URL` variable that will supersede all other configuration.
Limitations
-----------
@ -83,13 +101,6 @@ TODO List:
* Implement timeouts and other TODO's in the code
* Add option to cache signature keys
Services without TLS 1.3 support
--------------------------------
All AWS services should support TLS 1.3 at this point, but there are many regions
and several partitions, and not all of them have been tested, so your mileage
may vary. If something doesn't work, please submit an issue to let others know.
Dependency tree
---------------

135
build.zig
View file

@ -34,12 +34,12 @@ pub fn build(b: *Builder) !void {
"no-llvm",
"Disable LLVM",
) orelse false;
const broken_windows = b.option(
bool,
"broken-windows",
"Windows is broken in this environment (do not run Windows tests)",
) orelse false;
const no_bin = b.option(bool, "no-bin", "skip emitting binary") orelse false;
// TODO: Embed the current git version in the code. We can do this
// by looking for .git/HEAD (if it exists, follow the ref to /ref/heads/whatevs,
// grab that commit, and use b.addOptions/exe.addOptions to generate the
@ -67,17 +67,6 @@ pub fn build(b: *Builder) !void {
const smithy_module = smithy_dep.module("smithy");
exe.root_module.addImport("smithy", smithy_module); // not sure this should be here...
// Expose module to others
_ = b.addModule("aws", .{
.root_source_file = b.path("src/aws.zig"),
.imports = &.{.{ .name = "smithy", .module = smithy_module }},
});
// Expose module to others
_ = b.addModule("aws-signing", .{
.root_source_file = b.path("src/aws_signing.zig"),
.imports = &.{.{ .name = "smithy", .module = smithy_module }},
});
// TODO: This does not work correctly due to https://github.com/ziglang/zig/issues/16354
//
// We are working here with kind of a weird dependency though. So we can do this
@ -100,51 +89,73 @@ pub fn build(b: *Builder) !void {
const run_step = b.step("run", "Run the app");
run_step.dependOn(&run_cmd.step);
const gen_step = blk: {
const cg = b.step("gen", "Generate zig service code from smithy models");
const cg = b.step("gen", "Generate zig service code from smithy models");
const cg_exe = b.addExecutable(.{
.name = "codegen",
.root_source_file = b.path("codegen/src/main.zig"),
// We need this generated for the host, not the real target
.target = b.graph.host,
.optimize = if (b.verbose) .Debug else .ReleaseSafe,
});
cg_exe.use_llvm = !no_llvm;
cg_exe.root_module.addImport("smithy", smithy_dep.module("smithy"));
var cg_cmd = b.addRunArtifact(cg_exe);
cg_cmd.addArg("--models");
cg_cmd.addArg(try std.fs.path.join(
b.allocator,
&[_][]const u8{
try b.dependency("models", .{}).path("").getPath3(b, null).toString(b.allocator),
models_subdir,
},
));
cg_cmd.addArg("--output");
cg_cmd.addDirectoryArg(b.path("src/models"));
if (b.verbose)
cg_cmd.addArg("--verbose");
// cg_cmd.step.dependOn(&fetch_step.step);
// TODO: this should use zig_exe from std.Build
// codegen should store a hash in a comment
// this would be hash of the exe that created the file
// concatenated with hash of input json. this would
// allow skipping generated files. May not include hash
// of contents of output file as maybe we want to tweak
// manually??
//
// All the hashes can be in service_manifest.zig, which
// could be fun to just parse and go nuts. Top of
// file, generator exe hash. Each import has comment
// with both input and output hash and we can decide
// later about warning on manual changes...
const cg_exe = b.addExecutable(.{
.name = "codegen",
.root_source_file = b.path("codegen/src/main.zig"),
// We need this generated for the host, not the real target
.target = b.graph.host,
.optimize = if (b.verbose) .Debug else .ReleaseSafe,
});
cg_exe.root_module.addImport("smithy", smithy_module);
var cg_cmd = b.addRunArtifact(cg_exe);
cg_cmd.addArg("--models");
cg_cmd.addArg(try std.fs.path.join(
b.allocator,
&[_][]const u8{
try b.dependency("models", .{}).path("").getPath3(b, null).toString(b.allocator),
models_subdir,
},
));
cg_cmd.addArg("--output");
const cg_output_dir = cg_cmd.addOutputDirectoryArg("src/models");
if (b.verbose)
cg_cmd.addArg("--verbose");
// cg_cmd.step.dependOn(&fetch_step.step);
// TODO: this should use zig_exe from std.Build
// codegen should store a hash in a comment
// this would be hash of the exe that created the file
// concatenated with hash of input json. this would
// allow skipping generated files. May not include hash
// of contents of output file as maybe we want to tweak
// manually??
//
// All the hashes can be in service_manifest.zig, which
// could be fun to just parse and go nuts. Top of
// file, generator exe hash. Each import has comment
// with both input and output hash and we can decide
// later about warning on manual changes...
cg.dependOn(&cg_cmd.step);
break :blk cg;
};
cg.dependOn(&cg_cmd.step);
exe.step.dependOn(gen_step);
exe.step.dependOn(cg);
// This allows us to have each module depend on the
// generated service manifest.
const service_manifest_module = b.createModule(.{
.root_source_file = cg_output_dir.path(b, "service_manifest.zig"),
.target = target,
.optimize = optimize,
});
service_manifest_module.addImport("smithy", smithy_module);
exe.root_module.addImport("service_manifest", service_manifest_module);
// Expose module to others
_ = b.addModule("aws", .{
.root_source_file = b.path("src/aws.zig"),
.imports = &.{
.{ .name = "smithy", .module = smithy_module },
.{ .name = "service_manifest", .module = service_manifest_module },
},
});
// Expose module to others
_ = b.addModule("aws-signing", .{
.root_source_file = b.path("src/aws_signing.zig"),
.imports = &.{.{ .name = "smithy", .module = smithy_module }},
});
// Similar to creating the run step earlier, this exposes a `test` step to
// the `zig build --help` menu, providing a way for the user to request
@ -174,8 +185,9 @@ pub fn build(b: *Builder) !void {
.target = b.resolveTargetQuery(t),
.optimize = optimize,
});
unit_tests.root_module.addImport("smithy", smithy_dep.module("smithy"));
unit_tests.step.dependOn(gen_step);
unit_tests.root_module.addImport("smithy", smithy_module);
unit_tests.root_module.addImport("service_manifest", service_manifest_module);
unit_tests.step.dependOn(cg);
unit_tests.use_llvm = !no_llvm;
const run_unit_tests = b.addRunArtifact(unit_tests);
@ -199,11 +211,16 @@ pub fn build(b: *Builder) !void {
.optimize = optimize,
});
smoke_test.use_llvm = !no_llvm;
smoke_test.root_module.addImport("smithy", smithy_dep.module("smithy"));
smoke_test.step.dependOn(gen_step);
smoke_test.root_module.addImport("smithy", smithy_module);
smoke_test.root_module.addImport("service_manifest", service_manifest_module);
smoke_test.step.dependOn(cg);
const run_smoke_test = b.addRunArtifact(smoke_test);
smoke_test_step.dependOn(&run_smoke_test.step);
b.installArtifact(exe);
if (no_bin) {
b.getInstallStep().dependOn(&exe.step);
} else {
b.installArtifact(exe);
}
}

1
build.zig.zon
View file

@ -6,6 +6,7 @@
"build.zig",
"build.zig.zon",
"src",
"codegen",
"README.md",
"LICENSE",
},

2
src/servicemodel.zig
View file

@ -1,5 +1,5 @@
const std = @import("std");
const service_list = @import("models/service_manifest.zig");
const service_list = @import("service_manifest");
const expectEqualStrings = std.testing.expectEqualStrings;
pub fn Services(comptime service_imports: anytype) type {