Peace - Zero Stress Automation - 🪶 Blog

Checkpoint

2025-01-26T00:00:00+00:00

Does this project have to be an endless single player game?

TL;DR</a></h2>
I'm hoping for any combination of the following:

People who want to see the Peace framework developed to viable-existence, to fund my existence</a>.</li>
Someone to take over and continue development of the Peace project.</li>
Someone who finds the ✒️ `dot_ix</code></a> side-side project fun or fulfilling, to take over it / develop` layout-rs</code></a> as a replacement for GraphViz dot, and switch to encre-css</code></a> for styling.</li> </ol> 1. Motivation</a></h2> Each time I provide or use a tool that manages software infrastructure, there is always something lacking. Pragmatically, an organisation whose business is to ship a product, cannot invest disproportional resource on improving tools beyond "it's efficient enough". However, one of my desires is to have ridiculously good tools^{1, because they can:} switch dangerous buttons for safe ones</li> simplify overwhelming information into someting understandable</li> turn fear of mistakes into fun of learning</li> </ul> ^{1 ridiculous enough for a dreary-eyed 🦹 engineer to get excited}2. Demo</a></h2> envman</code> is a tool built using the Peace framework that: Downloads a file from Github.</li> Creates some AWS resources.</li> Uploads the file to AWS.</li> </ol> The videos below show the beginnings of "ridiculously good". 2.1. Like Git, But With Infrastructure</a></h3> </object> </video> Regardless of the process, a really nice tool would be able to do the following: Tell me what the current state of a system is.</li> Tell me what will happen, if I were to run the automation.</li> Don't create twice the resources when I run it twice.</li> Don't clutter my screen with irrelevant detail.</li> Do tell me the detail, when I care^{2.</li>}Make sure everything is cleaned up.</li> </ol> These capabilities (among others) allow me to do what I want without needing to worry about making mistakes, or forgetting to clean up resources. ^{2 Not shown, but detailed state is stored}2.2. Interruptible</a></h3> </object> </video> If I change my mind, let me stop the process (safely!), change a parameter, and resume the process. And don't throw away all the work that was done before -- determine what's there, and only change the parts that need changing. 2.3. Profiles (like Branches)</a></h3> </object> </video> Sometimes I work with multiple environments, so let me switch between them. And let me compare the state of two environments -- perhaps one is a replica of the other, so show me what's changed. 2.4. Communicate Clearly, Recover Cleanly</a></h3> </object> </video> When something goes wrong, clearly showing which step went wrong (the red one!) and how to recover saves both time and effort in continuing in the flow. Bonus points for recovering by simply pressing the "go" button again. 2.5. Visual</a></h3> 🚧 work-in-progress feature </blockquote> </object> </video> Not everyone works on the command line, and even for the software-inclined, being able to see what each step of the process does, without running the automation, helps build understanding faster than simply seeing a list of steps. Having real-time visibility of automation execution makes it clear which resources are interacting. </object> </video> If there is an error, appropriate styling immediately draws attention to the resources are involved in the error. Skip ahead to 5. Imagination</a> if you don't care how it's built. 3. Distributing Effort</a></h2> To build tools of this calibre everywhere, practically one would have to separate: Logic common to items managed by automation (e.g. save / load / display information).</li> Logic to manage a particular item (e.g. download a file).</li> Parameters / values used in the automation (e.g. use this particular URL).</li> The combining of these parts (organisation specific process / intellectual property).</li> </ol> That way the overall goodness can be provided by the framework, automation logic is extendable and shareable, and everyone gets to keep their secrets. </object> source</a> Using a general purpose programming language to define these parts means: Tooling from that language is available to adhere to best practices.</li> The standard packaging system handles publishing and distribution.</li> </ul> 3.1 Incentive</a></h3> Peace is intended to be low-friction to uptake^{3 -- developers who use Peace should not need to write much additional code than what they would have if they were writing basic automation logic.} It should be a matter of: Creating types for the parameters and state of the item managed by automation.</li> Fitting the automation logic into the Item</code></a> trait.</li> Instead of calling each automation step's function directly, putting that into a graph structure called a Flow</code>.</li> Calling the appropriate Peace Cmd</code> with the defined Flow</code>.</li> </ol> which should be similar effort to writing a tool without the Peace framework. For the web diagram, "item interactions"</a> need to be defined, which Peace will combine with other Item</code>s to create a diagram that contains all items. In practice, when switching from "just code a quick tool" to "write it using the Peace framework", anecdotally it's switching from 1 unit of effort producing 1 unit of value, to 1.5 units of effort producing 4 units of value. ^{3 this statement may not be well-fitting since it's written in Rust, but if you were to have written a small automation tool in Rust, it should be nearly identical effort, for significantly more value delivered} 4. Side-Side Projects</a></h2> </object> source</a> In building this project, besides restarting</a> twice</a>, a number of side, side projects were created. The ones that are likely useful on their own are: ✒️ dot_ix</code></a>: Interactive dot graphs (diagrams).</li> 🛑 interruptible</code></a>: Stops a future producer or stream from producing values when interrupted.</li> 🧬 fn_graph</code></a>: Runs interdependent logic concurrently, starting each function when predecessors have completed.</li> 🗂️ type_reg</code></a>: Serializable map of any type.</li> 🗂️ resman</code></a>: Runtime managed resource borrowing.</li> </ul> So in order for Peace to fulfill the "ridiculously good" notion, many side quests just had to be done. 4.1 Dot IX</a></h3> dot_ix</code> in particular has its own client side application</a> to create SVG diagrams from structured input. I frequently use this to create various technical diagrams. My wishlist for dot_ix</code></a>: For diagram generation, use layout-rs</code></a> to generate the SVG in-process instead of delegating to a WASM-compiled GraphViz dot</code>. Besides performance gains and portability, it may be possible to rasterize the SVG and render a diagram in the terminal. However, layout-rs</code></a> only supports basic text labels, so a lot more development effort needs to be poured into it to unlock this possibility. </li> For styling, instead of using tailwindcss</code></a>'s non-production CDN JS and hacky CSS extraction</a>, switch to encre-css</code></a>. Perhaps update encre-css</code> to synchronize with Tailwind CSS 4.0 as well. </li> </ol> It's too far down the priority list for myself to get to in the foreseeable future, but if someone were to do them I'll be grateful. 5. Imagination</a></h2> There are nice things I imagined being able to do: Visually select the items in the process that I want to run, then press go -- subset execution, with an interactive UI over it.</li> Selecting a node on the web diagram opens a panel with detailed information about that item.</li> Undo the last execution / make the environment as it was at this particular point.</li> </ol> Reaching that level of simplicity may take a few thousand additional hours of effort. 6. Summary</a></h2> Chipping away at this for 5 years, it certainly feels too much for one person. I had a plan to get this production ready</a> and use it at work, but being on the screen for both work and home activities doesn't sit well. So either this becomes what I work on full time (through sponsorship</a>), or someone else who has the energy could take it and build upon what's there (and hopefully we share the same level of code-pedantry 🕵️). Let me know (discord</a> / email: mail@azriel.im</code>).

Had I Had Time 2023-06-29T00:00:00+00:00 Had I had time, what would perfect automation look like? </blockquote> So Far</a></h1> For Users</a></h2> From the book of "don't overwhelm me while I'm learning" (or, ever), the user experience has been refined to be understandable and safe. Solid Foundations</a></h3> Automation software built using the Peace framework allows users to: See what would happen, before making it happen.</li> Press "go" multiple times, without adverse effects.</li> </ul> </video> Changes Of Mind</a></h3> There is some resilience to changes of mind and interruptions, and you are not punished for changing your mind: </video> _{ℹ️ Also, that state discovery step after interruption won't be necessary when}peace#141</a> is implemented Profile Support</a></h3> Peace is designed to support for working with multiple environments; though this part is still early days: </video> For Developers</a></h2> Write Code, Not Strings</a></h3> Type safety, compile time support, and code assist enables: Easy discovery of what parameters are available – no more searching and pasting large chunks of YAML / JSON.</li> Errors at compile time – no longer waste time deploying to discover minor issues.</li> Writing code to pass values between items, not string templates.</li> </ul> Real code from env_deploy_flow.rs@0.0.11</code></a>: let iam_role_params_spec = IamRoleParams::<WebApp>::field_wise_spec() .with_name(iam_role_name) .with_path(path.clone()) .with_managed_policy_arn_from_map(|iam_policy_state: &IamPolicyState| { if let IamPolicyState::Some { policy_id_arn_version: Generated::Value(policy_id_arn_version), .. } = iam_policy_state { Some(policy_id_arn_version.arn().to_string()) } else { None } }) .build(); </code></pre> No Minor Annoyances</a></h3> The little things are taken care of: Sorted values for clear and understandable diffs.</li> Sensible directory structure for separate and common values within a namespace.</li> </ul> ❯ diff -u .peace/envman/demo_1/env_deploy/{states_goal.yaml,states_current.yaml} iam_role: !Some name: demo_1 path: / - role_id_and_arn: !Tbd null + role_id_and_arn: !Value + id: AROASBLNZJUXOTGGPP7PX + arn: arn:aws:iam::140353555758:role/demo_1 .. </code></pre> repo/.peace/envman/demo_1/env_deploy ^ ^ ^ ^ | | | '--------- flow: cleanly separate information for different | | | workflows, e.g. "deploy" vs "load data" | | | | | '----- profile: logically separates environments | | | '----- tool name: allows multiple tools to be run in the same repo | '----- `.peace` data directory, like `.git` </code></pre> Take care of the little things, that make the little difference, that make the big difference. Up Ahead</a></h1> There are three dimensions that I'd like to develop Peace further: Production Readiness</a></h2> </div> The lifetime of an environment should be able to exist before, and live beyond, a version of the automation software that manages it. For the Peace framework to be a sensible choice to build a production tool, the framework and items must be able to handle evolving data and logic. Namely, handling cases like: Terminate these servers, recreate the VPC with resized CIDR blocks, then launch new servers in the new VPC.</li> This infrastructure was created manually, now it should be automatically managed, and migrated to use new naming conventions.</li> </ul> The Peace: Production Readiness</a> GitHub project captures this work. See Flow Versioning</a> for my technical thinking out loud notes. User Interface</a></h2> </div> Two common methods to access automation are the command line, and the web browser. Peace should provide support for: Invoking commands to interact with the environment.</li> Presenting common data – usually States</code>, but also ParamsSpecs</code>, Params</code>, errors, and progress.</li> </ul> Some of the CLI presentation logic currently lives in the envman</code> example, and should be moved into peace</code>, and the web presentation logic is currently "bundle leptos</code></a> with the command line app and render the flow graph". The Peace: UI</a> GitHub project captures the start of this work. For the web UI part, see DEVELOPMENT.md#web-development</code></a>. API Refinement</a></h2> </div> There is some API that doesn't prevent developers from making mistakes or having mild annoyances through natural usage of the API. The API should be refined to avoid setting up such situations, and implement additional functionality that makes it easier to work with information across profiles or across flows. The Peace: API Refinement and Features</a> GitHub project captures this work. Had I had time, developing these would be excellent. But I don't – I'm off to work! 🏃🏽‍♂️💨

Peace - Zero Stress Automation - 🪶 Blog

Checkpoint

2. Demo</a></h2> envman</code> is a tool built using the Peace framework that:</p> Downloads a file from Github.</li> Creates some AWS resources.</li> Uploads the file to AWS.</li> </ol> The videos below show the beginnings of "ridiculously good".</p>

Had I Had Time