Findy Agency Blog

• The Swiss Army Knife for the Agency Project, Part 3: Other GitHub Tools
• The Swiss Army Knife for the Agency Project, Part 2: Release Management with GitHub
• The Swiss Army Knife for the Agency Project, Part 1: GitHub Actions
• Issuing Chatbot
• I want mDL!
• Managing GitHub Branch Protections
• Path to Passwordless
• Test Coverage Beyond Unit Testing
• How To Write Modifiable & Readable Go Code
• How To Write Performant Go Code
• GopherCon UK 2023
• Beautiful State-Machines - FSM Part II
• The Agency Workshop
• The Time to Build SSI Capabilities Is Now
• Digital Identity Hack Part 3 – Introduction to SSI Presentation
• Digital Identity Hack Part 2 – Panel Discussion on Blockchain Projects
• Digital Identity Hack – Unlocking the Potential of SSI
• Deploying with CDK Pipeline
• Agency's IaC Journey
• No-Code SSI Chatbots - FSM Part I
• How to Equip Your App with VC Superpowers
• Getting Started with SSI Service Agent Development
• Ledger Multiplexer
• The Hyperledger Global Forum Experience
• The Findy Agency API
• Trust in Your Wallet
• SSI-Empowered Identity Provider
• Replacing Indy SDK
• The Missing Network Layer Model
• Fostering Interoperability
• Anchoring Chains of Trust
• The Arm Adventure on Docker
• Travelogue
• Announcing Findy Agency

The Swiss Army Knife for the Agency Project, Part 3: Other GitHub Tools

This post concludes the article series that overviews how our open-source project utilizes the GitHub platform for software development. In this last part, we will examine documentation, communication tools, and some utilities that help the development process.

GitHub Pages

Already in the early days, when we started to experiment with SSI technology, there arose a need to publish different experiment outcomes and findings to the rest of the community. However, we were missing a platform to collect these various reports and articles.

Finally, when we decided to open-source our agency code, we also established a documentation site and blog. This site has been valuable to us since then. It has worked as a low-threshold publication platform, where we have collected our various opinions, experimentation findings, and even technical guides. It has been a way of distributing information and an excellent source for discussion. It is easy to refer to articles that even date back some years when all of the posts are collected on the same site.

We use the Hugo framework for building the site, specifically the Docsy theme, which is especially suitable for software project documentation. The Hugo framework uses a low-code approach, where most of the content can be edited as Markdown files. However, when needed, modifications can also be made to the HTML pages quite easily.

We publish the site via GitHub Pages, a static website hosting solution that GitHub offers for each repository. When the Pages feature is enabled, GitHub serves the content of the pages from a dedicated branch. A GitHub action handles the deployment, i.e., builds the website and pushes the site files to this dedicated branch. The site URL combines organization and repository names. An exception is the <organisation-name> .github.io-named repositories that one can find in the similarly formatted URL, e.g., our findy-network/findy-network.github.io repository is served in URL https://findy-network.github.io.

Dependabot

Dependabot is a GitHub-offered service that helps developers with the security and version updates of repository dependencies. When Dependabot finds a vulnerable dependency that can be replaced with a more secure version, it creates a pull request that implements this change. A similar approach exists with dependency version updates: When a new version is released, Dependabot creates a PR with the dependency update to the new version.

The pull requests that the Dependabot creates are like any other PR. The CI runs tests on Dependabot PRs similar to those, as with the changes developers have made. One can review and merge the PR similarly.

Using Dependabot requires a comprehensive enough automated test set. The one who merges the PR needs to confirm that the change will not cause regression. Manual testing is required if the CI is missing the automated tests for the changes. If one needs manual testing, the benefit of using the bot for automated PR creation is lost, and it might even be more laborious to update the dependencies one by one.

In our project, we used the principle of testing all the crucial functionality with each change automatically. Therefore, it was safe for us to merge the Dependabot PRs without manual intervention. We even took one step further. We enabled automatic merging for those Dependabot PRs that passed the CI. In this approach, a theoretical risk exists that malicious code is injected from the supply chain through the version updates without us noticing it. However, we were willing to take the risk as we estimated that sparing a small team’s resources would be more important than shielding the experimental code from theoretical attacks.

You should estimate the risk/effort ratio when automating crucial tasks.

CI enables the automatic merging of Dependabot PRs on pull request creation:

name: "pr-target"
on:
  pull_request_target:

  # automerge successful dependabot PRs
  dependabot:
    # action permissions
    permissions:
      pull-requests: write
      contents: write
    # runner machine
    runs-on: ubuntu-latest
    # check that the PR is from dependabot
    if: ${{ github.event.pull_request.user.login == 'dependabot[bot]' }}
    steps:
      - name: Dependabot metadata
        id: dependabot-metadata
        uses: dependabot/fetch-metadata@v2
      - name: Enable auto-merge for Dependabot PRs
        # check that target branch is dev
        if: ${{steps.dependabot-metadata.outputs.target-branch == 'dev'}}
        # enable PR auto-merge with GitHub CLI
        # PR will be automatically merged if all checks pass
        run: gh pr merge --auto --merge "$PR_URL"
        env:
          PR_URL: ${{github.event.pull_request.html_url}}
          GH_TOKEN: ${{secrets.GITHUB_TOKEN}}

Notifications And Labeling

There are chat application integrations (for example Teams, Slack) one can use to send automatic notifications from GitHub. We have used this feature to get notified when PRs are created and merged to keep the whole team up-to-date on what changes are introduced. In addition, we always get a notification when someone raises an issue in our repositories. It has allowed us to react quickly to community questions and needs.

There have also been some problems regarding the notifications. The Dependabot integration creates a lot of pull requests for several repositories, and sometimes, the information flood is too much. Therefore, we have custom labeling functionality that helps us automatically label PRs based on the files the changes affect. The automatic labeling is implemented with GitHub Action, which is triggered on PR creation. We can filter only the PR notifications needed for our chat integration using these labels.

Automatic labeling can be tremendously helpful, especially with large repositories with many subprojects. Labels can also be used as a filter basis for GitHub Actions. For instance, one can configure a specific test for changes that affect only a particular folder. This approach can speed up the CI runtimes and save resources.

Codespaces And CLI Tool

GitHub Codespaces was a less-used feature for our project. Still, we found it helpful, especially when we organized workshops to help developers get started with agency application development.

Codespaces offers a cloud-based development environment that works as VS Code dev containers. A repository can have a configuration for the environment, and the codespace is created and launched based on that configuration. Once the codespace is up and running, the developers have a ready development environment with all the needed bells and whistles. It removes a lot of the setup hassle we usually encounter when using different operating systems and tools for development.

Example of a codespace configuration for our workshop participants. All the needed tools are preinstalled to the codespace that one may need when doing the workshop exercises:

{
  "name": "Findy Agency workshop environment",
  "image": "ghcr.io/findy-network/agency-workshop-codespace:0.1.0",
  "workspaceFolder": "/home/vscode/workshop",
  "features": {
    "ghcr.io/devcontainers-contrib/features/direnv-asdf:2": {}
  },
  "customizations": {
    "vscode": {
      "extensions": [
        "golang.go",
        "mathiasfrohlich.Kotlin"
      ]
    }
  },
  "postCreateCommand": "asdf direnv setup --shell bash --version latest"
}

The GitHub CLI tool was another helpful tool we found during the agency project. It offers the GitHub web application functionality via the command line. The tool is especially useful for those who suffer from context switching from terminal to browser and vice versa or who prefer to work from a terminal altogether.

In addition, the CLI tool can automate certain routines, both in CI and local environments. We use the Makefile to record often-used commands to support developers’ memory. The CLI is also a great fit for playing out with the GitHub API.

Thanks for Reading

This article completes our three-part saga of using GitHub in our agency project. I hope you have learned something from what you have read so far. I am always happy to discuss our approach and hear if you have ideas and experiences on how your team handles things!

Laura

The Swiss Army Knife for the Agency Project, Part 2: Release Management with GitHub

This article will examine how we handle releasing and artifact delivery with GitHub tools in our open-source project. When designing our project’s release process, we have kept it lightweight but efficient enough. One important goal has been ensuring that developers get a stable version, i.e., a working version of the code, whenever they clone the repository. Furthermore, we want to offer a ready-built deliverable whenever possible so that trying out our software stack is easy.

Branching Model

In our development model, we have three different kinds of branches:

• Feature branches, a.k.a. pull request branch: each change is first pushed to a feature branch, tested, and possibly reviewed before merging it to the dev branch.
• dev: the baseline for all changes. One merges pull requests first with this branch. If tests for dev succeed, one can merge the changes into the master branch. The dev branch test set can be more extensive than the one run for the pull request (feature) branch.
• master: the project’s main branch that contains the latest working version of the software. Changes to this branch trigger automatic deployments to our cloud environment.

The primary reason for using this branching style is to enable us to run extensive automated testing for changes introduced by one or multiple pull requests before merging them with the master branch. Therefore, this routine ensures that the master branch always has a working version. This popular branching model has other perks as well. You can read more about them here, for example.

Time for a Release

Our release bot is a custom GitHub action. Each repository that follows the branching model described above uses the bot for tagging the release. The bot’s primary duty is to check if the dev branch has code changes missing from the master branch. If the bot finds changes, it will create a version tag for the new release and update the working version of the dev branch.

The bot works night shifts and utilizes a GitHub scheduled event as an alarm clock. For each repository it is configured for, it runs the same steps:

1. Check if there are changes between the dev and master branches
2. Check if the required workflows (tests) have succeeded for the dev branch
3. Parse the current working version from the VERSION file
4. Check out the dev branch and tag the dev branch HEAD with the current version
5. Push tag to remote
6. Increase the working version and commit it to the dev-branch.

When step 5 is ready, i.e., the bot has created a new tag, another workflow will start. This workflow will handle building the project deliverables for the tagged release. After a successful release routine, the CI merges the tag to master.

Package Distribution

Each time a release is tagged, the CI builds the release deliverables for distribution. As our stack contains various-style projects built in various languages, the release artifacts depend on the project type and programming language used.

The CI stores all of the artifacts in GitHub in one way or another. Docker images and library binaries utilize different features of GitHub Packages, a GitHub offering for delivering packages to the open-source community. The Packages UI is integrated directly with the repository UI, so finding and using the packages is intuitive for the user.

Docker Containers for Backend Services

Our agency software has a microservice architecture, meaning multiple servers handle the backend functionality. To simplify cloud deployment and service orchestration in a local development environment, we build Docker images for each service release and store them in the GitHub container registry.

The GitHub Actions workflow handles the image building. We build two variants of the images. In addition to amd64, we make an arm64 version to support mainly environments running on Apple-silicon-based Macs. You can read more about utilizing GitHub Actions to create images for multiple platforms here.

The community can access the publicly stored images without authentication. The package namespace is https://ghcr.io, meaning one can refer to an image with the path ghcr.io/NAMESPACE/IMAGE_NAME:TAG, e.g., ghcr.io/findy-network/findy-agent:latest. Publishing and using images from the Docker registry is straightforward.

Libraries

We also have utility libraries that provide the common functionalities needed to build clients for our backend services. These include helpers for Go, Node.js, and Kotlin.

In the Go ecosystem, sharing modules is easy. The Go build system must find the dependency code in a publicly accessible Git repository. Thus, the build compiles the code on the fly; one does not need to distribute or download binaries. Module versions are resolved directly from the git tags found in the repository.

The story is different for Node.js and Kotlin/Java libraries. GitHub offers registries for npm, Maven, and Gradle, and one can easily integrate the library’s publishing to these registries into the project release process. However, accessing the libraries is more complicated. Even if the package is public, the user must authenticate to GitHub to download the package. This requirement adds more complexity for the library user than mainstream options, such as the default npm registry. I would avoid distributing public libraries via this feature.

Sample for publishing Node.js package to GitHub npm registry via GitHub action:

name: release
on:
  push:
    tags:
      - '*'
jobs:
  publish-github:
    # runner machine
    runs-on: ubuntu-latest
    # API permissions for the job
    permissions:
      contents: read
      packages: write
    steps:
      # checkout the repository code
      - uses: actions/checkout@v4
      # setup node environment
      - uses: actions/setup-node@v4
        with:
          node-version: '18.x'
          registry-url: 'https://npm.pkg.github.com'
          scope: 'findy-network'
      # install dependencies, build distributables and publish
      - run: npm ci
      - run: npm run build
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Executables And Other Artifacts

For other artifacts, we utilize the GitHub releases feature. For instance, we use the goreleaser helper tool for our CLI, which handles the cross-compilation of the executable for several platforms. It then attaches the binary files to the release, from where each user can download them. We even have an automatically generated installation script that downloads the correct version for the platform in question.

One can also attach other files to releases. We define our backend gRPC API with an IDL file. Whenever the API changes, we release a new version of the IDL using a GitHub release. We can then automate other logic (gRPC utility helpers) to download the IDL for a specific release and easily keep track of the changes.

Summary

This post summarized how our open-source project uses different GitHub features for versioning, release management, and artifact distribution. In our last article, we will review various other GitHub tools we have utilized.

The Swiss Army Knife for the Agency Project, Part 1: GitHub Actions

Our research project has been utilizing GitHub tools for several years. Most of the features are free for publicly hosted open-source projects. We have used these tools with an experimental spirit, gathering experience from the different features. Although our project does not aim directly at production, we take a production-like approach to many things we do.

In this article series, I summarize our team’s different experiments regarding GitHub features. As GitHub’s functionality is easily extensible through its API and abilities for building custom GitHub actions, I also explain how we have built custom solutions for some needs.

The series consists of three articles:

1. This first part overviews GitHub Actions as a continuous integration platform.
2. The second part concentrates on release management and software distribution.
3. Finally, in the last post, we introduce other useful integrations and features we used during our journey.

CI Vets the Pull Requests

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform for building automatic workflows. One defines the workflows as YAML configurations, which get stored along with repository code in a dedicated subfolder (.github/workflows). The GitHub-hosted platform automatically picks up and executes these workflows when defined trigger events happen.

Snippet below shows how the “test” job is configured:

name: test
on:
  # run whenever there is a push to any branch
  push:
jobs:
  test:
    # runner machine
    runs-on: ubuntu-latest
    steps:

      # check out the repository code
      - uses: actions/checkout@v4

      # setup node environment
      - uses: actions/setup-node@v4
        with:
          node-version: '18.x'

      # install dependencies, build, test
      - name: install deps
        run: npm ci
      - name: build
        run: npm run build
      - name: test
        run: npm test

      # upload test coverage to external service
      - name: upload coverage
        uses: codecov/codecov-action@v4
        with:
          files: ./coverage/coverage-final.json
          fail_ci_if_error: true
          token: ${{ secrets.CODECOV_TOKEN }}

In our project development model, developers always introduce changes through a pull request. CI runs a collection of required checks for the pull request branch, and only when the checks succeed can the changes be integrated into the main development branch.

Our code repositories typically have a dedicated workflow for running linting, unit testing, license scanning, and e2e tests. CI runs these workflow jobs whenever the developer pushes changes to the PR branch. The CI jobs are visible in the PR view, and merging is impossible if some required checks fail.

Actions Simplify the Workflows

The basic structure of a GitHub Actions job definition contains:

• Setting up the base environment for the workflow.
• Checking out the repository code.
• Execution steps that one can define as command-line scripts or actions.

Actions are reusable components written to simplify workflows. GitHub offers actions for the most commonly needed functionalities. You can also create custom actions specific to your needs or search for the required functionality in the actions marketplace, where people can share the actions they have made.

Example of using a custom action in a workflow:

name: test
on: push
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: setup go, lint and scan
        uses: findy-network/setup-go-action@master
        with:
          linter-config-path: .golangci.yml

For instance, we have written a custom action for our Go code repositories. These repositories need similar routines to set up the Go environment, lint the code, and check its dependencies for incompatible software licenses. We have combined this functionality into one custom action we use in each repository. This way, the workflow definition is kept more compact, and implementing changes to these routines is more straightforward as we need to edit only the shared action instead of all the workflows.

Custom action configuration example:

runs:
  using: "composite"
  steps:

    # setup go environment
    - uses: actions/setup-go@v5
      if: ${{ inputs.go-version == 'mod' }}
      with:
        go-version-file: './go.mod'
    - uses: actions/setup-go@v5
      if: ${{ inputs.go-version != 'mod' }}
      with:
        go-version: ${{ inputs.go-version }}

    # run linter
    - name: golangci-lint
      if: ${{ inputs.skip-lint == 'false' }}
      uses: golangci/golangci-lint-action@v4
      with:
        version: ${{ inputs.linter-version }}
        args: --config=${{ inputs.linter-config-path }} --timeout=5m

    # run license scan
    - name: scan licenses
      shell: bash
      if: ${{ inputs.skip-scan == 'false' }}
      run: ${{ github.action_path }}/scanner/scan.sh

In this shared action, setup-go-action, we can run our custom scripts or use existing actions. For example, we set up the Go environment with actions/setup-go, which is part of the GitHub offering. Then, we lint the code using the golangci/golangci-lint action, which is made available by the golangci-team. Lastly, the action utilizes our script to scan the licenses. We can efficiently combine and utilize functionality crafted by others and ourselves.

Runners Orchestrating E2E-tests

In addition to linting, scanning, and unit tests, we typically run at least one e2e-styled test in parallel. In e2e tests, we simulate the actual end-user environment where the software will be run and execute tests with bash scripts or test frameworks that allow us to run tests through the browser.

In these e2e tests, we typically set up the test environment using Docker containers, and the GitHub-hosted runners have worked well for orchestrating the containers. We can define needed container services in the workflow YAML configuration or use, e.g., Docker’s compose tool directly to launch the required containers.

Example of defining a needed database container in a GitHub Actions workflow:

    services:
      postgres:
        image: postgres:13.13-alpine
        ports:
          - 5433:5432
        env:
          POSTGRES_PASSWORD: password
          POSTGRES_DB: vault

External Integrations

Once the CI has run the tests, we use an external service to analyze the test coverage data. Codecov.io service stores the coverage data for the test runs. It can then use the stored baseline data to compare the test run results of the specific pull request. Once the comparison is ready, it gives feedback on the changes’ effect on the coverage. The report gets visualized in the pull request as a comment and line-by-line coverage in the affected code lines.

Codecov is an excellent example of integrating external services into GitHub Actions workflows. Each integration needs a GitHub application that gives the third-party tool access to your organization’s resources. The organization must review the required permissions and install the application before GitHub allows the integration to work. One can also build GitHub applications for custom purposes. Read more about how a GitHub application can be helpful when automating release versioning.

Next Up: Release Management

This article gave an overview of how Findy Agency uses GitHub Actions as a continuous integration platform. The next one will be about release management. Stay tuned.

Issuing Chatbot

This post is a development story (or hands-on workshop) of how we can use an FSM chatbot to implement SSI Services to allow any SSI/DID owner to be an issuer. Let’s start with the result and see what the state machine looks like. As you can see below, it’s simple, elegant, and, most importantly, easy to reason.

Issuing Service Chatbot

As you can see from the diagram there are two main paths. One for the issuer role and one for the rcvr role. But the most interesting thing is that how simple the machine is. As said, it’s very easy to reason and understand. And that’s the one of powers these machines. The second one comes from the computational model of FSMs in general, we could proof that they’re correct.

Note that since our original FSM engine release we have extended our model with transient state transitions or pseudostates, but our UML renderer doesn’t highlight them yet. We’ll publish an other blog post with new features of our FSM engine later.

What Problem Do We Solve?

The Hyperledger Indy-based SSI system is implemented with a CL signature scheme for ZKPs. That system needs the concept of Credential Definition stored in its creator’s wallet. The Credential Definition has an ID, which is quite similar to DID in the Indy-based AnonCreds system. The CredDefID is public. Everyone who knows it can request proof based on it or request to present a proof based on it.

But the CredDefID brings us some problems:

1. How do we find a correct CredDefID when it’s needed?
2. How about fully symmetric cases when everyone can simultaneously be an issuer, a holder, and a verifier? Everyone can issue credentials and receive proof of them in these use cases. For example, we have a use case where a seller (anybody in a marketplace) wants to issue a receipt for the transaction.

There are other problems, but the rest are based on the same core problem.

We’ll solve problem number 2 by using a notary-type service. We started with one service and implemented a reference chatbot to issue credentials on behalf of a logical issuer, aka seller. We also have implemented our version of a public DID. With these two, we have solved the problem quite elegantly.

Isn’t This Centralization?

In some ways, yes, but the result isn’t more centralized than the suggested trust registries for other or similar problems in the SSI field. In a certain way, this model adds self-sovereignty because now everyone can issue, and everyone can build these issuing services for their use cases.

More Reading

Before we continue, here’s a list of documents and places that are helpful when playing with these:

1. Getting Started With SSI Service Agent Development
2. Writing SSI Apps
3. FSM Documentation Part I
4. FSM Documentation Part II
5. The Workshop Material For Our CLI Tool

Note that the blog post is written from self-learning material point of view. Preferably, you should read and execute the commands it guides you to do simultaneously.

Prerequisites

Note, for now, we assume that the reader uses the local setup of the agency. It makes it easier to follow the core agent’s real-time logs.

Even though you have probably cloned the repo. But if not, please do it now:

git clone https://github.com/findy-network/findy-agent-cli

Follow the material in the repo’s readme files or the workshop material mentioned at the beginning to get your findy-agent-cli CLI tool working, i.e., communicating with your agency.

Helpers

1. Go to repo’s root:
2. Shorter name and autocompletion:

   alias cli=findy-agent-cli
   source ./scripts/sa-compl.sh
   

Document for now on assumes that CLI tool is named to cli.

Very Important

If you create new wallets directly with the CLI tool, make sure that auto-accept mode is ON for the agent. The check it after the cli agent ping works:

cli agent mode-cmd -r

Tip, if you have problems with CLI commands check your FCLI_ prefixed envs.

The result should be AUTO_ACCEPT. Note that allocation scripts do this automatically.

Chatbots work even when auto-accept isn’t ON. They can written to make decisions to decline or acknowledge presented proofs, for example.

Setup Agents And Wallets

It would be best to have at least three wallets and their agents up and running. Inside the findy-agent-cli repo, you have scrpits/fullstack directory. Let’s name it a tool root for the rest of the documentation. For example:

export MY_ROOT=`pwd`

Follow the workshop documentation on how to allocate new agents with their wallets.

Allocate the following agents (actual commands and the script calls follow):

1. issuing will be the issuing service
2. seller, will be a seller, aka logical issuer
3. buyer will be a buyer.
4. verifier will be a verifier for the VC. (Usage is out scoped from this document.)

Here’s an example of how you could

1. allocate the agents,
2. have access to FSM files by making links,
3. create schema and credential definition (done in issuing),
4. create DIDComm connections between issuing, seller, and buyer where the last is done in this phase just for testing the environment. During the service integration (e.g. marketplace app) to the app invitation or command to connect with is sent in its own step during the service use.

cd "$MY_ROOT"
make-play-agent.sh issuing seller buyer verifier
cd play/issuing
ln -s ../../fsm/issuing-service-f-fsm.yaml
ln -s ../../fsm/issuing-service-b-fsm.yaml
open `cli bot uml issuing-service-f-fsm.yaml` # UML rendering
source ./new-schema
source ./new-cred-def
./invitation | ../seller/connect
./invitation | ../buyer/connect

Optionally store a public DID of the Issuing Service Chatbot:

export PUB_DID=$(./pub-did print)

Note! Leave this terminal open and do not enter new commands to it yet.

Note! source ./new-cred-def initializes FCLI_CRED_DEF_ID environment variable. The issuing-service-f-fsm.yaml file references to this variable, i.e. it’s mandatory, or you could hard-code the credential definition value to your issuing-service-f-fsm.yaml.

Use The Issuing Service

1. open 2 separated terminals A and B (see the workshop material on how to init envs) to work as a seller, leave them to be.
2. open 2 separated terminals A and B (see the workshop material on how to init envs) to work as a buyer, leave them to be.
3. go back to the previous issuing terminal and start the chatbot:

   cli bot start --service-fsm issuing-service-b-fsm.yaml issuing-service-f-fsm.yaml -v=1
   

4. go back to the seller terminal A and enter a command: cli bot read. This is a read-only terminal window for the chatbot’s responses.
5. go back to the seller terminal B and enter cli bot chat. This is a write-only terminal window to send chat messages.
   1. (optional: enter ‘help’ to get used to what’s available)
   2. enter your session ID, select something easy like ‘SID_1’
   3. enter the text ‘issuer’ that’s our current role
   4. enter your attributes data value for credential, select something easy to remember during verfication
6. go back to the buyer terminal A and enter cli bot read. This is a read-only terminal window for the chatbot’s responses.
7. go back to the buyer terminal B and enter cli bot chat. This is a write-only terminal window to send chat messages.
   1. (optional: enter ‘help’ to get some reminders)
   2. enter your previous session ID, it was something easy like ‘SID_1’
   3. enter the text ‘rcvr’, it’s your role now
8. see the Buyer’s A terminal (cli bot read command running); the results should be that the credential is issued for the Buyer.
9. go to both B terminals and enter some text to move FSM instances to the start-again state.
10. it’s optional; you could rerun it with the same players.

Tip, when you started the Issuing Service Chatbot with -v=1 you could monitor it’s state transitions in real-time.

The Sequence Diagram

Notes about the current implementation:

• only one attribute value schema is implemented. Start with that and add cases where more attributes can be entered later. (Homework)
• every message sends a basic_message reply, which usually starts with ACK string. See the YAML file for more information. The reply messages aren’t drawn to the sequence diagram below to keep it as simple as possible.
• you can render state machines to UML:

  open `cli bot uml issuing-service-f-fsm.yaml` # give correct FSM file
  

  We have UML rendered state machine diagram in the beginning of this post.

sequenceDiagram
    autonumber

    participant Seller

    %% -- box won't work on hugo, or when this machine is running it --
    %% box Issuing Service
    participant IssuerFSM
    participant BackendFSM
    participant RcvrFSM
    %% end

    participant Buyer

    Seller -) IssuerFSM: 'session_id' (GUID)
    Seller -) IssuerFSM: issuer = role
    loop Schemas attributes
    Seller -) IssuerFSM: 'attribute_value'
    end

    alt Send thru existing connection 
    Seller -) Buyer: 'session_id' (same as above, design how app knows that this is a command)
    end

    Buyer -) RcvrFSM: 'session_id'
    Buyer -) RcvrFSM: rcvr = role

    RcvrFSM -) BackendFSM: receiver_arriwed
    BackendFSM -) IssuerFSM: rcvr_arriwed
    loop Schemas attributes
    IssuerFSM -) BackendFSM: 'attribute_value'
    BackendFSM -) RcvrFSM: 'attribute_value'
    end
    IssuerFSM -) BackendFSM: attributes done (not implemented, one attrib)
    BackendFSM -) RcvrFSM: attributes done (not implemented, one attrib)

    RcvrFSM -) Buyer: CREDENTIAL ISSUING PROTOCOL

Pre-steps (not in the diagram)

1. We can generate a public DID for the Issuing Service Chatbot.

   cd play/issuing # or where your bot is
   export PUB_DID=$(./pub-did print)
   

2. Save this PUB_DID to your app’s configuration. It’s where Issuing Service Chatbot can be found when needed. Note, PUB_DID is a URL which returns a new invitation on every load. You can treat is as a URL template:

   http://localhost:8080/dyn?did=8NCqkhnNjeTwMmK68DHDPx&label=<you_ad_number>
   

   You can enter your case specific data like ad number to the label arg.

Steps

1. Actual Seller role or app implementation for the role generates a sessionID (GUID) and sends it to Issuing Service Chatbot as a basic_message.
2. The Seller role is a logical issuer, so it sends the issuer string as a basic_message to the Issuer FSM instance.
3. The Seller role sends a <attr_val> (case specific in your schema) as a basic_message.
4. The Seller role sends the same sessionID directly to the buyer role. The communication channel can be their existing DIDComm connection or something else, but the buyer needs to know how to react to that line if it’s a basic_message.
5. The Buyer role or app implementation for the role sends the received sessionID to the chatbot, i.e., joins the same session.
6. The Buyer role sends the rcvr word to the chatbot to make explicit role selection. (We could leave this out in some version of FSM implementation and rely only on the order of the messages, but this allows us to understand better and keep things open for future extensions.)
7. The Rcvr FSM instance has now got the actual credential holder (Buyer/Receiver) and it sends a receiver_arriwed string to the Backend FSM.
8. The Backend FSM sends a rcvr_arriwed to the Issuer FSM as a basic_message.
9. Now the Issuer FSM loops thru all previously received (from Seller) attribute values and sends them to the Backend FSM.
10. The Backend FSM sends the attribute values to the Rcvr FSM as a basic_messages
11. Optional for the future: if there would be more attributes than one, this would be the place to send the information that all attributes are sent to the Backend FSM. Another way to implement these state machines would be to add information to both Issuing and Receiving FSMs how many attributes there are, and receiving states would be declared to rely on that knowledge.
12. Optional: see the previous step. The Backend FSM works as a forwarder for all of the cases where the issuing and the receiving FSM instances need to communicate with each other through the chatbot service.
13. Finally the RcvrFSM executes credential issuing protocol.

Conclusion

You have been blown away by how easy it is to implement these FSM-based chatbots, haven’t you? The Issuing Service is only one example of the potential of the FSM chatbots. We are excited to see what else you end up building. When doing so, please send your questions, comments, and feedback to our team. Let’s make this better—together.

Harri

I want mDL!

It’s funny that we don’t have a mobile driver’s license in Finland. In the Nordics, we are usually good with digital and mobile services. For example, we have had somewhat famous bank IDs from the early 90s.

For the record, Iceland has had a mobile driver’s license since 2020. Surprise, Finland was in the pole position in the summer of 2018. The government-funded mobile driver’s license app (beta) was used with 40.000 users. The project started in 2017 but was canceled in 2020 (link in Finnish).

How the heck have we ended up in this mess? We should be the number one in the world! Did we try to swallow too big a bite once when the country-level SSI studies started?

SSI Study

As you probably already know, our innovation team has studied SSI for a long time. We have started to understand different strategies you can follow to implement digital identity and services around it.

Trust-foundation

Christopher Allen, one of the influencers in the field of SSI, divided the Self-Sovereign identity into two primary tracks:

1. LESS (Legally-Enabled Self-Sovereign) Identity
2. Trustless Identity, or more precisely Trust Minimized Identity

These two aren’t mutually exclusive but give us a platform to state our goals. Which one do we prefer, a government or an individual?

The above table is from the Allen’s talk in YouTube.

I personally prefer Human Rights over Legally-Enabled.

However, from a researcher’s point of view, the LESS Identity track seems faster because it’s easier to find business cases. These business-driven use cases will pave the way to even more progress in censorship resistance, anonymity, etc. The mobile driver’s license is a perfect example of a LESS Identity practice. Let’s follow that for a moment, shall we?

Level Of Decentralization

Most internet protocols have started quite a high level of decentralization as their goal/vision through the history of computer science. There are many benefits to setting decentralization as a requirement: no single point of failure, easier to scale horizontally, etc.

Since blockchain, decentralization has become a hype word, and most of us need help understanding what it means to have fully decentralized systems. One easily forgotten is trust, and we will only achieve absolute decentralization once we have a proper model for Self-Certification.

Idealism vs Pragmatism

I see a paradox here. How about you? Why does anyone try to build maximally decentralized systems if their identities must be legally binding? Or why do we put a lot of effort into figuring out consensus protocols for systems that don’t need them?

Our legal system has solved all of these problems already. So, let’s stick on that be pragmatic only, shall we?

Pragmatism

My current conclusion is the old wisdom: don’t build a platform immediately, but solve a few use cases first and build the platform if needed.

Second might be don’t solve imaginary problems. Look for monetizable pain first and solve that with as small steps as possible.

Let’s see what that all means from SSI vs. mDL.

Example of Good UX

Apple Pay is the world’s largest mobile payment platform outside China. It’s been exciting to follow what happened in the Nordics, which already had several mobile payment platforms and the world’s most digitalized private banking systems when Apple Pay arrived.

Why has Apple Pay been so successful? Like many other features in Apple’s ecosystem, they took the necessary final technical steps to remove all the friction from setting up the payment trail. Significantly, the seller doesn’t need additional steps or agreements to support Apple Pay in the brick-and-mortar business. (That’s how it works in Finland.) That’s the way we all should think of technology.

Use Case -driven approach

The origins of SSI have been idealistic in some areas. The ISO mDL is the total opposite. Every single piece of idealism has been thrown away. Every design decision is hammered down to solve core use cases related to the use of mobile driver’s licenses. And no new technologies have been invented. Just put together features that we need.

I had to admit that it’s been refreshing to see that angle in practice after ivory towers of SSI ;-) For the record, there is still excellent work going on in the SSI area in general.

Differences Between mDL And SSI

mDL has almost a similar trust triangle as good old SSI-version.

But when you start to look more carefully, you’ll notice some differences, like the names of similar parties.

Concepts

ISO calls these roles as interfaces.

Connections

Also, connections between parties are different. Where SSI doesn’t allow direct communication between a verifier and an issuer, mDL explains that their communication is OK but optional. The only thing that matters is that the mDL Holder and mDL Reader can do what they need to execute the current use case. For example:

For offline retrieval, there is no requirement for any device involved in the transaction to be connected

‘Connected’ means connected to the internet. One of the scenarios is to support offline use cases, which makes sense if you think about instances where law enforcement officer needs to check the ID. That must be possible even when the internet connection is severed.

We continue with transport technologies at Mobile Driver’s License.

Revocations

We should also ask when that call-home is needed. The first and most obvious one is the validity checks. If the use case demands that the relying party checks that the mobile document (mDOC) is still valid on every use, a verifier can contact the issuer (Issuing Authority) and ask. All of this sounds pragmatic.

Call-home seems perfectly aligned with Finnish bank regulation and legislation, as far as I know. For example, the party who executes, let’s say, a transaction according to power-of-attorney (PoA) is the party who’s responsible for checking that a PoA is valid and not revoked. The responsibility to transfer revocation information is not the one who has given the PoA. It’s enough that the information is available for the party who relies on the PoA. It’s the relying party’s responsibility to access that information case-by-case.

It makes much sense and makes building revocation easier when you think about that. In summary, some call-home is the only way to make revocation requirements work. Note that the home isn’t necessarily the issuer, but it definitely can be seen as the Issuing Authority’s Infrastructure.

One schema

The most specific difference between mDL and SSI is that the schema is locked. It’s based on (mDOC) standard. That might first feel like a handicap, but the more you think about this, the better way it is to start implementing use cases in this area.

Mobile Driver’s License

mDL standard also has similarities to SSI, such as selective disclosure. But it and other features are designed with only one thing in mind: pragmatism. No Fancy Pancy features or saving-the-world idealism, just pure functionality.

The ISO standard defines the mDL standard, which is based on mDOC. The following diagram describes its most important architectural elements.

The diagram presents both logical and physical parts of the mDL architecture. At the bottom are supported data transport technologies: NFC/Bluetooth, Wi-Fi Aware (optional), etc. Transported data is CBOR coded, which guarantees the best possible performance. CBOR is a binary-coded data format optimized for limited resources and bandwidth.

Selective Disclosure

mDL’s selective disclosure is analog to SD-JWT’s mechanism, i.e., only disclosures’ digests are used when the issuer signs the credential document. That allows simple and easy-to-understand implementation, which is also efficient. At first glance, it only supports property-value pairs, but I don’t see why it couldn’t allow the use of hierarchical data structures as well. However, because the digest list is a one-dimensional array, it would prevent selection from inside a hierarchy.

No Need For ZKP

mDL doesn’t have ZKP but has solved similar use case requirements with the attestations. For example, the mDL issuer will include a set of age-over attestations into the mDL. The format of each attestation identifier is age_over_NN, where NN is from 00 to 99.

When mDL Reader sends the request, it can, for example, query the presence of the attestation age_over_55, and the response will include all the attestations that are equal to or greater than 55 years old. For example, if mDL doesn’t have age_over_55 but it has age_over_58 and age_over_65, it will send age_over_58.

Conclusion

mDL specification is excellent and ready for broad adoption. I hope we can build something over it ASAP. Unfortunately, the road I selected for PoCs and demos wasn’t possible because Apple’s ID Wallet requires that your device is registered in the US. There are ways to try it on an emulator, but it lacks too many core features to be interesting enough. Suppose you are asking why Apple and why not something else; the answer is that I’m looking at this on the operation system (OS) wallet track. Apple also has exciting features like Tap to ID.

The next step will be to study what we can learn from mDOC/mDL from the DID point of view. Is there some common ground between how mDL sees the world and how DIDComm and generic SSI sees the world—hopefully, the same world.

Until next time, see you!

Harri

Managing GitHub Branch Protections

GitHub has recently improved its automatic pull request handling and branch protection features. I have found these features handy, and nowadays, I enable them regardless of the project type to help me automate the project workflows and protect the code from accidental changes and regression bugs.

The features I find most useful are

• Require a pull request before merging: This setting enforces a model where the developers cannot accidentally push changes to the project’s main branches.
• Require status checks to pass before merging: This branch protection setting allows one to configure which status checks (e.g., unit tests, linting, etc.) must pass before merging the pull request.
• Allow auto-merge: This setting allows me to automate the merging of the PR once all the CI has run all the needed checks. I do not need to wait for the CI to complete the jobs. Instead, I can continue working on other things. In addition, I use this feature to merge, for example, dependency updates automatically. Note: auto-merging works naturally only when the branch protection checks are in place.

Until now, I have used GitHub’s branch protection feature to enable these additional shields. With this settings page, you can easily protect one or multiple branches by configuring the abovementioned options.

Branch Protection Prevents Releasing

However, when enforcing the branch protection, it applies to all users. That also includes the bot user I am using in many projects to release: creating a release tag, updating version numbers, and committing these changes to the main branch.

Suddenly, when the branch protection is enabled, the release bot cannot do its duties as it cannot push to the protected branch. Error states: Protected branch update failed for refs/heads/<branch_name>, X of X required status checks are expected.

Therefore, to overcome this problem, I have configured the bot to use pull requests. These workaround pull requests have slowed the process and made it unreliable. In some cases, I have been using a user token with administrative permissions to make the releases, which I want to avoid as it has evident problems in the security model.

Rulesets to the Rescue

Finally, this week, I reserved some time to investigate whether it is possible to avoid these limitations. I had two targets: first, I wanted to protect the main branch from accidental pushes so developers could make changes only via pull requests vetted by the CI checks. Second, I wanted the release bot to be able to bypass these rules and push the tags and version changes to the main branch without issues.

I googled for an answer for a fair amount of time. It soon became apparent that many others were struggling with the same problem, but also that GitHub had released a new feature called rulesets, intended to solve the problem. However, a few examples were available, and the configuration of the rulesets was not intuitive. Therefore, I have documented the steps below if you wish to use a similar approach in your project.

The instructions below are three-phased:

1. Creating a GitHub application for the release process operations
2. Configuring rulesets that protect the main branch but still allow releasing
3. Using the newly created GitHub application in the GitHub Actions workflow

GitHub Application

The first step is to create a GitHub application that handles the git operations in the CI release process for you.

Why to Use an Application?

There are multiple reasons why I chose to make a dedicated GitHub application instead of using a personal access token or built-in GitHub Actions token directly:

• The App installed in an organization is not attached to the user’s role or resource access as opposed to the personal access tokens.
• App does not reserve a seat from the organization. Creating an actual new GitHub user would reserve a seat.
• One can grant an application special permissions in rulesets. We want to treat all other (human) users similarly and only grant the bot user special access. This approach is impossible when using personal access tokens or built-in tokens.
• We want to activate other actions from pushes done by the releaser. For instance, if we create a tag with the releaser bot, we want the new tag to trigger several other actions, e.g., the building and packaging the project binary. If using a built-in GitHub Actions token, new workflows would not be triggered, as workflows are not allowed to trigger other workflows.

One can use GitHub Applications for multiple and more powerful purposes, but the releaser bot only needs minimal configuration as its only duty is to do the releasing-related chores.

1. Register Application

Start the application registration via user profile Developer settings or this link.

When creating the new application for the releasing functionality, the following settings need to be defined:

• Application name: e.g. releaser-bot
• Homepage URL: e.g. the repository URL
• Untick Webhook/Active, as we don’t need webhook notifications.
• Choose permissions: Permissions/Repository/Contents/Read and write.

  

• Choose selection: Where can this GitHub App be installed? Note: If you want to use the application in an organization’s repositories, make it public.
• Push Create GitHub App.

  

2. Download Private Key

After creating the app, you will receive a note saying, Registration successful. You must generate a private key to install your GitHub App. Navigate to the private keys section and push the Generate a private key button.

The private key file will download to your computer. Store it in a secure place; you will need it later.

3. Install the Application

Before using the application in your repository’s workflow:

1. Install the app in the target repository. In the created application settings, go to the Install App section.
2. Select the user or organization for which you want to install the application.

   

3. Select if you wish to use the application in a single repository or all account repositories.

   

4. Push the Install button.

Remove Existing Branch Protections

The rulesets feature will work on behalf of the branch protection settings. To avoid having overlapping configurations, remove first any existing branch protections.

Rulesets

The next step is to create the rulesets.

I crafted the following approach according to the original idea presented in the GitHub blog. The goal is to protect the main branch so that:

1. Developers can make changes only via pull requests that have passed the status check test.
2. The releaser bot can push tags and update versions in the GitHub Actions workflow directly to the main branch without creating pull requests.

You may modify the settings according to your needs. For instance, you may require additional status checks or require a review of the PR before one can merge it into the main branch.

Configuration

First, we will create a rule for all users. We do not allow anyone to delete refs or force push changes. Go to the repository settings and select Rulesets:

1. Create a New ruleset by tapping the New branch ruleset.
2. Give the Main: all name for the ruleset.
3. Set Enforcement status as Active.

   

4. Leave Bypass list empty.
5. Add a new target branch. Include default branch (assuming the repository default branch is main).

   

6. In Rules section, tick Restrict deletions and Block force pushes.

   

7. Push the Create button.

Then, we will create another ruleset that requires PRs and status checks for any user other than the releaser bot.

1. Create a New ruleset by tapping the New branch ruleset.
2. Give the Main: require PR except for releaser name for the ruleset.
3. Set Enforcement status as Active.
4. Add your releaser application to the Bypass list.

   

5. Add a new target branch. Include default branch (assuming the repository default branch is main).
6. Tick Require a pull request before merging.

   

7. Tick Require status checks to pass and Require branches to be up to date before merging. Add test as a required status check.

   

8. Push the Create button.

Use Bot in GitHub Actions Workflow

The final step is configuring the release process to use our newly created GitHub application.

Add Secrets for Release Workflow

To create a token for the releaser bot in the GitHub Actions workflow, we must have two secret variables available.

Go to repository Settings / Secrets and variables / Actions. Create two new secrets:

• RELEASER_APP_ID: Copy and paste your GitHub application ID (app ID) from the GitHub application settings page.
• RELEASER_APP_KEY: Copy and paste the contents of the private key file downloaded earlier.

Generate Token with the GitHub Application

In the release workflow, generate a token with the GitHub Application. For this, you can use a special action, actions/create-github-app-token, that utilizes the secrets defined in the previous step.

You can use the generated token for the repository cloning step. One can access the token using the outputs of the token generation step. Since the repository gets cloned with the bot token, the bot user will perform subsequent git actions.

You can find the sample workflow in GitHub.

So, the release script can now push directly to the main branch as we use the releaser bot token and have configured a bypass rule for the bot user in the rulesets. At the same time, other users cannot push to the main branch but must create a PR for their changes.

Summary

That’s it! The above steps show how to automate project release workflow in GitHub Actions with a dedicated bot user while still having repository branch protections to shield from accidental changes and unvetted code. I hope the instructions are of use to you. I’m always happy to have comments and ideas for improvement; you can contact me on LinkedIn!

Laura

Path to Passwordless

Passkeys and security keys have gained more and more popularity lately, and no wonder – they provide much more security and usability than passwords. These authentication methods utilize strong and phishing-resistant public key credentials that the keys or authenticators, as we call them, can create automatically. However, implementing passwordless support in your web application, or even replacing passwords altogether, might initially seem overwhelming.

FIDO2 and WebAuthn Standards Ease the Job

Integrating the authenticators into your web application happens through common standards created by the FIDO alliance. It is good news for us application developers: we don’t have to care about the dirty details of each authenticator implementation. Our job is to take into use the platform or browser APIs that enable the creation and use of the authenticator-managed credential. The operating system or browser handles needed user interaction and shows standard dialogs when appropriate.

Furthermore, we need to have a backend service in place that is capable of storing and verifying these credentials so that the authentication can take place correctly. The service must support the W3C WebAuthn standard so that the backend API responses are compatible with the client-side authenticator logic. The backend service is called quite often as a FIDO2 server.

To Buy or To Build?

Choosing one of many authentication service providers may be the most straightforward path on your passwordless journey. A service provider typically gives you access to a backend as a SaaS or a product you host yourself. The backend has the above capabilities to store and verify your users’ public keys. In addition to backend functionality, the service providers offer custom client libraries that enable you to add matching authenticator support to your application, whether a web or a native application.

Another option is to implement the WebAuthn support yourself. Some excellent open-source libraries already exist that ease the development of your backend service’s functionality for public key handling. Browser support for WebAuthn capabilities is rather good, and the integration to the web application is straightforward once the backend is in place. One can utilize dedicated client libraries for native applications (for example, iOS, Android, and Windows).

Example of Getting Hands Dirty

In our OSS decentralized identity agency project, we implemented WebAuthn logic ourselves. We created the authentication functionality in our backend server and web application. The core functionality consists of two features: registering a new user and authenticating an existing user (and returning an access token after a successful authentication). Of course, an end-product would have more features, e.g., for adding and removing authenticators but starting with the core features is the simplest.

We wrote the authentication server in Go. It utilizes go-webauthn library. Both of the core features need two API endpoints. On the client side, we use the navigator credentials JavaScript API in a React application.

The following sequence graphs demonstrate how the logic flows in more detail and describe the needed functionality at a high level.

User Registration

The first core feature is user registration. The user creates a new public-private key pair. The authenticator saves the private key to its secure storage, and the application sends the public key to the service backend.

sequenceDiagram
    autonumber
    participant Client
    participant Server
    Client->>Server: Request for credential creation options.<br/>/attestation/options
    Server-->>Client: Return credential creation options.
    Note left of Client: Create credential with received options.<br/>navigator.credentials.create
    Client->>Server: Send client data and public key to server.<br/>/attestation/result
    Note right of Server: Validate data. <br/> Store the public key and the credential id, <br/> and attach to user.
    Server-->>Client: Registration OK!

User Authentication

Another core feature is user authentication. The user creates a signature utilizing the authenticator-provided private key. The service backend verifies the signature using the stored public key and provides access if the signature is valid.

sequenceDiagram
    autonumber
    participant Client
    participant Server
    Client->>Server: Request for credential request options.<br/>/assertion/options
    Server-->>Client: Return credential request options.
    Note left of Client: Get credential with received options. <br/> Create signature with the private key.<br/>navigator.credentials.get
    Client->>Server: Send assertion to server.<br/>/assertion/result
    Note right of Server: Validate data and verify signature.
    Server-->>Client: Return access token.

For more information on how and which data is handled, see, for example, the WebAuthn guide. You can also find our client application source codes and the authentication service implementation on GitHub. You can also read more about how our project has utilized FIDO2 from this blog post.

As the example above shows, implementing a passwordless is not impossible. However, as with any new technology, it takes time and dedication from the team to familiarize themselves with the new logic and concepts. In addition, as there are strict security requirements, testing the authentication flows in a local development environment might be challenging at times, as the client and server need to belong to the same domain. The local testing environment is something that the team should resolve and enable together already early in the project so that it will not become a bottleneck.

Not All Things Are Standardized

One might still find using an external authentication service provider a safer bet. Especially when there is a need to support a variety of native devices, it may be tempting to pay someone to deal with the device-specific problems. However, in this case, there is one essential thing to notice.

Usually, the service provider solutions are proprietary. It means that the interaction between the client application and the backend API happens in a manner defined by the service provider. Even though the WebAuthn requests and responses are standardized, the payload wrapper structures and API endpoint paths depend on the implementation. There exists a FIDO recommendation on how the server API should look like, but it is a mere recommendation and not all service providers follow this guidance. Therefore, you cannot mix and match client and server implementations but will likely end up in a vendor lock when choosing a proprietary solution.

The path to passwordless is a journey that all web application developers will eventually travel. Will your team take the steps sooner rather than later?

Test Coverage Beyond Unit Testing

Automated testing is your superpower against regression. It prevents you from breaking existing functionality when introducing new code changes. The most important aspect of automated testing is automation. The testing should happen automatically in the continuous integration (CI) pipeline whenever developers do code changes. If the tests are not passing, the team shouldn’t merge the changes to the main branch. I often regard the automated test set as an extra team member – the team’s tireless knight against regression.

Complementing Unit Tests

The automated test set usually consists of many types of testing. Unit tests are the bread and butter of any project’s quality assurance process. But quite often, we need something more to complement the unit tests.

We usually write simulation code for external functionality to keep the unit tests relevant, modular, and fast to execute. However, wasting the team’s resources on creating the simulation code doesn’t always make sense. Furthermore, we don’t include simulation code in the end product. Therefore, we should also have testing that verifies the functionality with the proper external dependencies.

Application (integration/e2e) tests execute the software without unit testing tooling in the software’s “real” running environment and with as many actual external dependencies as possible. Those tests validate the functionality from the end user’s point of view. They complement unit tests, especially when verifying interoperability, asynchronous functionality, and other software features that are difficult or impossible to test with unit testing.

Keeping the Knight in Shape

Test coverage measurement is the process that can gather data on which code lines the test executes. Typically, this tooling is available for unit tests. Although coverage measurement and result data visualization can be handy in a local development cycle, I see the most value when the team integrates the coverage measurement with the CI pipeline and automated monitoring tooling that utilizes the coverage data.

The automated monitoring tool is an external service that stores the coverage data from the test runs over time. When the team creates new pull requests, unit tests measure the coverage data, and the pipeline sends the data to the monitor. It can then automatically alert the team if the test coverage is about to decrease with the new changes.

Thus, the automated coverage monitoring keeps the team’s knight in good shape. Developers cannot add new code without adding new tests to the knight’s toolbox. Or at least the decision to decrease the coverage needs to be made knowingly. Forgetting to add tests is easy, but it gets trickier when someone nags you about it (automatically).

However, as stated above, we are used to measuring the coverage only for unit test runs. But as we also test without the unit test tooling, i.e., application tests, we would like to gather and utilize the data similarly for those test runs.

Coverage for Application Tests

Go 1.20 introduced a new feature flag for the go build command, -cover. The new flag enables the developer to build an instrumented version of the software binary. When running the instrumented version, it produces coverage data for the run, as with the unit tests.

# build with –cover flag
go build –cover -o ./program

# define folder for result files
export GOCOVERDIR=”./coverage”

# run binary (and the application tests)
./program

# convert and analyze result files
go tool covdata textfmt –i=$GOCOVERDIR –o coverage.txt

Using the new feature means we can include the test coverage of the application test runs in the automated monitoring scope. We put this new feature to use in the Findy Agency project.

Instrumenting the Container Binary

Measuring the test coverage for application tests meant we needed to refactor our application testing pipeline so that CI could gather the data and send it to the analysis tool.

• We use a shared GitHub action to execute the same application acceptance test set for all our microservices. The action handles setting up the agency environment and running the test set.
• Service Docker images used in the application tests must have the instrumented binary. We modified the Docker image build phase to include the new -cover flag.
• We define the GO_COVERDIR environment variable in the service image, and the environment definition maps that container path to a folder in the local environment.
• Once the agency is running, GitHub Action executes the tests. After test execution, CI copies the result files, converts and sends them to the analysis tool as with unit tests.

Increased Coverage

The change has the expected effect: coverage increased in our repositories where we introduced this change. The coverage counts now in previously unmonitored code blocks, such as code in the main function that was missing unit tests. The change improves our understanding of our testing scope and helps us to keep our automated test set in better shape.

You can inspect the approach, for example, in the findy-agent-vault repository CI configuration.

Note! The coverage data is lost if the process is forced to be terminated and it doesn’t handle the termination signal gracefully. Ungraceful termination prevents the added instrumentation code from writing the coverage file. Therefore, the service needs to have graceful shutdown in place. Also, the Docker container runtime needs to pass the SIGTERM signal to the server process for this approach to work.

How To Write Modifiable & Readable Go Code

Since jumping on the OSS (Open Source Software) wagon, I have been learning new things about software development and getting more evidence to do certain things in a specific way.

Two of my favorite ’things’ at the code level are readability and modifiability. The latter is a very old friend of SW architecture’s quality attributes (Software Architecture in Practice, Len Bass, et al.). Still, it is not well aligned with the current practices and tools in the OSS scene because everything is so text-centric. Practise has taught that software architecture must be expressed in various notations; most are visual.

“A picture is worth a thousand words”

Most of us can reason a well-grafted state-machine diagram much faster than a code written our favorite programming language. For instance, the next state-machine diagram’s protocol implementation is constructed of thousands of lines of code structured in multiple files that depend on external modules and libraries. We need abstraction layers and modules to manage all that complexity.

@startuml

title Protocol State Machine

state "Waiting Start Cmd/Msg" as wstart

[*] -> wstart
wstart --> PSM: InvitationCmd
wstart --> PSM: ConnReqMsg

state "ConnectionProtocolRunning" as PSM {

	state "Sending ConnReq" as sreq
	state "Waiting ConnResp" as wresp

	'[*] -left-> sreq: InvitationCmd
	[*] --> sreq: InvitationCmd

	sreq --> wresp : http/200
	sreq: do/send until http/200
	
	wresp: exit/return http 200

	wresp --> [*]: ConnRespMsg

||

	state "Sending ConnResp" as sresp

	[*] --> sresp: ConnReqMsg

	sresp: do/send until http 200

	sresp --> [*]: http/200
}

PSM --> [*]

@enduml

Expressing things with the control-flow structures of imperative (or functional) programming languages is more challenging—especially when the correctness of the design should be verified. It seems that it’s easy to forget software quality attributes during fast-phasing programming if we use tools that only manage sequential text, i.e., code. At the code level, we should use functions that give us an abstraction hierarchy and help us to maintain modifiability—and readability as well, of course.

Moreover, since my studies of SW architecture’s quality attributes, I have understood that modifiability is more than modularity, re-usability, or using correct architectural styles like pipe and filter. Now we understand the importance of TTD, continuous deployment, DevOps, etc. These practices don’t work only on one of the engineering expertise. The best results are achieved through multi-disciplinary systems engineering practices.

At NASA, “systems engineering” is defined as a methodical, multi-disciplinary approach for the design, realization, technical management, operations, and retirement of a system. — NASA Systems Engineering Handbook

In this and my previous performance post, I’ll explain what multi-disciplinary software development practices mean when your development is fast-phasing OSS.

Code Modifiability

Most of the computer software is never ready. During its life cycle, it is under continuous changes: new features are needed, bugs must be fixed, technical debt needs to be amortized, etc.

In modern OSS-based software development, modifiability can be thought of as refactorability. Why?

Because we want to follow The Boy Scouting Rule:

Always leave the code you are working on a little bit better than you found it.

An excellent example of this rule is that when you find a bug, before fixing it to the mainstream, implement automatic tests to reproduce it.

Refactorability

Go programming language is the most refactor-friendly of them all I have used.

1. integrated test harness
2. benchmark tests
3. package naming
4. orthogonality
5. interface structure (no implementation declaration needed)
6. no self or this reserved words, you name object instances yourself
7. actual hybrid language combining OOP and functional programming features concentrating on simplicity
8. batteries included -standard library
9. type polymorphism, i.e., generics

I don’t think that’s even all of the Go’s features that help refactor your code, but these are the fundamentals in the critical order.

Like any other programming language, Go isn’t perfect. The current error-handling mechanism and strict community with their ‘idiomatic’ language policies restrict some of Go’s refactoring capabilities. But you can avoid them by using helper packages and your brains.

And, of course, there are two new language (main) releases every year. Let’s hope that upcoming versions help us keep our Go projects refactorable and maybe even help a little more.

Code Readability

We want to maximize our code’s readability. One of the Go code’s problems is that it overuses the if-statement, which prevents you from noticing the algorithm’s critical decision points.

For example, Go’s standard library includes quite many of the following code blocks:

func doSomething(p any, b []byte) {
    if p == nil {
        panic("input argument p cannot be nil")
    }
    if len(b) == 0 {
        panic("input argument p cannot be nil")
    }
    ...
    err := w.Close()
    if err != nil {
        log.Fatal(err)
    }
}

It’s easy to see that together with Go’s if-based error checking, these two hide the happy path and make it difficult to follow the algorithm and skim the code. The same thing can be found from Go’s unit tests if no 3rd party helper package is used:

     for _, tt := range tests {
          t.Run(tt.name, func(t *testing.T) {
               r := <-Open(tt.args.config, tt.args.credentials)
               if got := r.Err(); !reflect.DeepEqual(got, tt.want) {
                    t.Errorf("Open() = %v, want %v", got, tt.want)
               }
               w := r.Handle()
               if got := (<-Export(w, tt.args.exportCfg)).Err(); !reflect.DeepEqual(got, tt.want) {
                    t.Errorf("Export() = %v, want %v", got, tt.want)
               }
               if got := (<-Close(w)).Err(); !reflect.DeepEqual(got, tt.want) {
                    t.Errorf("Close() = %v, want %v", got, tt.want)
               }
          })
     }

The above code block is from a different test than the block below, but I think you get the idea. I’m speaking fast skimming of code where simplicity and appearance help a lot. Very much similar to syntax highlighting.

func TestNewTimeFieldRFC3339(t *testing.T) {
     defer assert.PushTester(t)()

     var testMsg Basicmessage
     dto.FromJSON([]byte(timeJSONRFC3339), &testMsg)
     timeValue := testMsg.SentTime

     assert.INotNil(timeValue)
     assert.Equal(timeValue.Year(), 2022)
     assert.Equal(timeValue.Month(), time.September)
     assert.Equal(timeValue.Day(), 30)
}

That’s clear and easy to skim. It’s also straightforward to extend to use table testing. I left it for the reader to figure out how that code renders without an assert package.

Conclusion

Go is built for hyper-large projects. Significant projects in every dimension: software engineers, code lines, modules, function points, whatever. And it does a pretty damn good job on that.

Unfortunately, the Go community isn’t as open and welcoming as, for example, Rust’s community. (Not my words directly, but I share the opinion). Remember, it’s your code and your project to maintain. So, keep your head and remember:

The whole problem with the world is that fools and fanatics are certain of themselves, while wiser people are full of doubts. — not entirely Bertrand Russell

How To Write Performant Go Code

I suppose all of us programmers have heard of the infamous premature optimization:

Premature optimization is the root of all evil in programming.

Really? I don’t think so.

The full quote from the must-read The Art of Computer Programming by Donald Knuth:

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.

Like so many pearls of wisdom, they are a child of their own time. They are usually dangerously separated from their context to underline the message the next author wants to emphasize their statement. I believe most of us have only read the shortened, i.e., wrong version of the quotation.

I claim that if keeping performance is your second nature, it’ll not ruin the other quality attributes of your code, but the opposite. All you need to do is to follow some basic rules with your muscle memory.

Performance Rules

In this post, I concentrate only on these three:

1. A function call is computationally expensive if the compiler cannot inline expanse it, i.e. do function inlining. When inlining, the compiler produce machine code, that doesn’t include real sub-routine calls with argument transportation either stack or CPU registers, and unwinding the stack and copy results to desired memory locations after the call returns. With function inlining you can think that the compiler copy/paste your function’s machine instructions to all of those places where it’s called.
2. Heap allocations are computationally expensive. (We out-scope garbage collection algorithms because they’re such a significant topic that even one book is insufficient. Still, it’s good to know that heap allocation pressurize a garbage collector.)
3. Minimize the problem space at every level of abstraction and the need for variables, i.e. especially in inner loops. Consider what parts of inputs are really varying and what parts are constant. For example, think twice if you need regex inside of your program.

Function Inlining

Let’s write our version of the famous assert function to show how function inlining can help readability outside of the tests without sacrificing performance.

func assert(term bool, msg string) {
    if !term {
        panic(msg)
    }
    ...

func doSomething(p any, b []byte) {
    assert(p != nil, "interface value cannot be nil")
    assert(len(b) != 0, "byte slice cannot be empty (or nil)")
    ...
    // continue with something important

By writing the benchmark function for assert with Go’s testing capabilities, you can measure the ‘weight’ of the function itself. You get the comparison point by writing the reference benchmark where you have manually inline-expansed the function, i.e. by hand. It would look like this:

func doSomethingNoAssert(p any, b []byte) { // for benchmarks only
    if p != nil {
        panic("interface value cannot be nil")
    }
    if len(b) != 0 {
        panic("byte slice cannot be empty (or nil)")
    }
    ...
    // continue with something important

Note, this would be your reference point only. (I’ll show how to turn off inlining with Go compiler flags, which would work as a good benchmarking reference for some cases.)

And, if you aren’t interested in the actual performance figures but just the information about successful inline expansion done by the compiler, you can ask:

go test -c -gcflags=-m=2 <PKG_NAME> 2>&1 | grep 'inlin'

The -gcflags=-m=2 gives lots of information, but we can filter only those lines that contain messages considering the inlining. Depending on the size of the packages there can be an overwhelming lot of information where most of them aren’t related to the task in your hand. You can always filter more.

The -gcflags will be your programming buddy in the future. To get more information about the flags, run:

go build -gcflags -help

Naturally, you can use compiler to give you a reference point for your inline optimizations as I said before.

Disable all optimizations:

go test -gcflags='-N' -bench='.' <PKG_NAME>

Disable inlining:

go test -gcflags '-l' -bench='.' <PKG_NAME>

Memory Allocations

Similarly, as function calls, the memory allocations from the heap are expensive. It’s good practice to prevent unnecessary allocations even when the programming platform has a garbage collector. With the Go, it’s essential to understand the basics of memory management principles Go uses because of memory locality, i.e., it has pointers and value types. Many other garbage-collected languages have object references, and the memory locality is hidden from the programmer, leading to poor performance, e.g., cache misses.

But nothing comes for free—you need to know what you’re doing. Go’s compiler analyzes your code and, without your help, can decide if a variable is escaping from its scope and needs to be moved from a stack to the heap.

Go’s tools give you extra information about escape analyzes. Use the -gcflags=-m=2 again, but grep escape lines from the output. That will tell you exactly what’s going on with the pointers for every function in the current compilation.

Usually, when benchmarking Go code, it’s good to understand what’s going on with heap allocations. Just add the following argument e.g., your test benchmark compilation, and you get the statistics of allocations in the benchmark run.

go test -benchmem -bench=. <PKG_NAME>

The -benchmem flag inserts two columns to benchmarking results:

Please note that five (5) columns are now instead of standard three. The extra two (rightmost and marked with red rectangle) are the about memory allocations. B/op is the average amount of bytes per memory allocation in the rightmost column allocs/op.

Fewer allocations, and the smaller the size of the allocations, the better. Please note that the performance difference between the above benchmark results isn’t because of the allocations only. Most of the differences will be explained in the following chapters. But still, allocations are something you should be aware of, especially about the variable escaping if it leads to heap allocations.

How Dynamic Are The Inputs?

How much do the variables in your program change, or maybe they are constant? Naturally, the smaller the actual input set of the function, the better chance we have to optimize its performance because the more deterministic the solution will be. Also, smaller machine code performs better in modern memory-bound CPUs. The same cache rules apply to instructions as variables. CPU doesn’t need to access RAM if all the required code is already in the CPU.

The above benchmark results are from two functions that do the same thing. This is the regex version of it (first row in the benchmark results):

var (
	uncamel = regexp.MustCompile(`([A-Z]+)`)
	clean   = regexp.MustCompile(`[^\w]`)
)

// DecamelRegexp return the given string as space delimeted. Note! it's slow. Use
// Decamel instead.
func DecamelRegexp(str string) string {
	str = clean.ReplaceAllString(str, " ")
	str = uncamel.ReplaceAllString(str, ` $1`)
	str = strings.Trim(str, " ")
	str = strings.ToLower(str)
	return str
}

Go’s regex implementation is known to be relatively slow, but if you think that regex needs its compiler and processor, it’s not so surprising.

The hand-optimized version of the Decamel function is almost ten times faster. It sounds pretty much like it’s natural because we don’t need all the versatility of the entire regex. We need to transform the inputted CamelCase string to a standard lowercase string. However the input strings aren’t without some exceptions in our use case because they come from the Go compiler itself. (The inputs are from Go’s stack trace.) And still, the input set is small enough that we quickly see the difference. And now, we can shrink the problem space to our specific needs.

The 1000%-faster version of Decamel that’s still quite readable:

func Decamel(s string) string {
     var (
          b           strings.Builder
          splittable  bool
          isUpper     bool
          prevSkipped bool
     )
     b.Grow(2 * len(s))

     for i, v := range s {
          skip := v == '(' || v == ')' || v == '*'
          if skip {
               if !prevSkipped && i != 0 { // first time write space
                    b.WriteRune(' ')
               }
               prevSkipped = skip
               continue
          }
          toSpace := v == '.' || v == '_'
          if toSpace {
               if prevSkipped {
                    continue
               } else if v == '.' {
                    b.WriteRune(':')
               }
               v = ' '
               prevSkipped = true
          } else {
               isUpper = unicode.IsUpper(v)
               if isUpper {
                    v = unicode.ToLower(v)
                    if !prevSkipped && splittable {
                         b.WriteRune(' ')
                         prevSkipped = true
                    }
               } else {
                    prevSkipped = false
               }
          }
          b.WriteRune(v)
          splittable = !isUpper || unicode.IsNumber(v)
     }
     return b.String()
}

Let’s take another example where results are even more drastically faster, but the reason is precisely the same. The input set is much smaller than what the first implementation function is meant to be used.

The results:

The first implementation:

func oldGoid(buf []byte) (id int) {
     _, err := fmt.Fscanf(bytes.NewReader(buf), "goroutine %d", &id)
     if err != nil {
          panic("cannot get goroutine id: " + err.Error())
     }
     return id
}

The above code is quite self explanatory, and that’s very good.

The second and fasterfastest implementation:

func asciiWordToInt(b []byte) int {
     n := 0
     for _, ch := range b {
          if ch == ' ' {
               break
          }
          ch -= '0'
          if ch > 9 {
               panic("character isn't number")
          }
          n = n*10 + int(ch)
     }
     return n
}

These two functions do precisely the same thing, or should I say almost because the latter’s API is more generic. (In a way, we are both narrowing and widening the scope simultaneously, huh?) The converted integer must start from the first byte in the slice of ASCII bytes.

It is much over 100x faster! Ten thousand percent. Why?

Because the only thing we need is to process the ASCII string that comes in as a byte slice.

You might ask whether this ruined the readability, which is fair. But no, because the function asciiWordToInt is called from GoroutineID, which is just enough—trust abstraction layering. (See the rule #1.)

Next time you are writing something, think twice—I do 😉

P.S.

There is so much more about performance tuning in Go. This piece was just a scratch of the surface. If you are interested in the topic, please get in touch with our project team, and we will tell you more. We would be delighted if you join our effort to develop the fastest identity agency.

GopherCon UK 2023

At the beginning of this year, I set myself a target to speak at a Go programming language conference. There were several reasons to do so. Go has been one of my favorite tools for years, and I have longed for an excuse to join a Go event. Giving a speech was perfect for that purpose. Plus, I have multiple excellent topics to share with the community as we do open-source development in our project, and I can share our learnings along with our code more freely. Furthermore, I want to do my part in having more diverse speakers at tech conferences.

As Go released 1.20, it inspired me to experiment with the new feature to gather coverage data for binaries built with Go tooling. I refactored our application testing pipelines and thought this would be an excellent topic to share with the community. I was lucky to get my speech accepted at GopherCon UK in London, so it was finally time to join my first Go conference.

The conference was held in the Brewery, a relaxed event venue in London City. The word on the conference halls was that the number of event sponsors had decreased from the previous year, and therefore, it had been challenging to organize the event. Luckily, the organizers were able to pull things still together.

Apart from the recession, the times are now interesting for Gophers. Many good things are happening in the Go world. As Cameron Balahan pointed out in his talk “State of the Go Nation,” Go is more popular than ever. More and more programmers have been adding Go to their tool pack in recent years, pushing language developers to add new and better features. Moreover, Go is not only a programming language anymore; it is a whole ecosystem with support for many kinds of tooling. Newcomers have a way easier job to start with development than, for example, I had seven years ago. Balahan stated that improving the onboarding of new developers is still one of the Go team’s top priorities. He mentioned that they are working on the libraries, documentation, and error messages to help newcomers and all Go developers be more productive.

Automated Testing and Test Coverage

The topic of my talk was “Boosting Test Coverage for Microservices.” I described in the presentation how vital automated testing has become for our small team. Automated testing is usually the part you skip when time is running out, but I tried to convince the audience that this might not be the best approach – missing tests may bite you back in the end.

Furthermore, I discussed test coverage in the presentation along with how one can measure test coverage for unit tests and now even - with the Go new tooling – for application tests, i.e., tests you run with the compiled application binary instead of unit testing tooling.

The audience received the talk well, and I got many interesting questions. People are struggling with similar issues when it comes to testing. It is tough to decide which functionality to simulate in the CI pipeline. Also, we discussed problems when moving on to automated testing with a legacy code base. The Go’s new coverage feature was unknown to most people, and some were eager to try it out instantly after my session.

Unfortunately, when you are a speaker at a conference, you cannot concentrate fully on the conference program because one needs to prepare for the talk. However, I was lucky enough to join some other sessions as well. There were mainly three themes that I gained valuable insights from.

Logging and tracing

Generating better logs and traces for applications seems to be a hot topic – and no wonder why. Services with high loads can generate countless amounts of data, and for the developers to use the logs for fixing issues efficiently, they must be able to filter and search them. The ability to debug each request separately is essential.

Jonathan Amsterdam from Google gave an inspiring presentation on the slog package, the newest addition to the standard library regarding logging. Go’s default logging capabilities have always lacked features. The missing log levels have been the greatest pain point in my daily developer life. More importantly, the ability to send structured data to analysis tools is crucial for production systems. Until now, teams have had to use different kinds of 3rd party libraries for this to happen.

Now, the slog package fixes these shortcomings, with the ability to handle and merge the data from existing structured logging tools. The presentation revealed how the team refined the requirements for the new package together with the developer community. Also, it was fascinating to hear which kind of memory management tricks the team used, as the performance requirements for logging are demanding.

Another exciting presentation handled also the capability of solving problems quickly, but instead of logs, the emphasis was on tracing. Tracing provides a more detailed view of the program’s data flow than logs and is especially useful when detecting performance bottlenecks. Konstantin Ostrovsky described how their team is using OpenTelemetry to add traceability to incoming requests. Using this approach, they do not need other logs in their codebase (excluding errors).

OpenTelemetry tracing uses the concept of spans in the code. One can utilize the spans to store the request data parameters and call relationships. Different analysis tools can then visualize this data for a single request. According to Konstantin, these visualizations help developers solve problems faster than searching and filtering ordinary logs. However, in the presentation Q&A, he reminded us that one should use the spans sparingly for performance reasons.

Service Weaver

Service Weaver is an open-source project that another Google developer, Robert Grandl, presented at the conference. The tool allows one to develop a system as a monolith, as a single binary, but the underlying framework splits the components into microservices in deployment time. Therefore, development is easier when you do not have to worry about setting up the full microservices architecture on your local machine. In addition, the deployment might be more straightforward when you can work at a higher level.

I participated in a workshop that allowed participants to try the Service Weaver hands-on. The target was to build a full-blown web application with a web UI and a backend service from which the UI could request data. Other sections described testing the weaver components, routing from one service to another, and even calling external 3rd party services.

The workshop suited me well; I could learn more than just listening to a presentation. Furthermore, the topic interested me, and I will dig into it more in the coming days to better understand which kind of projects would benefit from this kind of development model the most. The workshop organizer promises that Google will not stop investing in the product. They are searching for collaborators to get more feedback to develop the product further.

UI development with Go

Another topic that got my interest was a discussion group for UI development with Go. Andrew Williams hosted this discussion and presented a project called Fyne that allows Gophers to write applications with graphical user interfaces for several platforms. UI development is not my favorite thing to spend my time on; therefore, I am always curious to find better, more fun ways to implement the mandatory user-clickable parts. Using Go would undoubtedly click the fun box. So, I added another technology experiment to my TODO list.

In addition to these three themes, one session that handled JWT security was also memorable. Patrycja Wegrzynowicz hacked the audience live with the help of a small sample application she had built for this purpose. It demonstrated which kind of vulnerabilities we Go developers might have in our JWT implementations.

The presentation was informative and entertaining with the live hacking, and the audience understood the problems well due to the hands-on examples. The session proved that no well-known battle-tested documentation exists on handling the JWTs. We have (too) many different libraries with different qualities, and it is easy to make mistakes with the token generation and validation. No wonder the audience asked for a book on the subject from Patrycja – we need better resources for a topic as important as this.

See you in Florence

Overall, the event was well-organized, had a great atmosphere, and was fun to visit. Meeting fellow Gophers, especially the Women Who Go members, was a unique treat. Let’s set up our chapter in Finland soon. (If you are a Finland-based woman who writes Go code, please reach out!) I also got to spend some free time in London and share the World Cup final atmosphere with the English supporters cheering for their team.

Bye til the next event; I hope we meet in Florence in November! In the meantime, check out videos of the GopherCon UK 2023 sessions once they are published - I will do the same for the ones I missed live!

Beautiful State-Machines - FSM Part II

As I explained in my previous blog post, the idea of chatbots came quite naturally for our SSI development team. Think tanks or research labs are outstanding workplaces when you have something to chew. And how juicy topic the SSI has been, oh boy.

FSM chatbots started as a thought experiment, but now, when I think of the invention, it has been quite clever in foreseeing genuine user needs. It has lead us towards the following layer model.

The preceding drawing presents our ongoing layer architecture. It’s (still) based on DIDComm, but because our protocol engine implements a technology-agnostic API, we can change the implementation to something better without disturbing the above layers. The subject of this blog post is our FSM engine’s (3rd layer from bottom) CSP implementation that offers even more abstraction and helps the application development with no-code chatbot language.

Abstraction Layers

Implementing the FSM chatbots with our protocol-agnostic API has been eye-opening. With our API, we have solved most of the complexity problems and brought in an elegant communication layer that hides most of the horror of DID system, which, by to way, is not application development ready.

The objective of this blog post is to present a comprehensive exploration of my Go-based finite state-machine implementation that leverages Go’s CSP concurrency mechanism. This post will showcase the practical applications and benefits of using Go and CSP in building reliable and concurrent state management systems.

Concurrency \(\ne\) Parallelism

I have been using multithreading programming all my career and closely monitoring paradigms of multithreading programming since the 90s. When I started playing with Go’s channels and CSP programming model, I learned how easy it was compared to other asynchronous or multithreading programming models.

If I look at the phases I have gone thru, they follow these steps harshly:

1. Using interrupts to achieve concurrency. (MS-DOS drivers and TSRs, modem-based distributed systems, games, etc.)
2. Using the event-driven system to achieve concurrency. (Game development, GUI OS Win&Mac, transport-agnostic, i.e., modems, proprietary protocols for interconnection, Winsocks file transport framework, etc.)
3. Using OS and HW threads to achieve concurrency. (Unix daemon and Win NT service programming, C/S transport framework implementations, telehealth system implementation, etc.)
4. Sharing workloads between CPU and GPU to achieve maximum parallel execution. (The front/back-buffer synchronization, i.e., culling algorithms and LOD are running simultaneously in a CPU during the GPU is rendering the polygons of the previous frame, etc.)
5. Using libraries and frameworks (tasks, work queues, actor model, etc.) to achieve parallel execution for performance reasons. (Using TPL to instantiation (i.e., common transformation matrix) pipeline for tessellated graphic units, etc.)
6. Using frameworks to bring parallel execution to the application level, wondering why the industry starts taking steps back and prefers asynchronous programming models over, for example, worker threads. One answer: keep everything in the main thread. (iOS [recommended] network programming, Dart language, node.js, etc.)
7. Using CSP to hide HW details and still achieve a speedup of parallel execution, but only if it’s used correctly. (Example: Input routine -> Processing -> Output routine -pattern to remove blocking IO waits, etc.)

As you can see, distributed computing goes hand in hand with concurrency. And now distributed systems are more critical than ever.

And what goes around comes around. Now we’re using a hybrid model where the scheduler combines OS threads (preemptive multitasking) and cooperative event handling. But like everything in performance optimization, this model isn’t the fastest or doesn’t give you the best possible parallelization results for every algorithm. But it seems to provide enough, and it offers a simple and elegant programming model that almost all developers can reason with.

Note. Go offers runtime.LockOSThread function if you want to maximize parallelization by allocating an OS thread for a job.

FSM – State-Of-The-Art

Previous technology spikes have proven that state machines would be the right way to go, but I would need to try it with, should I say, complete state machines. And that was our hypothesis: FSM perfectly matches SSI and DIDComm-based chatbots.

One of my design principles has always been that never create a new language, always use an existing one. So, I tried to search for proper candidates, and I did find a few.

SCXML

The most promising and exciting one was SCXML, but I didn’t find a suitable embedded engine to run these machines, especially as an open-source. Nevertheless, very interesting commercial high-end tools supported correctness verification using proving theorems.

Lua

During these state-machine language searches, I discovered a Go-native implementation of Lua, which was open-source. Naturally, I considered using embedded Lua’s model of states. But I realized that some event engine would be needed, and it would have brought at least the following challenges:

1. Whole new API and bridge to Aries protocols would be needed. A simple C/S API wouldn’t be enough, but a notification system would also be required.
2. Lua is a Turing-complete language (correctness guarantees, the halting problem, etc.)
3. And final question: what extra would we be brought in when compared to offering Lua stubs to our current gRPC API, i.e., Lua would be treated the same as all the other languages that gRPC supports?

YAML

I soon discovered that YAML might be an excellent language for FSM because it’s proven to work for declarative solutions like container orchestration and many cloud frameworks.

Naturally, I tried to search OSS solution that would be based on YAML and would offer a way to implement a state-machines event processor. Unfortunately, I didn’t find a match, so we made our hybrid by offering two languages, YAML and Lua.

Lua is now included as an embedded scripting language for our FSM. It can be used to implement custom triggers and custom output events.

You can write Lua directly to YAML files or include a file link and write Lua scripts to external files. (Depending on the deployment model, that could be a security problem, but we are in an early stage of the technology. Our deployment model is closed. The final step to solve all security problems related to deployment and injection is when we have an integrated correctness tool.)

The Event Processor

Our FSM engine follows the exact same principle as the SCXML processor does:

An SCXML processor is a pure event processor. The only way to get data into an SCXML state machine is to send external events to it. The only way to get data out is to receive events from it.

In the software industry, there are some other similar systems, like, for example, eBPF, where correctness is provided without formal language. eBPF automatically rejects programs without strong exit guarantees, i.e., for/while loops without exit conditions. That’s achieved with static code analysis, which allows using conventional programming languages.

Now that we have brought Lua-scripting into our FSMs, we should also get code analysis for the exit guarantees, but let’s first figure out how good a fit it is. My first impressions have been excellent.

Services and Conversations

The drawing below includes our multi-tenant model. FSM instances and their memory containers are on the right-hand side of the diagram. The smaller ones are conversation instances that hold the state of each pairwise connection status. The larger one is the service state machine.

The service FSM is needed if the SSI agent implements a chatbot that needs to keep track of all of the conversations like polling, voting, chat rooms (used as an example), etc.

title Chat Room Example

actor "Dude" as Dude
participant "Dude" as ChatUI  <<chat UI>>
collections "Dudes" as ReadUI  <<read UI>>
collections "Conversation Bots" as PWBot <<pairwise FSM>>
control "Chat Room Service Bot" as Backend

== A Dude writes line ==
Dude -> ChatUI: Yo!

ChatUI -> PWBot: BasicMessage{"Yo!"}
|||
== Conversation FSM forwards msg to service FSM ==
PWBot -> Backend: BackendMsg{"Yo!"}
loop conversation count
   Backend -> PWBot: BackendMsg{"Yo!"}
== transport message thru pairwise connection ==
   PWBot -> ReadUI: BasicMessage{"Yo!"}
end
|||

As the sequence diagram above shows, the service FSM conceptually presents a chat room and works as a mediator between chat room participants.

The next Go code block shows how the above FSM instances are declared at the programming level. Note that ServiceFSM is optional and seldom needed. The rest of the data is pretty obvious:

• gRPC connection, conn
• conversation FSM, md
• and interrupt channel is given as a startup argument, intCh, because here, the machine is started from the CLI tool

	intCh := make(chan os.Signal, 1)
	signal.Notify(intCh, syscall.SIGTERM, syscall.SIGINT)

	chat.Bot{
		Conn:        conn,      // gRPC connection including JWT, etc.
		MachineData: md,        // primary FSM data, i.e. pairwise lvl
		ServiceFSM:  mdService, // optional backend/service lvl FSM
	}.Run(intCh)                // let machine to know when it's time to quit

By utilizing the CSP model, we embrace a paradigm that emphasizes the coordination and communication between independently executing components, known as goroutines, in a safe and efficient manner. This combination of Go’s built-in concurrency primitives empowers us developers to create highly concurrent systems while maintaining clarity and simplicity in their code.

CSP & Go Code

The app logic is in the state machines written in YAML and Lua. But surprising is how elegant the Go implementation can be when CSP is used for the state-machine processor. And all of that without a single mutex or other synchronization object.

The code block below is the crucial component of the FSM engine solution. It is presented as it currently is in the code repo, because I want to show honestly how simple the implementation is. Even the all lines aren’t relevant for this post. They are left for you to study and understand how powerful the CSP model for concurrency is.

func Multiplexer(info MultiplexerInfo) {
	glog.V(3).Infoln("starting multiplexer", info.ConversationMachine.FType)
	termChan := make(fsm.TerminateChan, 1)

	var backendChan fsm.BackendInChan
	if info.BackendMachine.IsValid() {
		b := newBackendService()
		backendChan = b.BackendChan
		b.machine = fsm.NewMachine(*info.BackendMachine)
		try.To(b.machine.Initialize())
		b.machine.InitLua()

		glog.V(1).Infoln("starting and send first step:", info.BackendMachine.FType)
		b.send(b.machine.Start(fsm.TerminateOutChan(b.TerminateChan)))
		glog.V(1).Infoln("going to for loop:", info.BackendMachine.FType)
	}

	for {
		select {
		// NOTE. It's OK to listen nil channel in select.
		case bd := <-backendChan:
			backendMachine.backendReceived(bd)

		case d := <-ConversationBackendChan:
			c, alreadyExists := conversations[d.ToConnID]
			assert.That(alreadyExists, "backend msgs to existing conversations only")
			c.BackendChan <- d
		case t := <-Status:
			connID := t.Notification.ConnectionID
			c, alreadyExists := conversations[connID]
			if !alreadyExists {
				c = newConversation(info, connID, termChan)
			}
			c.StatusChan <- t
		case question := <-Question:
			connID := question.Status.Notification.ConnectionID
			c, alreadyExists := conversations[connID]
			if !alreadyExists {
				c = newConversation(info, connID, termChan)
			}
			c.QuestionChan <- question
		case <-termChan:
			// One machine has reached its terminate state. Let's signal
			// outside that the whole system is ready to stop.
			info.InterruptCh <- syscall.SIGTERM
		}
	}
}

It runs in its goroutine and serves all the input and output at the process level. For those who come from traditional multi-thread programming, this might look weird. You might ask why a lock doesn’t make conversations map thread-safe. That’s the beauty of the CSP. Only this goroutine modifies conversations data—no one else.

You might ask if there’s a performance penalty in this specific solution, but there is not. The Multiplexer function doesn’t do anything that’s computationally extensive. It listens to several Go channels and delegates the work to other goroutines.

This model has proven to be easy to understand and implement.

Discrete State Transitions

As we saw, the Multiplexer function calls a function below, backendReceived, when data arrives from backendChan.

func (b *Backend) backendReceived(data *fsm.BackendData) {
	if transition := b.machine.TriggersByBackendData(data); transition != nil {
		b.send(transition.BuildSendEventsFromBackendData(data))
		b.machine.Step(transition)
	}
}

Both state-machine types (conversation/service level) follow typical transition logic:

1. Do we have a trigger in the current state of the machine?
2. If the trigger exists, we’ll get a transition object for that.
3. Ask the transition to build all send events according to input data.
4. Send all output events.
5. If previous steps did succeed, make a state transition step explicit.

“Channels orchestrate; mutexes serialize”

I’m not a massive fan of the idiomatic Go or Go proverbs. My point is that you shouldn’t need them. The underlying semantics should be strong enough.

Luckily the world isn’t black and white. So, let’s use one proverb to state something quite obvious.

“Channels orchestrate; mutexes serialize”

That includes excellent wisdom because it isn’t prohibiting you from using the mutexes. It clearly states that we need both, but they are used differently. That said, the below code block shows the elegance you can achieve with Go channels.

func (c *Conversation) Run(data fsm.MachineData) {
	c.machine = fsm.NewMachine(data)
	try.To(c.machine.Initialize())
	c.machine.InitLua()
	c.send(c.machine.Start(fsm.TerminateOutChan(c.TerminateChan)), nil)

	for {
		select {
		case t := <-c.StatusChan:
			c.statusReceived(t)
		case q := <-c.QuestionChan:
			c.questionReceived(q)
		case hookData := <-c.HookChan:
			c.hookReceived(hookData)
		case backendData := <-c.BackendChan:
			c.backendReceived(backendData)
		}
	}
}

If you have never seen a code mess where a programmer has tried to solve a concurrence task by only having common control flow statements like if-else, attempt to envision an amateur cook that tries to make at least four dishes simultaneously with poor multitasking capabilities. He has to follow recipes literally for every dish from several cookbooks. I think then you get the picture.

Conclusion

I hope I have shown you how well FSM and CSP fit together. Maybe I even encouraged you to agree that SSI needs better abstraction layers before we can start full-scale application development. If you agree we’re on the right path, please join and start coding with us!

The Agency Workshop

During the Findy Agency project, our development team initially placed greater emphasis on running and deploying the agency in terms of documentation. Due to this prioritization, instructions on building agency clients have gotten less attention. We have now fixed this shortcoming by publishing the agency workshop to educate developers on using, building, and testing agency clients.

Agency Clients

What exactly is an agency client? A client is a piece of software operated by an entity with an identity (individual, organization, or thing). In the agency context, we have three types of clients:

1. CLI: a tool for managing the identity agent in the cloud using the command line interface.
2. Web Wallet: a web-based user interface for holding and proving credentials through a web browser.
3. API client: any application that manages the identity agent with Findy Agency API.

The workshop shows how to use the ready-built clients (CLI tool and web wallet) and start the development using the API. The workshop participant can choose from two tracks, the CLI track, which demonstrates the agent operations with the CLI tool, and the code track, which concentrates on the API calls and testing them with the web wallet. Both workshop branches teach hands-on how SSI’s basic building blocks work. Consequently, the participant learns how to issue, receive and verify credentials utilizing the Findy Agency platform.

The code track instructs on building a simple web application that issues and verifies credentials.

The workshop material is suitable for self-studying. In this case, if the developer does not have an instance of the agency running in the cloud, using a localhost deployment with Docker containers is straightforward. The material is available in a public GitHub repository, containing step-by-step instructions that are easy to follow. Developers can choose the development environment according to their preferences, from native localhost setting to VS Code dev container or GitHub Codespaces.

Live Workshops

The Findy Agency team has organized the workshop also in a live setting. We invited developers from our company interested in future technologies to technical sessions for learning together. These sessions took place in both Helsinki and Oulu during Spring 2023.

The events started with Harri explaining the basic SSI principles and then Laura presenting demos of agency client applications that the team has built in the past. The introductory presentations generated many exciting questions and discussions that helped the participants better understand the concept.

After the introduction part, the participants could pick the track they were going to work on, and they had a chance to advance with the tasks at their own pace. The developers could use a shared agency installation in the cloud, so, in principle, setting up the development environment was relatively effortless.

Unfortunately, during the first session, we encountered challenges with the shared cloud agency and observed a few inconsistencies in the instructions for configuring the development environment. We addressed these findings before the second session was held, which proved highly successful. In fact, the second workshop was so seamless that the instructors found it almost uneventful, as nearly no participants needed assistance.

Both events received good feedback, as working hands-on helped the participants understand more deeply the idea behind the technology and how one can integrate the tooling into any application. The participants also thought these workshops were an inspiring way to learn new things. Future ideas were to organize a hackathon-style event where the teams could concentrate more on the actual use cases now that they understand the basic tooling.

Are You Ready to Take the Challenge?

We recommend the workshop for all developers who are interested in decentralized identity. The tasks require no special skills and have detailed instructions on how to proceed to the next step. Even non-technical people have been able to do the workshop successfully.

We are happy to take any feedback on how to make the material even better! You can reach us via SoMe channels:

Laura

Harri

The Time to Build SSI Capabilities Is Now

During hallway discussions with various teams and organizations in recent months, I have been surprised by the emphasis on the underlying technology in SSI implementations. Even non-technical folks are wondering, e.g., which credential format will overcome them all. Don’t get me wrong; it’s not that I don’t see the importance of choosing the most interoperable, best performant, and privacy-preserving cryptography. I am afraid more critical problems keep hiding when we concentrate only on technical issues.

Focus on the Use Cases

Teams implementing flows utilizing SSI capabilities should focus primarily on the use case and, more importantly, the user experience. If the flow requires an individual to use a wallet application and several different web services, many users have a steep learning curve ahead of them. It can even be too steep. Furthermore, we should also think outside of “the individual-user-box.” How can we utilize this technology in a broader range of use cases, for instance, inter-organizational transactions?

Therefore, we need more real-world use cases implemented with SSI technology to develop it correctly. After all, the use cases should ultimately be the source of the requirements, the drivers of how to bring the technology further. And if use case implementors are waiting for the technology to become ready, we have a severe chicken-egg problem.

Instead of overthinking the low-level crypto operations, etc., the teams should concentrate on the high-level building blocks of SSI. The starting point should clarify the roles of the issuer, holder, and verifier. What exactly happens when data is issued and verified? And how can the team apply the verifiable data in the use case in question: who are the issuer, holder, and verifier? Furthermore, what data does the issuer sign, and which part is available to the verifier?

It Is All About Communication

After figuring out answers to these questions, the selection of the technical stack becomes more relevant. However, even then, I wouldn’t emphasize the credential format selection. What is even more important is the used credential exchange protocol.

For instance, if your primary use case is user identification and authentication, and you wish to create an architecture that suits well and fast in the legacy world, the most straightforward choice is a protocol offering strict client-server roles and HTTP-request-response style API. In this style, the issuer, holder, and verifier roles are hard-coded, and one party cannot simultaneously play multiple roles.

However, choosing a symmetric protocol may widen the spectrum of your use cases. Participants can execute different functions during the interaction in a symmetric protocol. So, for example, in the user authentication scenario, in addition to the service (server) authenticating the user (client) based on verified data, the individual could also get a verification from the service-operating organization of its authenticity.

For more information, I recommend to familiarize with Daniel Hardman’s publications, for example the summary about the overlaps of different SSI technologies.

Integrate Carefully

Whichever stack you choose, there are two things to keep in mind

1. Implement the integration to the library or service providing SSI functionality following well-known modular principles so that it is easy to replace with a different library or service.
2. Do not let the underlying technology details contaminate your application code. Thus, ensure the SSI tooling you use hides the implementation details related to credentials formats and protocol messages from your application. This approach ensures that changes, for example, to the credential data format, have a manageable effect on your application.

The importance of layered architecture has been discussed also before in our blog.

Decentralized Identity Demo is a cool SSI simulation. Its original developer is Animo, and the SSI functionality was done using aries-framework-javascript. As an experiment for SSI technology portability, I took the open-sourced codes and converted the application to use Findy Agency instead of AFJ. You can find both the original codes and my version in GitHub.

The best option is to pick open-source tooling so that if some functionality is missing, you can roll up your sleeves and implement it yourself. If this is not an option, select a product that ensures compatibility with multiple flavors of SSI credentials and data exchange protocols so that interoperability with other parties is possible. After all, interoperability is one of the decentralization enablers.

The Time Is Now

The recent advancements in LLM technology and ChatGPT offer a glimpse of what’s possible when large audiences can suddenly access new technology that provides them added value. SSI can potentially revolutionize digital services similarly, and organizations should prepare for this paradigm shift. Is your organization ready when this transformation takes place?

In conclusion, now is the time to plan, experiment and build SSI capabilities. Begin by understanding the high-level building blocks and experimenting with the technology that best aligns with your use case. Opt for tools that support a layered architecture design, like Findy Agency. This approach prevents the need for significant alterations to the application layer as the technology evolves.

Digital Identity Hack Part 3 – Introduction to SSI Presentation

The rise of self-sovereign identity (SSI) presents a groundbreaking opportunity for progress and expansion in the financial sector, with the capacity to reshape digital financial services, simplify transactions, and facilitate seamless interactions. To capitalize on this potential, OP Lab and the EU funded Powered by Blockchain project organized a seminar day and online hackathon, exploring the innovative power of SSI technologies and institutional decentralized finance, setting the stage for a safer, more inclusive, and more efficient worldwide financial ecosystem.

At the seminar, participants engaged in a panel discussion on blockchain projects and a presentation delving into self-sovereign identity. The following section will examine the essential insights and highlights that emerged from the SSI presentation.

Introduction to Self-Sovereign Identity

In the event day presentation on Self-Sovereign Identity, Harri Lainio from OP Lab introduced SSI, its potential impact on the world, and Findy Agency’s open-source solution for implementing it. The speaker highlighted the power dynamics of the current digital identity model, which has centralized power in the hands of a few online platforms. In the next evolution of the internet, Web 3 aims to shift power back to individuals and things, thus empowering users to take control of their digital identities.

Learn more about Findy Agency here.

Take Control of Your Identity

SSI offers the promise of an internet where individuals are sovereign, building their own networks, and serving as the roots of these networks. By examining the incentives for SSI, the speaker noted that humans are cooperative but inherently distrustful, making the exchange of value challenging. Institutions like banks and governments have historically reduced uncertainty, but the speaker questioned whether there are alternative ways to achieve this. He acknowledged that blockchain technology had solved some issues, such as double spending, but still has a long way to go.

The speaker emphasized that SSI is about putting the user at the center of their digital identity: “I want to be in the center. And that’s something that SSI promises to handle.” By empowering individuals to take control of their digital selves, SSI can reshape the internet and transform how people interact and exchange value online.

Root of Trust

As the presentation progressed, the speaker highlighted that new digital identities could be represented by leveraging real-world entities while maintaining clarity and trustworthiness. He touched on the idea of using FIDO2 authenticators as a “root of trust” and the importance of flawless one-on-one mapping between the digital and real worlds.

Find more information about the FIDO2 here.

Placing the User at the Heart of Trust

The speaker introduced the concept of verifiable credentials and decentralized identifiers based on secure communication protocols, as well as the trust triangle, which consists of an issuer, holder, and verifier. He explained that the verifier’s trust in the issuer is crucial and that the current technology should be able to handle it. The presenter mentioned the Findy Agency they have been working on and encouraged the audience to explore and improve upon their open-source solutions.

In a memorable quote from the presentation, Harri said, “And with the help of cryptography, we can build a better or at least more interesting world, digital world.” By pushing the boundaries of SSI and digital identity, the speaker envisions a future where individuals have greater control over their digital selves and the information they share.

Project and Event Information

OP Lab arranged the seminar day in Oulu, Finland, and the two-week online hackathon in collaboration with the Powered by Blockchain project during February. Powered by Blockchain project is a collaboration project between University of Vaasa, University of Oulu, University of Lapland and Kajaani University of Applied Sciences. The project is funded by the European Social Fund.

Find more information about the project here.

Digital Identity Hack Part 2 – Panel Discussion on Blockchain Projects

The emergence of self-sovereign identity (SSI) offers a revolutionary opportunity for innovation and growth in the financial world, potentially transforming digital financial services, streamlining transactions, and enabling seamless interaction. To harness this potential, OP Lab and the EU-funded Powered by Blockchain project recently held a seminar day and online hackathon to investigate the disruptive capabilities of SSI technologies and institutional decentralized finance, paving the way for a more secure, inclusive, and efficient global financial ecosystem.

During the seminar, attendees participated in a panel discussion on blockchain projects and a presentation exploring self-sovereign identity. The following section will delve into the key insights and highlights from the panel discussion.

Panel Discussion on Blockchain Projects

On the event day, a panel of experts, Pekka Kaipio from OP Lab, Jani Partanen from Joisto Oy, Sandun Dasanayake from Q4US, and Rainer Koirikivi from Interjektio, shared their insights into the world of blockchain. The panel was moderated by Laura Vuorenoja from OP Lab, and the discussion centered on the challenges of blockchain projects, the lessons learned, and the potential of blockchain technology for digital identity.

Blockchain projects present unique challenges, with the panelists highlighting issues such as the complexity of the technology, regulatory hurdles, and the difficulty of building consortia. To overcome these challenges, the experts emphasized the importance of educating stakeholders about blockchain’s potential beyond cryptocurrencies and selecting the right use cases for blockchain technology.

The panelists also touched on their experiences with blockchain projects in various industries. Topics covered included the digitization of real estate trading, the facilitation of parcel information sharing across European entities, and the secure storage of document integrity and provenance. These examples demonstrated the versatility and growing importance of blockchain solutions in the modern tech and business landscape.

Exploring Identity Management

One of the focal points of the discussion was digital identity management on blockchain. The panelists explored the need for decentralized identity systems to improve the scalability of blockchain networks. However, they noted that many current projects still rely on centralized identity systems or traditional public key infrastructure.

Regulatory Challenges in the Blockchain Sphere

Panelists agreed that the General Data Protection Regulation (GDPR) has been one of the most significant regulatory challenges they have faced. While the GDPR aims to protect users’ privacy, it sometimes limits how blockchain can be utilized, even necessitating the removal of certain platform functionalities. In the financial sector, KYC (Know Your Customer) and AML (Anti Money Laundering) are the most obvious challenges that need to be implemented properly.

The Importance of Open Source

Many blockchain projects are built upon open-source technologies like Hyperledger Fabric. Panelists highlighted the importance of open-source contributions in advancing blockchain technology and fostering a spirit of collaboration within the community. While not all panelists’ companies have made code contributions, they have shared knowledge and participated in meetups and user groups.

Effective Talent Sourcing

Sourcing skilled blockchain talent has proven to be complicated. Panelists revealed that growing their own talent through training and education has been the most effective strategy. Some have also turned to partnerships with startups, consultants, or other companies to fill talent gaps.

Future Prospects of Blockchain Technology

The panelists agreed that blockchain technology shows promise for the future, as it matures and becomes more widely adopted. Blockchain’s potential as an infrastructure solution for industries like finance and the emerging metaverse was also noted. However, the panelists acknowledged that regulatory challenges and the rise of artificial intelligence (AI) could impact the technology’s trajectory.

One Wish for Blockchain

When asked about their one wish for blockchain, panelists expressed a desire for more streamlined regulations, increased adoption, and a broader talent pool. They hope these changes will lead to a more decentralized, fair, and trusted way for people to exchange value and information.

In conclusion, the panelists emphasized that despite its challenges, blockchain has the potential to revolutionize various industries, including finance, real estate, logistics, and digital identity management. By sharing their experiences and lessons learned, the panelists provided valuable insights to help others navigate the complex world of blockchain technology.

Project and Event Information

OP Lab arranged the seminar day in Oulu, Finland, and the two-week online hackathon in collaboration with the Powered by Blockchain project during February. Powered by Blockchain project is a collaboration project between University of Vaasa, University of Oulu, University of Lapland and Kajaani University of Applied Sciences. The project is funded by the European Social Fund.

Find more information about the project here.

Digital Identity Hack – Unlocking the Potential of SSI

The emergence of self-sovereign identity (SSI) technologies has the potential to revolutionize the digital landscape of the financial world, presenting unparalleled opportunities for innovation and growth. As SSI gains momentum, it can transform how we interact with digital financial services, streamline transactions, and enable seamless interactions between users and services. By embracing these cutting-edge technologies, we have the unique chance to actively shape the future of the global financial ecosystem, fostering a more secure, inclusive, and efficient digital environment for individuals and businesses alike.

To seize these opportunities, OP Lab and the EU-funded Powered by Blockchain project organized a seminar day and an online hackathon in collaboration. The aim was to explore the disruptive potential of SSI technologies and institutional decentralized finance.

Kicking Off the Hackathon Challenge

The event’s main theme was Digital Identity as an Enabler for Open Decentralized Finance. The emergence of SSI technologies has opened new opportunities by enabling verifiable credentials for identity and data verification. The hackathon and seminar day aimed to educate participants on leveraging these technologies to transform the world of finance. The live event, bringing together tech professionals, students, and companies, was recently held in Oulu, Finland, and streamed online for a broader audience to participate.

Read the full Digital Identity Hack description here.

Challenge Objectives and Key Topics

The objective of the hackathon challenge was to develop financial products and services that are transparent, open, and accessible to anyone on the internet. The organizers believed that SSI technologies could offer significant tools for achieving this goal.

Key topics covered during the challenge presentation included combining verifiable credentials with blockchain technology, checking identity and other data attributes in transactions, controlling transactions automatically through smart contracts based on verifiable data, and creating confidential private transactions while verifying identity data.

Participants were also presented with example approaches to the challenge, such as how to verify financial compliance like Know Your Customer (KYC) and Anti-Money Laundering (AML), how to check company employee credentials to proceed with company transactions, and how to verify investor accreditation.

Inspirational Case Studies

During the event, several intriguing proof-of-concept case examples were highlighted, providing valuable insights into the promising potential of this innovative and uncharted domain.

Project Guardian

Onyx by J.P. Morgan explored the use of verifiable credentials and blockchain technology in foreign exchange transactions through Project Guardian. This concept demonstrated that by utilizing tokenized forms of cash and proper authorization, it’s possible to create decentralized financial services with the necessary safeguards in place.

Find more information about the project here.

Project Jupiter

An intriguing case from Finland, Project Jupiter, explored trading non-listed company shares with SSI and blockchain technology. The network simplifies the stock ledger, issuance, and trading processes while ensuring seamless information flow between stakeholders. This innovation reduces administrative burdens and enables new financial products and aftermarket services.

Find more information about the project here.

Submission Evaluation and Objectives

The hackathon submissions were expected to be technical solutions rather than just concepts, with must-haves including a five-minute video presentation with a technical demo and shared code. Evaluation criteria were based on technical implementation, innovativeness, scalability, and commercial potential.

The awards ceremony took place two weeks after the kick-off in a separate online event, where participants had the opportunity to pitch their solutions and winners of the challenge were announced. The hackathon was challenging, as it required participants to implement new technologies and produce innovative solutions for specific business challenges.

A major focus of the hackathon was the implementation of SSI with the Findy Agency to create a new generation of online solutions. Findy Agency empowers individuals with control over their digital identities, allowing them to share their information securely and selectively.

Learn more about Findy Agency here.

Team Submissions

The team submissions demonstrated a great variety of interesting future solutions. One team of innovators tackled the financial sector’s pressing issue of global online identity verification for salary payments. By developing a global digital employee verification solution, they streamlined the process and reinforced compliance, thus easing the challenges faced by institutions in confirming identities across borders.

Another topic addressed in the team submissions was the urgent need for innovative solutions to combat climate change. One standout concept involved a circular economy for environmental certificates, making it possible to invest in or directly sell forest resources. This model promotes sustainable forest management and encourages participation in green initiatives.

One of the submissions also displayed a practical solution for job applications using digital credentials, streamlining the process by providing verified information on work experience and education. This approach holds promise for simplifying the hiring process and increasing trust between job seekers and employers.

All in all, the hackathon demonstrated the incredible potential of self-sovereign identity and verifiable credentials in addressing critical issues faced by society today. As these tools continue to be refined and implemented, we can expect to see a more trusted and efficient digital landscape in the years to come.

Pioneering the Future of Technology

Self-sovereign identity and verifiable credentials are powerful tools that can create the much-needed trust in the online space. These tools allow individuals to own and manage their personal data and control how it is shared, making it more secure and reliable. With the rise of digital transformation, these tools are becoming increasingly important in ensuring the security and privacy of individuals and businesses online.

As the future of decentralized finance continues to unfold, this event offered a glimpse into the innovative ideas and technologies that may shape the industry in the coming years.

Project and event information

OP Lab arranged the seminar day in Oulu, Finland, and the two-week online hackathon during February in collaboration with Powered by Blockchain project. Powered by Blockchain project is a collaboration project between University of Vaasa, University of Oulu, University of Lapland and Kajaani University of Applied Sciences. The project is funded by the European Social Fund.

Find more information about the project here.

Deploying with CDK Pipeline

My previous post described how we have been using the native AWS IaC tools for defining and updating our PoC environment infrastructure. The story ended with taking AWS CDK v2 in use and switching the deployment process on top of CDK pipelines. In this article, I will describe the anatomy of our CDK pipeline in more detail.

Self-Mutating Pipelines

CDK Pipeline offers a streamlined process for building, testing, and deploying a new version of CDK applications. It tries to simplify the developer’s life by hiding the dirty details of multiple services needed to build working pipelines in AWS. As usual with CDK tools, it provides no new AWS services but is a convenient abstraction layer on top of the existing AWS continuous integration and deployment products.

The main idea of the pipeline is that instead of the developer deploying the application from her local computer, a process implemented through AWS CodePipelines (CDK pipeline) handles the deployment. Thus, in the agency case, instead of me running the script locally to create all needed AWS resources for our AWS deployment, I create locally only the CDK pipeline, which, in turn, handles the resource creation for me.

The CDK pipeline also handles any subsequent changes to the deployment (or even in the pipeline process itself). Therefore, developers modify the CDK deployment only through version control after the pipeline creation. This feature makes it self-mutating, i.e., self-updating, as the pipeline can automatically reconfigure itself.

This model reduces the need for running the tools from the developer’s local machine and enforces a behavior where all the deployment state information is available in the cloud. Using CDK pipelines also reduces the need to write custom scripts when setting up the pipeline.

The use of CDK Pipelines converted the Agency deployment model to a direction where the pipeline does most of the work in the cloud, and the developer only commits the changes to the version control.

Agency Deployment

Regarding the agency deployment, we have a single CDK application that sets up the whole agency platform to AWS. A more production-like approach would be to have a dedicated deployment pipeline for each microservice. However, having the resources in the same application is handy for the time being as we occasionally need to set the agency fully up and tear it down rapidly.

If one wishes to deploy the agency to AWS using the CDK application, there are some prerequisities:

• The needed tools for Typescript CDK applications
• AWS Codestar connection to GitHub via AWS Console
• A hosted zone for Route53 for the desired domain
• Agency configuration as environment variables.

The setup process itself consists of two phases. Firstly, one must store the required configuration to parameter store and secrets manager. Secondly, one should deploy the pipeline using CDK tooling. I have written bash scripts and detailed instructions to simplify the job.

Pipeline Stages

Each CDK pipeline consists of 5 different stages. It is essential to have a basic understanding of these stages when figuring out what is happening in the pipeline. CDK pipeline creates these stages automatically when one deploys the pipeline constructs for the first time. The developer can modify and add logic to some stages, but mainly the system has a hard-coded way of defining the pipeline stages. This functionality is also why AWS calls the CDK pipelines “opinionated.” Therefore some projects will find the CDK pipeline philosophy for building and deploying assets unsuitable.

1/5 Source

The source stage fetches the code from the source code repositories. In the agency case, we have five different source repositories in GitHub. Whenever we push something to the master branch of these repositories, i.e., make a release, our pipeline will run as we have configured the master branch as the pipeline trigger.

We don’t need the code for the backend services for anything but triggering the pipeline. We use only the front-end and infra repositories code to build the static front-end application and update the CDK application, i.e., the application containing the CDK code for the agency infrastructure and pipeline. GitHub actions handle building the backend Docker containers for us and the images are stored publicly in GitHub Packages.

2/5 Build

The build stage has two roles: it converts the CDK code to CloudFormation templates and builds any assets that end up in S3 buckets.

The developer can define this workflow, but the steps must produce the CDK synthesizing output in a dedicated folder. The phase when CDK tooling converts the CDK code to the CloudFormation templates is called synthesizing.

With the agency, we have some custom magic in place here as we are fetching the CDK context from the parameter store for the synthesizing. The recommendation is to store the context information in the CDK application repository, but we don’t want to do it as it is open-source.

Pipeline creation in the CDK application code:

    const pipeline = new CodePipeline(this, "Pipeline", {
      pipelineName: "FindyAgencyPipeline",
      dockerEnabledForSynth: true,
      // Override synth step with custom commands
      synth: new CodeBuildStep("SynthStep", {
        input: infraInput,
        additionalInputs: {
          "../findy-agent": CodePipelineSource.connection(
            "findy-network/findy-agent",
            "master",
            {
              connectionArn: ghArn, // Created using the AWS console
            }
          ),
          ...
        },
        installCommands: ["npm install -g aws-cdk"],
        ...
        // Custom steps
        commands: [
          "cd aws-ecs",

          // Prepare frontend build env
          "cp ./tools/create-set-env.sh ../../findy-wallet-pwa/create-set-env.sh",

          // Do cdk synth with context stored in params
          `echo "$CDK_CONTEXT_JSON" > cdk.context.json`,
          "cat cdk.context.json",
          "npm ci",
          "npm run build",
          "npx cdk synth",
          "npm run pipeline:context",
        ],
        ...
        // The output of the synthing process
        primaryOutputDirectory: "aws-ecs/cdk.out",
      }),
      ...
    });

The building of assets happens automatically as part of the synthesizing. The pipeline orchestrates it based on the instructions that one defines for the deployment assets.

3/5 UpdatePipeline

UpdatePipeline makes any changes to the pipeline, i.e., modifies it with new stages and assets if necessary. The developer cannot alter this stage. One thing to notice is that the pipeline process is always initially run with the currently saved version. If a change in the version control introduces changes to the pipeline, the pipeline execution is canceled in this stage and restarted with the new version.

4/5 Assets

In the assets stage, the pipeline analyzes the application stack and publishes all files to S3 and Docker images to ECR that the application needs for deployment. CDK Pipelines stores these assets using its buckets and ECR registries. By default, they have no lifecycle policies, so the CDK developer should ensure that the assets will not increase their AWS bill unexpectedly.

Assets building utilizes aws-s3-deployment module for the frontend application:

    // Source bundle
    const srcBundle = s3deploy.Source.asset('../../findy-wallet-pwa', {
      bundling: {
        command: [
          'sh', '-c',
          'npm ci && npm run build && ' +
          'apk add bash && ' +
          `./create-set-env.sh "./tools/env-docker/set-env.sh" "${bucketName}" "${process.env.API_SUB_DOMAIN_NAME}.${process.env.DOMAIN_NAME}" "${GRPCPortNumber}" && ` +
          'cp -R ./build/. /asset-output/'
        ],
        image: DockerImage.fromRegistry('public.ecr.aws/docker/library/node:18.12-alpine3.17'),
        environment: {
          REACT_APP_GQL_HOST: bucketName,
          REACT_APP_AUTH_HOST: bucketName,
          REACT_APP_HTTP_SCHEME: 'https',
          REACT_APP_WS_SCHEME: 'wss',
        },
      },
    });

    new s3deploy.BucketDeployment(this, `${id}-deployment`, {
      sources: [srcBundle],
      destinationBucket: bucket,
      logRetention: RetentionDays.ONE_MONTH
    });

5/5 Deploy

Finally, the Deploy stage creates and updates the application infrastructure resources. There is also a chance to add post steps to this stage, which can run the post-deployment testing and other needed scripts.

For the agency, we are using the post-deployment steps for three purposes:

1. We have a custom script for updating the ECS service. This script is in place to tweak some service parameters missing from CDK constructs.
2. We do the configuration of the agency administrator account.
3. We are running an e2e test round to ensure the deployment was successful.

Conclusions

The CDK pipeline is initially a bit complex to get your head around. For simple applications, the use is easy, and there isn’t even a need to deeply understand how it works. However, when the deployment has multiple moving parts, it is beneficial to understand the different stages.

There are still some details that I would like to see improvement in. The documentation and examples need additions, especially on how to use the assets correctly. There have been improvements already, but complete example applications would make the learning curve for the CDK pipelines more gentle. They state that CDK pipelines are “opinionated,” but users should know better what that opinion is.

However, the CDK pipeline model pleases me in many ways. I especially value the approach that has reduced the steps needed to run in the developer’s local environment compared to how we used the previous versions of AWS IaC tools. Furthermore, the strategy enables multiple developers working with the same pipeline as the state needs to be available in the cloud. Finally, I am happy with the current state of our deployment pipeline, and it works well for our purposes.

If interested, you can find all our CDK application codes in GitHub and even try to deploy the agency yourself!

Agency's IaC Journey

In the early days of Findy Agency’s development project, it became evident that we needed a continuously running environment in the cloud to try out different proofs-of-concept (PoCs) and demos quickly and easily. As our team is small, we wanted to rely heavily on automation and avoid wasting time on manual deployments.

Setup Infra without a Hassle

At that time, I was interested in improving my AWS skills, so I took the challenge and started working on a continuous PoC environment for our project. As an innovation unit, we experiment constantly and want to try different things quickly, so the PoC environment also needed to accommodate this agility.

From the start, it was clear that I wanted to use IaC (infrastructure as code) tools to define our infra. My target was to create scripts that anyone could easily take and set up the agency without a hassle.

I had been using Terraform in some of my earlier projects, but using a third-party tool typically requires compromises, so I wanted to take the native option for a change with the IaC-tooling as well. The initial target was not to find the ultimate DevOps tools but more in the experimentation spirit to find tooling and methods suitable for our flexible needs.

From Manual YAML to CDK

Back then, the only AWS-native choice for IaC-tooling was to manually start writing the CloudFormation templates. CloudFormation is an AWS service that one can use to provision the infrastructure with JSON- or YAML-formatted templates. CloudFormation stack is an entity that holds the infrastructure resources. All the resources defined in the stack template are created and removed together with the stack. One can manage the infrastructure stacks through the AWS CLI or the CloudFormation UI.

So the first iteration of the agency’s AWS infra code was about writing a lot of YAML definitions, deploying them from the local laptop using the AWS CLI, fixing errors, and retrying. The process could have been more efficient and successful in many ways. For example, there was just not enough time to figure out everything needed and, in many cases, the desired level of automation required to write countless custom scripts.

After some time of struggling with the YAML, AWS released the CDK (Cloud Development Kit). The purpose of the CDK is to allow developers to write the CloudFormation templates using familiar programming languages. CDK tooling converts the code written by the developer to CloudFormation templates.

Writing declarative definitions using imperative programming languages felt a bit off for me at first, but I decided to try it. There were some evident benefits:

• CDK offers constructs that combine CloudFormation resources with higher-level abstractions. There is less need to know the dirty details of each resource.
• Sharing, customizing, and reusing constructs is more straightforward.
• One can use her favorite IDE to write the infra code. Therefore tools like code completion are available.
• Also, there are other language-specific tools. One can apply dependency management, versioning, and even unit testing to the infra code similarly to other software projects.

The Missing Puzzle Piece

Switching to CDK tooling boosted my performance for the infra work significantly. Also, the manual hassle with the YAML templates is something I have not longed for at all. Still, it felt like something was missing. I was still running the infra setup scripts from my laptop. In my ideal world, the pipeline would create the infra, keeping things more reproducible and less error-prone. Also, defining the build pipeline and the deployment needed custom steps that made the initial agency setup still complex, which was something that I wanted to avoid in the first place.

Well, time went on, and we were happy with the deployment pipeline: regardless of the setup process, it worked as expected. However, in the spring of 2022, I saw an OP Software Academy course about the CDK. The Academy is our internal training organization that offers courses around several topics. I decided to join the class and learn more about CDK and get some validation if I had done things correctly.

In the course, I found the missing piece of my puzzle. As it happened, AWS had just released CDK v2, which introduced a new concept, CDK pipelines. CDK pipeline is yet another higher-level abstraction, this time for AWS continuous integration and deployment tools. It utilizes AWS CodePipeline to build, test and deploy the application. The CDK pipeline’s beauty lies in its setup: it is deployed only once from the developer’s desktop. After the creation, the pipeline handles the infra-deployment and subsequent changes to the deployment or the pipeline via version control.

After porting our deployment on top of the CDK pipeline, the setup has finally reached my standards. However, the future will show us how the Agency deployment will evolve. Perhaps we will introduce a more platform-agnostic approach and remove AWS native tooling altogether.

You can do a deep dive into the anatomy of our CDK pipeline in my next blog post. And as always, you can find the codes on GitHub!

No-Code SSI Chatbots - FSM Part I

In this blog post, I’ll explain the syntax of our chatbot language. The good news is that the language is simple, and we already offer some development tools like UML rendering. There’ll be a second post where we’ll dive deeply into the implementation details. Our ultimate goal is to find a proper model checker and theorem proofer for the correctness of the chatbot applications.

Our team got the idea of chatbots quite early after we started to play with verifiable credentials and SSI.

I think that chatbots and zero UI are some sorts of a lost opportunity for SSI/DID. The backbone of the DID network is its peer-to-peer communication protocol. Even though the client/server API model is very convenient to use and understand, DIDComm based apps need something different – more conversational. What would be more conversational than chatting?

Anyhow, we have been positively surprised by how far you can get without NLP but just a strict state machine-guided conversation where each party can proof facts about themselves when needed. And, of course, you can build perfect hybrid, where you fix structural parts of the discussion with the FSM and leave unstructured details for the NLP engine.

Hello World

The chatbot state machines are written in YAML (JSON accepted). Currently, a YAML file includes one state machine at a time.

As all programming books and manuals start with the hello world app, we do the same.

initial:                                             # (1)
  target: INITIAL
states:                                              # (2)
  INITIAL:                                           # (3)
    transitions:                                     # (4)
    - trigger:                                       # (5)
        protocol: basic_message                      # (6)
      sends:                                         # (7)
      - data: Hello! I'm Hello-World bot.
        protocol: basic_message                      # (8)
      target: INITIAL                                # (9)

The previous machine is as simple as it can be in that it does something. Let’s see what the lines are for:

1. Initial state transition is mandatory. It’s executed when the machine is started. It’s the same as all the state transitions in our syntax but doesn’t have a transition trigger.
2. States are listed next. There are no limits to how many states the machine holds.
3. This machine has only one state named INITIAL. Each state must have a unique name.
4. States include transitions to the next states (target). We have one in this machine, but no limit exists to how many transitions a state can have.
5. Each transition has a trigger event.
6. Triggers have protocol that is in this case basic_message.
7. We can send limitless a mount of events during the state transition.
8. In this machine, we send a basic_message where the data is Hello! I'm Hello-World bot.
9. Our transition target is the INITIAL state. It could be whatever state exists in the machine.

Did you get what the machine does? You can try it by following the instructions in Findy CLI’s readme to setup your playground/run environment. After you have set up a pairwise connection between two agents, and needed environment variables set like FCLI_CONNN_ID, execute this to the first agent’s terminal:

findy-agent-cli bot start <Hello-World.yaml> # or whatever name you saved the script above

For the second agent, use two terminals and give these commands to them:

# terminal 1
findy-agent-cli bot read # listens and shows other end's messages

# terminal 2
findy-agent-cli bot chat # sends basic_message's to another end thru the pairwise

And when you want to render your state machine in UML, give this command:

findy-agent-cli bot uml <Hello-World.yaml> # name of your FSM

The result looks like this:

The UML rendering may help with understanding. It’s also an excellent tool for manual verification. Automatic model checking is something we are studying in the future.

The FSM Language

The YAML-based state machine definition language is currently as simple as possible.

State Machine

The first level is the states, which are the primary building blocks of the machine. A machine has one or more states. During the execution, the machine can be in only one state at a time. Sub- or embedded states aren’t supported because they are only convenient, not mandatory. Also, parallel states are out-scoped.

One of the states must be marketed as initial. Every chatbot conversation runs its own state machine instance, and the current implementation of machine termination terminates all running instances of the machine. The state machine can have multiple termination states.

Note, because the final multi-tenant deployment model is still open, and we have yet to decide on the persistence model, we recommend being extra careful with the state machine termination. Even though the ending is especially convenient for the one-time scripts.

Each state can include relations to other states, including itself. These relations are state-transitions which include:

• a trigger event
• send events
• a target state

Memory

Each state machine instance has one memory register/dictionary/map. All of the memory access (read/write) are integrated state-transitions, and their rules. If we bring some scripting language onboard, the memory model integration is the first thing to solve. Also current memory model isn’t production ready for large-scale service agents because there isn’t any discarding mechanism. However, this will be fixed in the next minor release, where a transition to the initial state frees the state machine instance’s memory register. Edit: memory cleanup is implemented, and Lua is onboard.

Meta-Model

Information about the meta-model behind each state machine can be found in the following diagram. As you can see, the Machine receives and sends Events. And States controls which inputs, i.e., triggers are valid, when and how.

Next, we will see how the Event is used to run the state machine. After the next chapter, we should learn to declare all supported types of input and output events.

Event

As we previously defined, state transitions are input/output entities. Both input and output are also event-based. An input event is called trigger: and outcomes are sends:.

The event has necessary fields, which we describe next in more detail.

• rule: Defines an operation to be performed when sending an event or what should happen when inputting an event.
• protocol: Defines a protocol to be executed when sending or a protocol event that triggers a state transition.
• data: Defines additional data related to the event in a string format.

      ...
      trigger:                                             # 1
        data: stop                                         # 2
        protocol: basic_message                            # 3
        rule: INPUT_EQUAL                                  # 4

1. Simple example of the trigger event.
2. stop is a keyword in this trigger because of the rule (see # 4).
3. The keyword is received thru the basic_message Aries DIDComm protocol.
4. INPUT_EQUAL means that if incoming data equals the data: field, the event is accepted, and a state transition is triggered.

Rule

The following table includes all the accepted rules and their meaning for the event.

Protocol

The following table includes all the accepted Aries protocols and their properties.

The following table includes currently recognized general protocols and their properties. Recognized protocols aren’t yet thoroughly tested or implemented, only keywords are reserved and properties listed.

On the design table, we have ideas like REST endpoints, embedded scripting language (Lua, now implemented), file system access, etc.

Data

The data field is used to transport the event’s data. Its function is determined by both rule and protocol. Please see the next chapter, Event Data.

Event Data

The event_data field transports the event’s type-checked data. Its type is determined by both rule and protocol. Currently, it’s explicitly used only in the issue_cred protocol:

  ...
  data:
  event_data:
    issuing:
      AttrsJSON: '[{"name":"foo","value":"bar"}]'
      CredDefID: <CRED_DEF_ID>
  protocol: issue_cred

We are still work-in-progress to determine what will be the final role of data and event_data. Are we going to have them both or something else? That will be decided according to the feedback from the FSM chatbot feature.

Issuing Example

The following chatbot is an illustration of our chatbot from our Identity Hackathon 2023 repository. It’s proven extremely handy to kick these chatbots up during the demo or development without forgetting the production in the future.

Omni-Channel Chatbot

The diagram below presents another example of the automatic issuing chatbot for verifying an email address. Please read the state transition arrows carefully. They define triggers and events to send. There is a transition that sends an Aries basic_message and an email in the same transition. The email message built by the machine includes a random PIN code. As you can see, the state machine can adequately verify the PIN code.

It’s been rewarding to notice how well chatting and using verifiable credentials fit together. As an end-user, you won’t face annoying context switches, but everything happens in the same logical conversation.

Future Features

The most critical task in the future will be documentation. Hopefully, this blog post helps us to get it going.

Something we have thought about during the development:

• transition triggers are currently SSI-only which can be changed in the future
  • transient states
  • Edit: Now embedded Lua that solves limitless trigger types and make need for transient states obsolete
• straightforward memory model
  • no persistence model
• verification/simulation tools: a model checker
• simple scripting language inside the state machine, Edit: Lua is now implemented
• deployment model: cloud, end-user support
• end-user level tools

Please take it for a test drive and let us know your thoughts. Until the next time, see you!

How to Equip Your App with VC Superpowers

My previous blog post explained how to get started with SSI service agents using our CLI tool. But what about the next step, the integration into your application?

As I have previously described in my earlier blog post, Findy Agency API is the interface to Findy Agency for clients that wish to use the agency services programmatically. You can compile the gRPC API for multiple languages. However, the most straightforward is to start with the ones for which we provide some additional tooling: Go, Typescript (JavaScript), or Kotlin.

The Server Samples

The sample repository has an example server for each of the three languages. You can run the samples by cloning the repository and following the instructions in the README.

The sample servers demonstrate a similar issuing and verifying example as the CLI script in the previous post. The server has two endpoints, /issue and /verify. Both endpoints display QR codes for connection invitations. The server starts a listener who gets notified when a new connection is established (the holder has read the invitation). The notification triggers either the credential issuance or verification, depending on the endpoint that displayed the invitation.

The Server Code

I have gathered snippets to which you should pay close attention when checking the sample code. The snippets chosen here are from the Go sample, but each server contains similar functionality.

Client Registration and Authentication

The agency API clients use headless FIDO2 authentication. Before the client can make any API requests, it must authenticate and acquire a JWT token using the FIDO2 protocol. This Go example uses the authenticator functionality from the github.com/findy-network/findy-agent-auth package. Other implementations (Typescript, Kotlin) currently utilize the findy-agent-cli binary for headless authentication.

The sample code tries to authenticate first, and only if the authentication fails will it try the client registration. The registration binds the client key to the client account, and after a successful registration, the client can log in and receive the JWT token needed for the API calls.

import (
  ...

  "github.com/findy-network/findy-agent-auth/acator/authn"
)

...

// use command from the headless authenticator package
var authnCmd = authn.Cmd{
  SubCmd:   "",
  UserName: os.Getenv("FCLI_USER"),// client username
  Url:      os.Getenv("FCLI_URL"), // FIDO2 server URL
  AAGUID:   "12c85a48-4baf-47bd-b51f-f192871a1511",
  Key:      os.Getenv("FCLI_KEY"), // authenticator key
  Counter:  0,
  Token:    "",
  Origin:   os.Getenv("FCLI_ORIGIN"),
}

...

func (a *Agent) register() (err error) {
  defer err2.Handle(&err)

  myCmd := authnCmd
  myCmd.SubCmd = "register"

  try.To(myCmd.Validate())
  try.To1(myCmd.Exec(os.Stdout))
  return
}

func (a *Agent) login() (err error) {
  defer err2.Handle(&err)

  myCmd := authnCmd
  myCmd.SubCmd = "login"

  try.To(myCmd.Validate())
  r := try.To1(myCmd.Exec(os.Stdout))

  // store token for successful authentication
  a.JWT = r.Token
  return
}

With a valid JWT, the client can create the gRPC API connection and the real fun can begin!

import (
  ...

  "github.com/findy-network/findy-common-go/agency/client"
  agency "github.com/findy-network/findy-common-go/grpc/agency/v1"
  "google.golang.org/grpc"
)

  ...
  // set up API connection
  conf := client.BuildClientConnBase(
    os.Getenv("FCLI_TLS_PATH"),
    agent.AgencyHost,
    agent.AgencyPort,
    []grpc.DialOption{},
  )

  conn := client.TryAuthOpen(agent.JWT, conf)

  agent.Client = &AgencyClient{
    Conn:           conn,
    AgentClient:    agency.NewAgentServiceClient(conn),
    ProtocolClient: agency.NewProtocolServiceClient(conn),
  }

Schema and Credential Definition Creation

When the server starts for the first time, it creates a schema and a credential definition. The issuer always needs a credential definition to issue credentials. For the verifier, it is enough to know the credential definition id.

  // use the agent API to create schema and credential definition
  schemaRes := try.To1(a.Client.AgentClient.CreateSchema(
    context.TODO(),
    &agency.SchemaCreate{
      Name:       schema.Name,
      Version:    "1.0",
      Attributes: schema.Attributes,
    },
  ))
  ... 
  // tag the credential definition with our client username
  res := try.To1(a.Client.AgentClient.CreateCredDef(
    context.TODO(),
    &agency.CredDefCreate{
      SchemaID: schemaRes.ID,
      Tag:      authnCmd.UserName,
    },
  ))
  credDefID = res.GetID()

The app stores the created credential definition ID in a text file. The app will skip the credential definition creation step if this text file exists on the server bootup.

The credential definition ID is essential. You should share it with whoever needs to verify the credentials issued by your issuer.

Creating the Invitation

After the start routines, the server endpoints are ready to display the pairwise connection invitations. The holder agent can establish a secure pairwise connection with the information in the invitation.

The client uses CreateInvitation-API to generate the invitation:

  res := try.To1(ourAgent.Client.AgentClient.CreateInvitation(
    context.TODO(),
    &agency.InvitationBase{Label: ourAgent.UserName},
  ))

The endpoint returns HTML that renders the invitation as QR code.

When using a mobile device, the invitation can be read with the web wallet camera or desktop browser, copy-pasting the invitation URL to the Add connection -dialog.

Listening to Notifications

Another core concept in the client implementation is listening to the agent notifications. The client opens a gRPC stream to the server and receives notifications of the agent events through the stream.

  // pick notifications from stream channel
  ch := try.To1(agent.Client.Conn.ListenStatus(
    context.TODO(),
    &agency.ClientID{ID: agent.ClientID},
  ))

  // start go routine for the channel listener
  go func() {
    for {
      chRes, ok := <-ch
      if !ok {
        panic("Listening failed")
      }
      notification := chRes.GetNotification()
      log.Printf("Received agent notification %v\n", notification)

      ...

      switch notification.GetTypeID() {
      ...
      case agency.Notification_STATUS_UPDATE:
        switch notification.GetProtocolType() {
        case agency.Protocol_DIDEXCHANGE:
          agent.Listener.HandleNewConnection(status.GetDIDExchange().ID)
          ...
        }
        ...
      }
    }
 }()

For instance, when the server creates a pairwise connection so that the holder can connect to the issuer, the notification of the new connection is sent through the stream.

sequenceDiagram
  autonumber
    participant Server
    participant Agency
    participant Web Wallet

    Server->>Agency: Start listening
    Server-->>Web Wallet: Show QR code 
    Web Wallet->>Agency: Read QR code
    Agency->>Server: Connection created!
    Note right of Server: Conn ID for issue
    Server->>Agency: Send credential offer
    Agency->>Web Wallet: Cred offer received!
    Web Wallet->>Agency: Accept offer
    Agency->>Server: Issue ok!

In the sequence above, steps 4 and 8 are notifications sent through the listener stream.

The client can then react to the event accordingly. For example, when a new connection is established from the issue endpoint, the client sends a credential offer:

func (a *AgentListener) HandleNewConnection(id string) {
  ...
  // pairwise based on the connection id
  pw := client.NewPairwise(ourAgent.Client.Conn, id)

  ...
  // credential values
  attributes := make([]*agency.Protocol_IssuingAttributes_Attribute, 1)
  attributes[0] = &agency.Protocol_IssuingAttributes_Attribute{
   Name:  "foo",
   Value: "bar",
  }

  // send the offer
  res := try.To1(pw.IssueWithAttrs(
   context.TODO(),
   ourAgent.CredDefID,
   &agency.Protocol_IssuingAttributes{
    Attributes: attributes,
   }),
  )
  ...
}

A similar flow happens when the proof is verified. The exception with the proof is that there is an additional step where the client can reject the proof if the proof values are not valid according to the business logic (even though the proof would be cryptographically valid).

Start with Your Use Case

Now you should have the technical setup ready to start implementing your use case. But as with always in software development, we need more than technical capabilities. Before starting the implementation, try to figure out the roles and duties of your process participants:

Define the credential(s) content. Think about which data attributes you need. The sample issues the type “foobar” credentials with a single attribute “foo.” Foobar data is probably not the type you wish to issue and utilize.

Define the data flow participants. Draft your data flow and resolve which parties are the issuer, verifier, and holder or are some parties even in multiple roles. In the sample, a single service agent handles both issuing and verifying. That is not a probable scenario in a real-world use case, as the whole point of SSI and verified data use is to verify credentials that other parties have issued.

Define how you operate each verified data flow participant. The sample had a CLI tool or API client as the issuer and verifier and a web wallet user as the holder. Depending on the use case, you might have a similar setup or, for example, another service agent as the holder in a service-to-service scenario. Or you would like to integrate the holder capabilities into your end-user application instead of using the agency web wallet.

Of course, in cases where we have multiple real-world parties involved, the detailed design is done together with the other participants.

Feedback

Let us know if you have any feedback regarding the Findy Agency functionality or documentation. It would also be cool to hear about the PoCs, demos and applications you have built using the agency.

You can also reach us via SoMe channels:

Laura

Harri

Getting Started with SSI Service Agent Development

In the new SSI world, we craft digital services according to the self-sovereign identity model. We will have applications that issue credentials for the data they possess and applications that can verify these credentials. The central entity is the digital wallet owner that can hold these credentials and present them when needed.

“Ok, sounds great!” you may think. “I want to utilize credentials also in my web application. But where to start?”

Developing decentralized applications is tricky as it usually requires setting up multiple applications on your local computer or acquiring access to services set up by others. Using Findy Agency tackles this hurdle. It is an SSI solution that offers a complete set of tools for managing your digital wallet and agent via a user interface or an API.

The Agency tooling provides you with a web wallet and a CLI tool that you can use to test your service’s issuing and verifying features. You can easily setup the whole Findy Agency software to your local computer using Docker containers and a simulated ledger. Or, if you have an agency cloud installation available, you can utilize it for your service agent development without using any extra proxies or network tools.

“So, I have the agency up and running. What next?”

TL;DR Take a look at the examples found in the sample repository!

The sample repository provides simple yet comprehensive examples to start issuing and verifying using the CLI tool or with the agency API. The easiest path is to start with the CLI.

Run the CLI

“findy-agent-cli” is a command-line tool that provides all the required agent manipulation functionality. It provides means to quickly test out the issuing and verifying before writing any code.

The sample script is a good starting point. It shows how to allocate an agent in the cloud and issue and verify credentials using a simple chatbot. You can run it by cloning the repository and following the instructions in the README.

CLI Script Initialization Phase

The sample script initialization phase allocates a new agent from the agency (1) and authenticates the CLI user (2-3). The authentication returns a JWT token exposed to the script environment so that further CLI calls can utilize it automatically.

For the agent to issue credentials, an applicable schema needs to exist. The schema describes the contents of a credential, i.e., which attributes the credential contains. The sample script creates a schema “foobar” with a single attribute “foo” (4-5).

There needs to be more than the mere schema for the issuing process; the agent needs also to create and publish its credential definition (6-7) attached to the created schema so that it can issue credentials and verifiers can verify the proof presentations against the published credential definition.

We assume that the holder operates a web wallet and has taken it into use. Note that you can play the holder role also with the CLI tool.

sequenceDiagram
  autonumber
    participant CLI
    participant Agency
    participant Web Wallet

    CLI->>Agency: Register
    CLI->>Agency: Login
    Agency-->>CLI: JWT token
    CLI->>Agency: Create schema
    Agency-->>CLI: Schema ID
    CLI->>Agency: Create cred def
    Agency-->>CLI: Cred def ID

CLI Script Issuing Credential

The next task is to create a pairwise connection between the agent operated by the CLI user and the web wallet user. The pairwise connection is an encrypted pipe between the two agents that they can use to exchange data securely. The CLI script creates an invitation (1-2) and prints it out (3) as a QR code that the web wallet user can read (5).

Once the new connection ID is known, the CLI script starts a chatbot (4) for the new connection. The bot logic follows the rules for changing the bot states in the YAML file. Therefore, the bot handles the rest of the issuing process (6-7).

Once the issuer bot notices that credential issuing succeeded, it stops the bot (10-11), and the sample script moves on to verifying the same credential.

sequenceDiagram
  autonumber
    participant CLI
    participant Issue Bot
    participant Agency
    participant Web Wallet

    CLI->>Agency: Create invitation
    Agency-->>CLI: Invitation URL
    CLI-->>Web Wallet: <<show QR code>
    CLI->>Issue Bot: Start
    Web Wallet->>Agency: Read QR code
    Agency-->>Issue Bot: Connection ready!
    Issue Bot->>Agency: Issue credential
    Agency-->>Web Wallet: Accept offer?
    Web Wallet->>Agency: Accept
    Agency-->>Issue Bot: Issue ready!
    Issue Bot->>Issue Bot: Terminate

CLI Script Verifying Credential

Steps 1-6 proceed similarly to the issuing: first, the agents form a new pairwise connection. However, the process continues with a proof request sent by the verifier bot (7). The proof request contains the attributes the bot wishes the holder to present. The web wallet user sees the requested data once they receive the message (8), and they can either accept or reject the request.

After the proof is accepted (9), the agency verifies it cryptographically. If the verification succeeds, the agency notifies the verifier bot with the proof values (10). It can reject the proof if the values are not acceptable by the business logic. The sample bot accepts all attribute values, so the verifying process is continued without extra validation (11). The bot exits when the proof is completed (12-13).

sequenceDiagram
  autonumber
    participant CLI
    participant Verify Bot
    participant Agency
    participant Web Wallet

    CLI->>Agency: Create invitation
    Agency-->>CLI: Invitation URL
    CLI-->>Web Wallet: <<show QR code>
    CLI->>Verify Bot: Start
    Web Wallet->>Agency: Read QR code
    Agency-->>Verify Bot: Connection ready!
    Verify Bot->>Agency: Proof request
    Agency-->>Web Wallet: Accept request?
    Web Wallet->>Agency: Accept
    Agency-->>Verify Bot: Proof paused
    Verify Bot->>Agency: Resume proof
    Agency-->>Verify Bot: Proof ready!
    Verify Bot->>Verify Bot: Terminate

CLI as a Test Tool

Note that you can also utilize the CLI for testing. It is an excellent tool to simulate the functionality on the other end.

For instance, let’s say you are developing an issuer service. You can use the CLI tool to act as the holder client and to receive the credential. Or you can use the web wallet to hold the credential and create another client with the CLI tool to verify the issued data.

Feedback

The CLI sample script presented above demonstrates all the core features of verified data flow. It should make you well-equipped to play around with the CLI tool yourself!

Your tool pack will extend even more with our next blog posts. They will describe how to use the agency API programmatically and dive deep into crafting verified data supporting chatbot state machines.

Let us know if you have any feedback regarding the Findy Agency functionality or documentation. You can reach us, for example creating an issue or starting a discussion in GitHub.

You can also reach us via SoMe channels:

Laura

Harri

Good luck on your journey into the SSI world!

Ledger Multiplexer

In this technical blog post, I’ll explain how I implemented a plugin system into our Indy SDK Go wrapper and then extended it to work as a multiplexer. The plugin system allows us to use a key/value-based storage system instead of the normal Indy ledger. And multiplexing extended the functionality to use multiple data sources simultaneously and asynchronously. For instance, we can add a memory cache to help Indy ledger, which has been proving to make a considerable difference in a multi-tenant agency that can serve thousands of different wallet users simultaneously.

As you can see in the picture above plugin package defines just needed types and interfaces that the addons package’s plugins implement. The FINDY_LEDGER plugin routes transactions to the authentic Indy ledger. We will show in this post how it’s implemented and used.

Reduce Setup Complexity

Indy ledger has been bugging me since we met each other:

• Why was it written in Python when everybody had to know that it would be a performance-critical part of the system?
• Why it was implemented with ZeroMQ when it used so simple TCP/IP communication? (Was it because of Python?) Anyway, every dependency is a dependency to handle.
• Why didn’t it offer a development run mode from the beginning? For instance, a pseudo node would offer local development and test environment?
• Why it didn’t offer a straightforward own separated API? With Indy SDK, you had to build each transaction with three separate functions that weren’t general but entity-specific, like indy_build_cred_def_request().
• Why the transaction model was so unclear and ‘hid’ from the rest of the Indy SDK functions? (See previous one.)
• Why could Indy nodes not idle? When no one was connected to the ledger, it still used a shit load of CPU time per node, and there was a four (4) node minimum in the local setup.

For the record, first, it’s too easy to write the previous list when someone has done everything ready; second, I appreciate all the work Hyperledger Indy has done.

So why bother nagging? To learn and share that learning. If our team or I will ever build anything where we would use a distributed ledger from scratch, we would follow these guidelines:

• Give a straightforward and separated API for ledger access.
• Implement some out-of-the-box mocking for development, e.g., use a memory database.
• Offer a way to set up a production system without an actual consensus protocol and DLT but offer a single database and tools to migrate that to real DLT when the time comes.

Support TDD (Test-Driven Development)

I have noticed that we programmers far too easily avoid automating or documenting our work. Luckily, Go as a language selection supports that very well. Fortunately, our two-person team focused on how we should rely on code-centric automated testing in everything from the beginning.

I wanted to support both unit and integration tests without the mock framework’s help and the need to set up a complex environment just for simple testing. My choice was to have a memory ledger. That would also help with instrumentation and debugging of our code.

Soon I noticed that the memory ledger was insufficient to support fast phase development, but we would need some persistence. A JSON file, aka file ledger, seemed an excellent way to start. The JSON would support tracing and offer another interface for us humans.

When putting these two together, I ended up building our first plugin system for VDR.

We were so happy without the bloated ledger that we all started to think about how we could remove the whole ledger out of the picture permanently, but that’s it is own story to tell.

Reverse-engineering Indy SDK

Before I could safely relay that my solution won’t hit my face later, I had to check what Indy functions don’t separate wallet and ledger access, i.e., they take both wallet handle and ledger connection handle as their argument. I found two that kinds of functions that we were using at that time:

1. indy_key_for_did()
2. indy_get_endpoint_for_did()

Both functions check if they can find information from the wallet, and the ledger is the backup. For those listening to our presentation on Hyperledger Global Forum, I mistakenly said that I used -1 for the wallet handle, which is incorrect. Sorry about that. (1-0, once again, for documentation.)

I discovered that I could enumerate our ledger connection handles starting from -1 and going down like -1, -2, and so forth. So I didn’t need any extra maps to convert connection handles, which would add complexity and affect performance. I could give connection handles with negative values to the above functions, and libindy accepted that.

Here you can see what the first function (indy_key_for_did()) looks like in our wrapper’s API. And I can assure you that c2go.KeyForDid internal implementation wrapper for Go’s CGO, which is a C-API bridge, doesn’t treat pool and wallet handles differently before passing them to Indy SDK’s Rust code. The pool handle can be -1, -2, etc.

// Key returns DIDs key from wallet. Indy's internal version tries to get the
// key from the ledger if it cannot get from wallet. NOTE! Because we have our
// Ledger Plugin system at the top of the wrappers we cannot guarantee that
// ledger fetch will work. Make sure that the key is stored to the wallet.
func Key(pool, wallet int, didName string) ctx.Channel {
	return c2go.KeyForDid(pool, wallet, didName)
}

Some versions of libindy worked so well that if the connection handle wasn’t valid, it didn’t crash but just returned that it could not fetch the key. Of course, that helped my job.

The Plugin Interface

I started with a straightforward key/value interface first. But when we decided to promote Indy ledger to one of the plugins, which it wasn’t before multiplexing, we brought transaction information to a still simple interface. It has only Write and Read functions.

// Mapper is an property getter/setter interface for addon ledger
// implementations.
type Mapper interface {
	Write(tx TxInfo, ID, data string) error

	// Read follows ErrNotExist semantics
	Read(tx TxInfo, ID string) (string, string, error)
}

Naturally, the plugin system has an interface Plugin, but it’s even more straightforward, and it does not interest us now, but you see it in the UML picture above.

The following code block shows how transaction information is used to keep the public interface simple and generic.

func (ao *Indy) Write(tx plugin.TxInfo, ID, data string) error {
	switch tx.TxType {
	case plugin.TxTypeDID:
		return ao.writeDID(tx, ID, data)

	case plugin.TxTypeSchema:
		return ao.writeSchema(tx, ID, data)

	case plugin.TxTypeCredDef:
		return ao.writeCredDef(tx, ID, data)

	}

	return nil
}

The following code block is an example of how the Indy ledger plugin implements schema writing transaction with libindy:

func (ao *Indy) writeSchema(
	tx plugin.TxInfo,
	ID string,
	data string,
) (err error) {
	defer err2.Return(&err)

	glog.V(1).Infoln("submitter:", tx.SubmitterDID)

	r := <-ledger.BuildSchemaRequest(tx.SubmitterDID, data)
	try.To(r.Err())

	srq := r.Str1()
	r = <-ledger.SignAndSubmitRequest(ao.handle, tx.Wallet, tx.SubmitterDID, srq)
	try.To(r.Err())

	try.To(checkWriteResponse(r.Str1()))
	return nil
}

The readFrom2 is the heart of the cache system of our multiplexer. As you can see, it’s not a fully dynamic multiplexer with n data sources. It’s made for just two, which is enough for all our use cases. It also depends on the actuality that the Indy ledger plugin is the first, and the cache plugin is the next. Please note that cache can still be whatever type of the plugins, even immuDB.

Thanks to Go’s powerful channel system, goroutines, and an essential control structure for concurrent programming with channels, the select -statement, the algorithm is quite simple, short, and elegant. Faster wins the reading contest, and if the Indy ledger wins, we can assume that the queried data is only in the Indy ledger. Like the case where other DID agents use the same Indy ledger, we are using DLT for interoperability.

And yes, you noticed, we think that the ledger is always the slower one, and if it’s not, it doesn’t matter that we tried to write it to cache a second time. No errors and no one waits for us because writing is async.

func readFrom2(tx plugin.TxInfo, ID string) (id string, val string, err error) {
	defer err2.Annotate("reading cached ledger", &err)

	const (
		indyLedger  = -1
		cacheLedger = -2
	)
	var (
		result    string
		readCount int
	)

	ch1 := asyncRead(indyLedger, tx, ID)
	ch2 := asyncRead(cacheLedger, tx, ID)

loop:
	for {
		select {
		case r1 := <-ch1:
			exist := !try.Is(r1.err, plugin.ErrNotExist)

			readCount++
			glog.V(5).Infof("---- %d. winner -1 (exist=%v) ----",
				readCount, exist)
			result = r1.result

			// Currently first plugin is the Indy ledger, if we are
			// here, we must write data to cache ledger
			if readCount >= 2 && exist {
				glog.V(5).Infoln("--- update cache plugin:", r1.id, r1.result)
				tmpTx := tx
				tx.Update = true
				err := openPlugins[cacheLedger].Write(tmpTx, ID, r1.result)
				if err != nil {
					glog.Errorln("error cache update", err)
				}
			}
			break loop

		case r2 := <-ch2:
			notExist := try.Is(r2.err, plugin.ErrNotExist)

			readCount++
			glog.V(5).Infof("---- %d. winner -2 (notExist=%v, result=%s) ----",
				readCount, notExist, r2.result)
			result = r2.result

			if notExist {
				glog.V(5).Infoln("--- NO CACHE HIT:", ID, readCount)
				continue loop
			}
			break loop
		}
	}
	return ID, result, nil
}

I hope you at least tried to read the previous function even when you aren’t familiar with Go because that might be the needed trigger point to start to understand why Go is such a powerful tool for distributed programming. I, who have written these kinds of algorithms with almost everything which existed before Go, even to OS, which didn’t have threads but only interrupts, has been so happy. Those who are interested in computer science, please read them Hoare’s paper of CSP. Of course, Go inventors aren’t the only ones using the paper since 1978.

Putting All Together

At the beginning of the document, I had a relatively long list of what’s not so good in Indy SDK, and that was even all. I tried to leave out those things caused because C-API has put together quite fast, I think. The libindy is written in Rust.

But still, the C-API has hints of namespacing, and luckily I have been following the namespace structure in our Go wrapper’s package structure by the book. So, we have these Go packages:

• did
• wallet
• pairwise
• crypto
• ledger
• pool, this was very important, because it gave as the entry point to above layer. The layer was using our Go wrapper.

The following picture illustrates the whole system where the ledger connection pool is replaced with our own pool package.

The code using our Go wrapper looks like the same as it has been since the beginning.

// open real Indy ledger named "iop" and also use Memory Cache
r = <-pool.OpenLedger("FINDY_LEDGER,iop,FINDY_MEM_LEDGER,cache")
try.To(r.Err())

try.To(ledger.WriteSchema(pool, w1, stewardDID, scJSON))

sid, scJSON = try.To2(ledger.ReadSchema(pool, stewardDID, sid))

All of this just by following sound software engineering practices like:

• abstraction hiding
• polymorphism
• modular structures

I hope this was helpful. Until the next time, see you!

The Hyperledger Global Forum Experience

Our team started utilizing Hyperledger technologies in 2019 when we began the experimentation with decentralized identity. Since then, we have implemented our identity agency with the help of two Hyperledger Foundation projects: Hyperledger Indy for implementing the low-level credential handling and Hyperledger Aries for the agent communication protocols specification.

We released our work as open-source in 2021. When the pandemic restrictions finally eased in 2022, we thought the Hyperledger Forum would be an excellent chance to meet fellow agent authors and present our work to an interested audience. Luckily, the program committee chose our presentation among many candidates, so it was time to head for Ireland.

The three-day event consisted of two days of presentations and demos and one workshop day where the participants had a chance to get their hands dirty with the actual technologies. When over 600 participants and 100 speakers gather together, there is also an excellent chance to meet old friends and make new ones.

Presenting Findy Agency

We had the opportunity to present our project’s achievements on the second day of the conference, Tuesday, 13th September. Our overall Hyperleger-timing was perfect because the other SSI agent builders had comforted the issues we had already solved. And for that reason, for example, our ledger-less running mode got lots of attention.

We had a little luck that the feature stayed in the slides. Previous conversations with Hyperledger Aries’s core team had not raised any interest in the subject. Now they had an idea that AnonCreds (Hyperledger Indy verified credentials) could stay in version 1.0 format, but VDR (verified data registry, ledger in most of the DID methods) could be something else in Indy SDK.

Our ledger multiplexer solves the same issue. It makes it possible to use Indy SDK with whatever VDR, or even multiple VDRs if necessary, for example, the second one as a cache.

Summary of the rest of the QA section of the talk:

• There was a question about the business cases we have thought about or are keeping potential. Everybody seemed eager to find good use cases that could be referenced for their own use. Many sources confirmed that we should first solve something internal. Something that we can decide everything by ourselves. As a large enterprise, that would still give us a tremendous competitive advantage.
• Questions about how our solution differs from systems based on DIDComm routing, i.e., you must have multiple agents like a relay, a mediator (cloud agent), and an edge agent. We explained that you wouldn’t lose anything but get a much simpler system. There were questions about wallet export and import, which both we have implemented and tested.
• There were multiple reasonable questions about where DIDComm starts in our solution and where it’s used. Because our wallet app doesn’t use (anymore) DIDComm, e.g., for edge agent onboarding, it wasn’t easy to get your head around the idea that we integrated all the agent types into one identity domain agent (the term we used in the talk). Our web wallet was a UI extension of it.
• Luckily, we had some extra time because there was a break after our talk. We could answer all the questions. We can say that we had an excellent discussion at the end of the presentation.

The organizer recorded our talk and it is published on YouTube.

As shown in the Findy Agency demo at #HyperledgerForum, using FIDO2 to authenticate to a web wallet is a great way to keep access to an online web wallet passwordless. pic.twitter.com/GGuHnecnI7

— Animo (@AnimoSolutions) September 13, 2022

The Technical Demos

In addition to keeping our presentation, our goal was to participate in the sessions handling self-sovereign identity and verifiable data. There would have been exciting sessions covering, e.g., the Hyperledger Fabric project. However, unfortunately, many of the sessions were overlapping, so one needed to cherry-pick the most interesting ones, naturally keeping our focus on the technical track.

Aries Bifold project is a relatively new acquaintance in the Aries community. It is a long-awaited open-source mobile agent application built on react-native. The community was long missing an open-source solution for edge agents running on mobile devices, and they had to use proprietary solutions for this purpose. Aries Bifold tries to fill this gap and provide an easily customizable, production-ready wallet application.

In the Aries Bifold demo, we saw two wallet applications receiving and proving credentials, one being the basic version of the app and another with customization, the BC Wallet application. The BC Wallet is a digital wallet application developed by the Government of British Columbia in Canada and is even publicly available in application stores.

Another open-source demo was about building controllers for ACAPy. ACAPy intends to provide services for applications (controllers) that aim to utilize Aries credential flows in their logic. In the demo, we saw how the ACAPy controllers could handle their agent hosted by ACAPy using the ACAPy REST API. However, this demo was a bit disappointing as it did not show us anything we hadn’t seen before.

”Decentralized Identity Is Ready for Adoption Now”

One of the most exciting keynotes was a panel talk with four SSI legends, Drummond Reed, Heather Dahl, Mary Wallace, and Kaliya Young. Their clear message was that we are now done with the mere planning phase and should boldly move to execution and implement real-world use cases with the technology. Also, the panelists raised the public sector to the conversation. The point was that the public sector should act as an example and be the pioneer in using the technology.

Some public sector organizations have already listened to this message as we heard about exciting pilot projects happening in the Americas. The state of North Dakota is issuing a verifiable credential to each graduating senior, and potential employers and universities can then verify these credentials. The city of San Francisco uses verified credentials to replace legacy authentication mechanisms in over 100 public sites. The change effectively means that users must remember 100 fewer passwords, significantly improving the user experience. Furthermore, the Aruba Health Pass allows Aruba travelers to enter the island using the digital health pass. Hyperledger Indy and Aries technologies empower all of the abovementioned cases.

Workshopping

The last day of the conference was about workshopping. We joined the workshop intended for building SSI agents with the Aries JavaScript framework. The framework approach for building the agent functionality differs from the approach we use with our agency: the framework makes building an agent more effortless compared to starting from scratch, but one still needs to host the agent themselves. In the agency way, agency software runs and hosts the agent even though the client application would be offline.

The workshop’s purpose was to set up a web service for issuing and verifying credentials and a mobile application for storing and proving the credentials. Both projects, the Node.js and the React Native applications, used the Aries JavaScript Framework to implement the agent functionality under the hood. The organizers provided templates – all the participants needed to do was fill out the missing parts.

The organizers arranged the workshop quite nicely, and after we solved the first hiccups related to the building with Docker in ARM architecture, it was pretty easy to get going with the development – at least for experienced SSI developers like us. The session showed again how important these hands-on events are, as there was a lot of valuable technical discussion going on the whole day.

The Most Important Takeaways

The technology is not mature, but we should still go full speed ahead.

Kaliya Young’s blog post listing the challenges in the Hyperledger Indy and Aries technologies shook the Hyperledger SSI community just a week before the conference started. The article was a needed wake-up call to make the community start discussing that much work is still required before the technology is ready for the masses.

It is essential to take into account these deficiencies of the technology. Still, it shouldn’t stop us from concentrating on the most complicated problems: figuring out the best user experience and how to construct the integrations to the legacy systems. To get started with the pilot projects, we should take conscious technical debt regarding the underlying credential technology and rely on the expert community to figure it out eventually. This approach is possible when we use products such as Findy Agency that hide the underlying technical details from client applications.

Creating a first-class SSI developer experience is challenging.

There are many participants in a credential utilizing application flow. Quite often end-user is using some digital wallet on her mobile device to receive and prove credentials. There might be running a dedicated (web) application both for issuing and verifying. And, of course, a shared ledger is required where the different participants can find the needed public information for issuing and verifying credentials.

Sounds like a complex environment to set up on one’s local computer? Making these three applications talk with each other and access the shared ledger might be an overwhelming task for a newcomer, not to mention grasping the theory behind SSI. However, getting developers quickly onboard is essential when more and more developers start working with SSI. We think we have successfully reduced the complexity level in the Findy Agency project, which positively impacts the developer experience.

There’s no substitute for live events and meeting people f2f.

The event proved that live interaction between people is often more efficient and creative than many hours of online working and countless remote meetings. Meeting people in informal discussions is sometimes needed for new ideas to be born. Also, the change of knowledge is so much more efficient. For example, we have had the ledgerless run mode in our agency for several years, but only now did our fellow agent developers realize it, and they may utilize this finding as well.

The Findy Agency API

The Findy Agency clients can control their agents through the Findy Agency API over the high-performing gRPC protocol. The API design results from iterative planning and multiple development cycles. Initially, we implemented it using a custom protocol that utilized JSON structures and DIDComm as the transport mechanism. The initial design seemed like a good plan, as we got the chance to test the exact DIDComm implementation we were using to send the agent-2-agent messages (used in the credential exchange flows).

However, as we gained more experience using the API and the agent-2-agent protocol evolved as the community introduced Hyperledger Aries, we realized our API was too laborious, clumsy, and inefficient for modern applications.

One option would have been switching our API implementation to traditional REST over HTTP1.1. However, we wanted something better. We were tired of JSON parsing and wanted a better-performing solution than REST. We also wanted to be able to use the API using multiple different languages with ease. The obvious choice was gRPC, which provided us with protocol buffer messaging format, HTTP2-protocol performance gains, and tooling suitable for the polyglot, i.e., multilingual environment.

The technology choice was even better than we expected. Given that we have an agency installation available in the cloud, we can listen to agent events through the gRPC stream without setting up external endpoints using tools such as ngrok. Thus, gRPC streaming capabilities have considerably simplified client application development in a localhost environment. Also, the logic of how gRPC handles errors out-of-the-box have helped us trace the development time problems efficiently.

The API Contract

Our API contract defined with proto files resides in a separate repository, findy-agent-api. Using the gRPC tooling, one can take these proto files and compile them to several target languages, which enables the use of API structures directly from the target implementation.

A brief example is displayed below. We define the structures and an RPC call for schema creation in protobuf language:

// SchemaCreate is structure for schema creation.
message SchemaCreate {
  string name = 1; // name is the name of the schema.
  string version = 2; // version is the schema version.
  repeated string attributes = 3; // attributes is JSON array string.
}

// Schema is structure to transport schema ID.
message Schema {
  string ID = 1; // ID is a schema ID.
}

/*
AgentService is to communicate with your cloud agent. With the cloud agent
you can Listen your agent's status, create invitations, manage its running environment,
and create schemas and credential definitions.
 */
service AgentService {
  ...

  // CreateSchema creates a new schema and writes it to ledger.
  rpc CreateSchema(SchemaCreate) returns (Schema) {}

  ...
}

The protobuf code is compiled to target languages using gRPC tooling, and after that the structures can be used natively from the application code.

Go example:

    agent := agency.NewAgentServiceClient(conn)
    r := try.To1(agent.CreateSchema(ctx, &agency.SchemaCreate{
        Name:       name,
        Version:    version,
        Attributes: attrs,
    }))
    fmt.Println(r.ID) // plain output for pipes

Typescript example:

    log.info(`Creating schema ${JSON.stringify(body)}`);

    const msg = new agencyv1.SchemaCreate();
    msg.setName(body.name);
    msg.setVersion(body.version);
    body.attrs.map((item) => msg.addAttributes(item));

    const res = await agentClient.createSchema(msg);

    const schemaId = res.getId();
    log.info(`Schema created with id ${schemaId}`);