Delivering a high quality user experience is our pinnacle goal in the design of Meshery's CLI: mesheryctl. Delivering a high quality user experience means testing both qualitatively and quantatively. As a concept, quality - whether you're talking about software or anything else - can be directly measured by consistency. A quality user experience is consistent and consistency of mesheryctl is reinforced by its many unit tests.

mesheryctl is written in Golang. In the Meshery project, we use CodeCov to calculate the code coverage across mesheryctl's line of code. And there are plenty of lines of code. Achieving high percentages of code coverage in Meshery's CLI with grows in criticality as we bring more features into mesheryctl as a robust and sophisticated command line client of Meshery. Unit and integration tests bolster and roll up into broader end-to-end, functional testing performed across Meshery and the rest of its components.

High levels of code coverage makes it easier for each project contributor to be confident that their code changes aren’t breaking any preexisting mesheryctl commands, side-swiping their functionality unknowingly, and consequently, unwittingly lowering the quality of Meshery's overall user experience.

mesheryctl lines of code

1270 text files.
2 106 unique files.
3 166 files ignored.
5 cloc v 1.92 T=0.15 s (692.3 files/s, 110063.3 lines/s)
6 ------------------------------------------------------------------
7 Language files blank comment code
8 ------------------------------------------------------------------
9 Go 88 2128 1330 13045
10 YAML 12 0 0 215
11 JSON 4 0 0 42
12 make 1 10 2 31
13 Markdown 1 23 0 27
14 ------------------------------------------------------------------
15 SUM: 106 2161 1332 13360
16 ------------------------------------------------------------------

Unit Tests can be written in two ways:

  1. Test mesheryctl subcommand
    1. Mock Meshery(backend) response if needed
    2. Grab console output
    3. Store standard/verified mesheryctl output in a golden file(a text file)
    4. Match the stored/expected output with what we grab from the console
    5. Cover as many scenarios as possible, test the situations where errors must be thrown
    6. This is a standard format, changes can be made accordingly
  2. Test mesheryctl functions
    1. This is the standard testing you may have come across in every project
    2. You write one test dedicated to one function covering all possible test-cases, matching expected outputs

Example tree graph

This example of a tree graph (from meshery/meshery/pull/4823) shows the impact given changes make on the level of code coverage.

Integration Tests

Integrations tests come into view when you cannot mock something easily and when a given behavior is cross-functional / cross-component. Integration tests put more, but not all of the focus validating system behavior and that the system completes all the necessary actions. Take mesheryctl system start for example. This command deploys Meshery, its adapters, and Kubernetes Operator; it starts Meshery. So, you run the command to start Meshery in a GitHub workflow and after running the mesheryctl subcommand through the tests and make sure that Meshery actually started. And if the workflow successfully runs the test, then boom!, you aced your integration writing test... errr... test writing.

The integration test writing exam can initially be challenging, but... that’s the fun in it. :) Running tests and making sure a command that is otherwise hard to verify, hard to test, and making it automatic through GH workflows.

How you can make an impact

Writing tests and making them work is in itself a tough task to do, so writing one test for a single function counts and makes a big difference over the long term. Don't hesitate. Start writing tests now. A single, new test is worthy of raising a pull request and will get you well on your way to learning much more about Golang, Docker, Kubernetes, and service meshes.

  • Codecov is used to check code coverage in mesheryctl (login with GitHub to get an in-depth idea of lines-of-code being covered in code coverage).
  • You can check the files on CodeCov’s website to figure out which mesheryctl commands haven’t been covered by existing tests.
  • Guide: Writing tests for mesheryctl. This guide will briefly introduce how tests are supposed to be written for mesheryctl.
  • Contributing to Tests doesn’t always mean writing new tests, It also means improving the effectiveness of already written-ones.
  • Search for Issues marked with area/tests in GitHub - meshery/meshery: Meshery, the cloud native management plane.

The impact that you make with by writing even a single test is quite high. Your tests will be executed and used time-and-time again. I can’t explain in words (I don’t have that good of a vocabulary). Your Tests are going to run in each pull request raised against the Meshery project, in each commit someone pushes to a pull request, in every nightly regression test suite, in every release. Just think: if another contributor incidently introduces a defect or doesn't consider for an edge case, and fails to uphold the intended behavior of the code, your tests are going to catch them redhanded. How cool is that?

So, what are you waiting for? Jump in and write a few tests today! When you raise your pull request, label it with area/tests and component/mesheryctl and I'll be there to review! Let’s get mesheryctl 100% code coverage badge.

Related Blogs

Layer5, the cloud native management company

An empowerer of engineers, Layer5 helps you extract more value from your infrastructure. Creator and maintainer of cloud native standards. Maker of Meshery, the cloud native manager.