Starting SMI Conformance Testing with Meshery

About SMI

The Service Mesh Interface (SMI) is a specification for service meshes that run on Kubernetes. It utilizes CRDs to modify the behavior of service meshes. The project is under development at SMI Github Repository. Please visit the github repository page to know the latest advancements in the characteristics. The complete spec can be found here.

APIs in SMI

Currently SMI supports 4 set of APIs:

• Traffic Specs - Used to define traffic, currently only supports TCP, HTTP traffic.
• Traffic Access Control - Used to specify whether a particular form of traffic is allowed or not
• Traffic Split - Used to redirect/divide a request for a resource between 2 or more resources. Useful in canary testing
• Traffic Metrics - Used to expose common traffic metrics like p99 in a specific format that can be utilized by single dashboard for all the service meshes.

About SMI Conformance Tests

SMI Conformance Tests check whether the service mesh that is installed in the kubernetes cluster, conforms to SMI specs or not. This involves asking some major questions. Does it have the required CRDs? Do these CRDs perform as they should when applied or not applied? Are we able to get metrics in a proper format?

All these questions will be answered by SMI conformance tests. The biggest benefit of your service mesh conforming to SMI is that it makes building tools an easier process.Also, one time development of any tool would in turn support all the meshes that conform to SMI.

Sounds fun, right? Let's dig deeper into the SMI conformance project and find out.

Meshery'ing with Conformance Tests

Meshery is _the cloud native management plane__. It supports all the popular meshes, teaches you how to manage them, assists you in applying custom or recommended configurations, tests for compatibility, performs performance tests for meshes and a lot more. The SMI conformance testing requires performance testing capabilities, load generation (Meshery is about to support distributed load generation as well), and other functionalities such that these conformance tests can be easily used in the pipelines of all the popular service meshes. mesheryctl does have a perf command that can be used in the pipelines of service meshes. I aim at making such capabilities for SMI conformance as well.

As you have made your way halfway through the post (thank you for your patience), you should now be aware of SMI, its conformance practices and how Meshery’s incredible engineering can be utilized for conformance tests. We can now tackle the larger questions and hope to see the bigger picture.

The Bigger Picture

Do you know that almost all of the test cases that we would write in this project would be raw YAML files? To those doing traditional unit and integration tests, we might sound unhinged at this point. We assure you that we are completely sober and serious (if you don’t count the temporary euphoria from geeky jokes).

Forking kuttl

kuttl is a tool for writing tests against Kubernetes operators and controllers. It can ascertain whether any kind of resources exist or not in the Kubernetes cluster, spring up a kind cluster and do other convenient things. Another plus is that It's entirely declarative. Consider a scenario in which we have a special use case where we wanted to run some go code after each individual step in the test case was executed. To accomplish this, we forked kuttl and modified it a little. You can see our modified version here.

We are planning to use kuttl for all the APIs in SMI. We are also planning to use the Meshery load generator with the modified version of kuttl.

Learn Layer5

Learn Layer5 is a sample app which is very lightweight and simple to deploy. It is a simple sample app for learning about service meshes. We are planning to use the same app for testing for SMI conformance. The app is deployed when we test for SMI conformance and traffic is generated from one service/deployment/pod to another. This is a great tool for playing around with service meshes and the tools provided by Layer5 and its projects without investing any personal resources.

This is where all the test cases along with the code will be placed. Currently, there is a large overlap in the learn-layer5 and smi-conformance testing files. All those changes will be transferred here.

If any of this sounds remotely exciting, I implore you to give this a chance. You won’t regret it. Head over to our Slack Channel and join the #smi channel where everything related to conformance testing is discussed. We would love to hear your feedback. Stay tuned for more blogs related to SMI Conformance and all things meshy!!!