For SIP applications to communicate efficiently and reliable we use gRPC, which allows us to describe interfaces using Protocol Buffers. To organize, distribute and control the protocol buffers we created SerVIS. Introducing a central authority for all interfaces descriptions gives us some nice properties. With SerVIS we can allow organization specific implementations of the same interface (see AMIV Messaging, VIS Messaging that are used by the organization specific instances of the same application). SerVIS also enforces policies on formatting, naming and prevents breaking api changes, so consumers get reliable interfaces to work with. A similar concept was initially developed by the association of computer science students (VIS) — therefore the name.

How to use Servis

Explore services

All published services can be explored directly in the serVIS Registry.

Setting up the CLI

To interact with SerVIS (i.e. use a servis in your application) we provide servisctl CLI, which can be downloaded from tools.vseth.ethz.ch. *Currently servisctl is also available under the name servis, this is a legacy version that reminds the user to switch to servisctl.

How to consume an existing service

You can use the servisctl CLI to consume and provide interfaces. Run servisctl --help for documentation on the command line interface.

In this example, the VIS likes to create the Tassenpranger application. The Tassenpranger provides its own service and consumes the vseth.sip.people service to get information about students.

servis:
  provides:
    - name: vseth.vis.tassenpranger
      ingress: default
      webingress: web
      ssl: true
  consumes:
    - name: vseth.sip.people

To load all the services, you can simply run servisctl pull inside your project. Not that this will automatically be done by CI when deploying your application.

How to create a new service

To create a new service, write us an email to vseth-it-support@ethz.ch and propose a name for your servis (e.g. vseth.vis.tassenpranger). For an initial proof of concept, you can also just create your tassenpranger.proto and add it to under servis/vseth/vis/tassenpranger/tassenpranger.proto. We add all the services we consume to your sip.yml.

If you do not have a protofile yet, you can simply run `servisctl init your.service.name`, it will create a protofile with the following header (which must be present in every service). 

syntax = "proto3";

package vseth.sip.people;

option go_package = "servis/vseth/sip/people";
option java_multiple_files = true;
option java_package = "ch.vseth.sip.people";

Read the section on how to name services. If you already have a protofile make sure it is a valid servis by running servisctl check your.servis.name. If everything looks fine, ask an administrator (vseth-it-support@ethz.ch) to publish the service.

Using dependencies

Only dependencies on the registry are available to be used. Dependencies outside of the vseth namespace can only be added by administrators. If you would like such a protofile to be added, ask an administrator to add it (vseth-it-support@ethz.ch). Administrators can commit the protofile directly to the registry https://gitlab.ethz.ch/vseth/0403-isg/sip-infrastructure/apis, using the import path as filename (i.e. a protofile imported using "import github.com/foo/bar.proto" has to be named github.com/foo/bar.proto).

How to update a Servis

If you have modified a service, you can push changes by running

servisctl push vseth.vis.tassenpranger

Note that you should first pull remote changes (if multiple people are working on the service, as you otherwise might overwrite other people's experimental changes). All your changes are tagged as experimental and will expire at a given date. Once you are happy with your modifications and like them to be added to the core servis description, you need contact the person administrating the service (currently all services are administrated by vseth-it-support@ethz.ch). You can add to the protofile and remove from the experimental code of your group, this ensures that there are no breaking changes on the core protofile. 

Core Concepts

On the coding day of 10.04.2020 there was a small intro on how to use serVIS:

Download video: mp4 format

On the SIP services are a core concept for development. That's why we provided a central place to manage all services.

The core concepts guiding the system are:

• All SIP API definitions are available publicly.
• Every developer can easily make experimental changes to try out new things, before making the changes binding for all APIs.

Experimental Changes

To add new features to a service, protocol buffers have to be updated and extended. With servis we introduce the notion of experimental features, which can be added to work with but not depended upon. To introduce an experimental feature simple use servisctl push <your updated service>, the additions will automatically be tagged with a comment and an expiration date. As long as an entity is tagged with such a comment, it can be modified and removed at any time. Once you're happy with the change you can promote (vseth-it-support@ethz.ch) it to be added to the core (that is the comment is removed and the entity can no longer be modified). 

Access Roles

Modifications to a service are always performed under a role. If you have multiple access roles for service you will be asked under with role you would like to make a modification. All your modifications can only be changed by users using the same role and users with administrative rights on the service. 

Roles are managed by Keycloak. Each student association can request their own access role and define which members to map into the role. The role receives the name of the association (e.g. vis), should an association require more than one role, they may request additional roles in their namespace (e.g. vis-con).

Protofile Management

First, your protofile will be formatted and linted. If a previous version of the protofile exists, your changes get tagged and afterwards the tagged and validated.

Formatted

The files are formatted according to the V2 Uber Protobuf Style Guide.

Linting

The linter checks against the following rules:

# rules recommended by google 
- ENUM_FIELD_NAMES_UPPER_SNAKE_CASE
- ENUM_NAMES_CAMEL_CASE
- ENUM_NAMES_CAPITALIZED
- FILE_HEADER
- MESSAGE_FIELD_NAMES_LOWER_SNAKE_CASE
- MESSAGE_NAMES_CAMEL_CASE
- MESSAGE_NAMES_CAPITALIZED
- RPC_NAMES_CAMEL_CASE
- RPC_NAMES_CAPITALIZED
- SERVICE_NAMES_CAMEL_CASE
- SERVICE_NAMES_CAPITALIZED
# rules currently required by servis
- COMMENTS_NO_C_STYLE
# additional rules
- SYNTAX_PROTO3
- RPCS_HAVE_COMMENTS
- ENUMS_HAVE_COMMENTS
- FILE_OPTIONS_REQUIRE_GO_PACKAGE
- FILE_OPTIONS_GO_PACKAGE_SAME_IN_DIR
- FILE_OPTIONS_REQUIRE_JAVA_PACKAGE
- FILE_OPTIONS_JAVA_PACKAGE_SAME_IN_DIR
- FILE_OPTIONS_EQUAL_JAVA_MULTIPLE_FILES_TRUE
- FILE_OPTIONS_JAVA_MULTIPLE_FILES_SAME_IN_DIR
- PACKAGES_SAME_IN_DIR
- PACKAGE_NO_KEYWORDS
- PACKAGE_LOWER_CASE
- PACKAGE_IS_DECLARED
- WKT_DIRECTLY_IMPORTED

More information about the linting rules can be found here: https://github.com/uber/prototool/tree/dev/internal/lint

Tagging

The old and new proto are compared line by line. If a line appears in the new proto that does not appear in the old protofile, an experimental tag comment will be added, that matches the following lines until the entity ends (that is a blank line after a comment block or the end of a message, enum etc.).

Validation

The experimental sections of the user's group are removed from the new and old protofile. If the files do not match, the update is invalid.

Admin

This section contains some information about administration topics.

Recover SerVIS

If the build pipeline fails because servis no longer runs…

If you still can access old builds:

1. Use the most recent image that did run fine and the configs (sip.yml)
2. Redeploy the old image with the old configurations

If you need to rebuild SerVIS:

1. You first need to get the protofiles from the registry (gitlab)
2. Add them to the project and make sure they get build correctly (currently this is the case, but if servis will distribute compiled files, you might have to generate them on your own).
3. Edit the build config, so the build of the servis backend no longer requires servis
4. Now you can re-build servis

Debugging Servis

• could not refresh repo:

  This error appears if Servis cannot get the current state of the git repository. This might also indicate that git is misconfigured (wrong repository URL or invalid credentials), especially if not even pulling works. To further debug such a problem, the docker environment can be run locally and attached to the remote git repository (it is recommended to fork the current master branch for testing).

How to extend serVIS

On the coding day of 10.04.2020 there was a small intro on how to extend serVIS:

Download video: mp4 format