Although there may be services that require no inout upon initiation, in most cases, you would probably want to provide your executable with some input parameters when executing it. Opereto has been designed to allow the developer a maximum flexibility regarding how to pass input parameters to his code and process them.


Passing input parameters to service code

As mentioned in previous article, Input and output parameters are specified in the service.yaml as item_properties and passed to the service code during runtime. Assuming that your service specification YAML is the following: 


...
...
item_properties:
- key: param1
  direction: input
  type: integer
  value: 50
- key: param2
  direction: input
  type: text
  value: client_123
- key: param3
  direction: input
  type: boolean
  value: true
- key: param4
  direction: input
  type: json
  value: 
      a: b
      c: d
...
... 

 

Opereto provides three ways to read the input parameters in your service code:


Option 1: via property files

Opereto run services in an isolated environment on the remote agent host. It creates a temporary work directory for every new process and stores in it all service files and directories. In addition, Opereto adds two files containing a JSON and YAML representations of the input parameters (arguments.json and arguments.yaml in correspondence).


arguments.json  

{
   "param1" : 50,
   "param2" : "client_123",
   "param3" : true,
   "param4" : {
                 "a" : "b",
                 "c" : "d"
              }
} 

  

arguments.yaml      

param1: 50
param2: client_123
param4: true
param4:
    a : b
    c : d 

    

The developer can simply process one of these argument files as shown in the following python example:

  

## my python service script..
import os
import simplejson

work_directory = os.environ['opereto_workspace'] ## the work directory where arguments files resides 
                                                   

with open(os.path.join(work_directory,'arguments.json'), 'r') as property_file:
    properties=simplejson.loads(f.readlines())

param1 = int(properties['param1'])
param2 = str(properties['param2'])
param3 = bool(properties['param3'])
param4 = properties['param4']
.....
.....

   

In the last example, opereto_workspace is a built-in input parameter that Opereto passes as environment variables or YAML/JSON arguments to the process at run time. We will discuss built-in input parameters in the next section.


If you use python to write your services, we recommend using pyopereto, Opereto official python client. Pyopereto simplifies service input handling by reading the arguments.json into a dictionary property of the client object. 



from pyopereto import OperetoClient

client = OperetoClient()
input = client.input## input is a dictionary containing all key-value input properties 
...
...

   

Otherwise, you can build a simple wrapper function that reads the arguments.json file and store it in some data structure or class. 



Option 2: via environment variables

As mention above, Opereto creates an isolated process environment per new process. In addition to the argument files, it also passes all input parameters and built-in parameters as environment variables specific to that isolated environment. Thus, the developer can easily access these environment variables in his code. For example:

   

## run.py
import os
import simplejson

bool = lambda x: x=='true' and True or False

param1 = int(os.environ['param1'])
param2 = str(os.environ['param2'])
param3 = bool(os.environ['param3'])
param4 = simplejson.loads(os.environ['param4'])
.....
.....

   

While this option is less useful for higher level languages like python, ruby, java (that include libraries that can process the JSON or YAML argument files quite easily), reading the environment variables may be very useful if your service code is a simple batch file. For instance:   

#/bin/bash

if [ -z $param1 ] then;
    echo "param1 is missing.." 
fi

...
... 

   

Option 3: via the command line


If you have a legacy or a third-party executable code that you cannot or don't want to change, you may use this option to wrap it as an Opereto service. Assuming that this executable gets input parameters via the command line, you can build your service.yaml as follows:   

cmd: my.exe ${param1} ${param2}
item_properties:
- key: param1
  value: 
  mandatory: true
  type: text
  direction: input
- key: param2
  value: 50
  mandatory: true
  type: integer
  direction: input
...
... 

     



Opereto build-in parameters


In addition to the service input provided by the user, Opereto stores a set of useful parameters for every service process executed that may be processed as described above. Some are required for flow orchestration as you will see in coding examples from time to time. Following is the list of opereto built-in parameters:


Parameter Name
Parameter Description
pid
current running Opereto process id
opereto_home
opereto home directory (it is ~/.opereto where ~ is the home directory of the user started the agent.
opereto_workspace
current process work directory (where all service files, as well as input property files) resides
opereto_agent
opereto_agentagent name
opereto_agent_pid
agent os process id
opereto_agent_home
agent home directory
opereto_host
operetobox host url (e.g. https://10.0.0.2)
opereto_user
the username that the current agent logged-in with
opereto_password
the password that the current agent logged-in with
opereto_source_flow_id
The Opereto process id of the ancestor in case that the process is a flow member, otherwise the id of the current process.
opereto_parent_flow_id
The Opereto process id of the direct parent if the process is a flow member, otherwise an empty string.
opereto_product_id
Product under test
opereto_service_version
The service version if passed or empty string.
opereto_originator_username
The user initiated this process or the flow.
opereto_originator_email
The email address of the user started this process or the flow, otherwise empty string.
opereto_originator_mobile
The mobile number of the user initiated this process or the flow that this process is a member of or otherwise empty string.
opereto_execution_mode
'production" or "development" (sandbox)


Handling service output

Opereto handles the output of a service execution in tree ways: process log, process output properties, and process status.


Process log: the standard output and error streams

Opereto collects anything written to the console (standard output and error). Thus the developer only needs to use the standard and error streams (print functions) to write data to log. Opereto indexes all log entries allowing a granular search of words and phrases in the process log.


Process output properties

Process properties may be used as well to store process output. It is especially convenient when passing the output of the previous process to the successive one within a flow. Only properties that are marked as “output” in the direction specification in the service.yaml can be used to output data to. To update process properties, the developer may use the Modify Process Output REST call. For example:


## run.py
import os
import json
import requests

s0 = requests.Session()
s0.auth = (os.environ['opereto_user'], os.environ['opereto_password'])
.....
.....

r = s0.post(os.environ['opereto_host']+'/processes/'+os.environ['pid']+'/output', data=json.dumps({
      "key": "new_param",                  
      "value": "this is a new property value"               
    }))
.....
.....

   

And again, using a simple wrapper as PyOpereto make things super easy: 

 

from pyopereto.client import OperetoClient

client = OperetoClient()
...
...
client.modify_process_property('my_output_property', 100) 

     


Process status

Opereto supports the following process states: 

STATUS
ICON
DESCRIPTION
success

Set by service code: In case that the process exited with 0, and no exceptions occurred.
failure

Set by service code: In case that the process exited with 2.
warning

Set by service code: In case that the process exited with 3.
error

Set by service code: In case that the process exited with 1 or exit code > 3 or exception raised.
timeout

Set automatically by Opereto if the process runs time is higher then the service timeout specification. In that case, the process will be terminated by the agent running it.
terminate_pending

Set automatically by Opereto when the user terminates a given process. Opereto will change it to “terminated” after the agent will complete the termination.
terminated

Set automatically by Opereto. Indicates that the user has terminated the process.
registered

Set automatically by Opereto for new processes. At this point, the process is registered but had not been collected by the relevant agent yet.
in_process

Set automatically by Opereto. Indicates that the process had been collected by the relevant agent and is now running.

 


PyOpereto provides a list of predefined process status code: SUCCESS, FAILURE, WARNING, ERROR   


from pyopereto.client import OperetoClient
client = OperetoClient()

def my_service():
    try:
...
        ...

        return client.SUCCESS  
    except Exception, e:
        return client.FAILURE

if __name__ == "__main__":
    exit(my_service())







System Global Variables 


A common case to share input properties with many processes is using system global parameters. If for example, many of our tests use a common username and password to login into the software under test application, it makes sense to keep those properties as globals and only pass a pointer to them in each test. This way, we can change the values only once rather than maintaining each and every service.yaml code. Global parameters can be also used to store secret keys and token. For example, the access and secret keys to access the testing AWS account, internal server credentials, licenses, private keys etc.   


Opereto provide a mechanism to manage globals and use them in automation services. Globals may be modified by the system administrators only. However, every developer can use every global when writing automation services.

Globals in Opereto may be specified as "hidden". If so, Their value will no be shown in process execution forms and I/O tabs. This option is useful for passwords, secret configuration items etc. 


Manage globals via UI


** See also Opereto REST API - Manage Globals



Passing globals as input property values is done by specifying the prefix "GLOBALS." followed by the global name. For instance:

...
...
item_properties:
-   editor: text
    key: aws_secret_key
    mandatory: true
    direction: input
    store: []
    type: text
    help: AWS account secret key
    value: GLOBALS.aws_secret_key
...
...





Process runtime cache

In addition to the static global properties mechanism just described, Opereto provides another mechanism to work with global params during runtime: process runtime cache. Let us take an example of one useful use case of this mechanism:


Let us assume that we have a flow that creates a new test environment on AWS account. Let us assume now that we have a few test accounts and we want to be able to change the account credentials at runtime based on some algorithm that calculates the load on each account. All AWS related services use a default global called account_id that is defined as a global parameter in Opereto. However, we can use the runtime cache to override this account during runtime. 


In the parent flow, we add the account_id runtime parameter based on the load algorithm as follows:


from pyopereto.client import OperetoClient

client = OperetoClient()
...
...

## Some algorithm calculating the load on each account and returns a valid AWS account to use

def get_aws_account():
      ....
      ....

      return aws_account


client.set_process_runtime_cache('aws_account_id', get_aws_account())

     ....
     ....

## start a sub process 
client.create_process('aws_create_cloud_formation_stack', )...


In the sub process service code (e.g. aws_create_cloud_formationf_stack in our example) we add this:


....
....
## look for the account id in the runtime cache and if not found, use the default one provided as an input property for this service
 
aws_account_id = client.get_process_runtime_cache('aws_account_id') or client.input['aws_account_id']

....
....



*   We used PyOpereto in the example but runtime caching have a REST API too and may be accessed from any code. For more detail see Opereto REST API - Process Runtime Cache.

** Caution: using a central cache includes also some risk and if not properly designed and handled, may lead to hard-to-debug runtime errors. As a thumb rule, it is always advised to have only one process writing once a given key-value cache and other sub processes that only read this key at runtime as needed (same as described in the above example).