However, there are some situations where you might not want to run tests against something other than a live server, or maybe you just want to use curl to extract some better usage statistics out of your requests. Tavern’s plugin system can be used to override this default behaviour (note however that it still ONLY supports HTTP and MQTT requests at the time of writing).
The best way to introduce the concepts for making a plugin is by using an
example. For this we will be looking at a plugin used to run tests against a
local flask server called tavern_flask
or another plugin used to run tests against FastAPI/Starlette
TestClient called tavern_fastapi.
The entry point¶
Plugins are loaded using two setuptools entry points, namely
HTTP tests and
tavern_mqtt for MQTT tests. The built-in requests and paho-mqtt
functionality is implemented using plugins, so looking at the
in the Tavern repository will also be useful as a reference when writing a
The entry point needs to point to either a class or a module which defines a preset number of variables.
Something like this should be in your
setup.cfg to make sure
Tavern can pick it up at run time:
# setup.cfg # A http plugin. tavern_http is the entry point that Tavern searches for, # 'requests' is the name of your plugin which is selected using the # --tavern-http-backend command line flag. This points to a class in the # tavernhook module. tavern_http = requests = tavern._plugins.rest.tavernhook:TavernRestPlugin # An MQTT plugin. Like above, tavern_mqtt is the entry point name and # 'paho-mqtt' is the name of the plugin. This points to a module. tavern_mqtt = paho-mqtt = tavern._plugins.mqtt.tavernhook
Extra schema data¶
If your plugin needs extra metadata in each test to be able to make a request,
extra schema data can be added with a
schema key in your entry point. This
should be a dictionary which is just merged into the base schema
There is currently only one key supported in the schema dictionary,
initialisation. This defines a top level key in each test which your session
or request classes can use to set up the test (see the mqtt
for an example of how this is used to connect to an MQTT broker).
session_type should return a class which describes a “session” which will be
used throughout the entire test. It should be a class that fulfils two
- It must take the same keyword arguments as the ‘base’ session object to
create an instance for testing. For
HTTP tests this is the same arguments as a
object, and for MQTT tests it is the same arguments as specified in the
If your plugin does not support some of these arguments, raise a
NotImplementedErrorwhich a short message explaining that it is not supported.
- After creating the instance, it must be able to be used as a context manager.
If you don’t need any functionality provided by this, you can define empty
__exit__methods on your class like so:
class MySession(object): def __enter__(self): pass def __exit__(self, *args): pass
is fairly simple, it just creates a flask test client from the
flask::appdefined for the test (see schema documentation above) and dumps the body data for later use when making the request.
request_type is a class that encapsulates the concept of a ‘request’ for your
plugin. It takes 3 arguments:
sessionis the session instance created as described above, for that request type at that stage. There may be multiple request types per test, but only one request is made per stage.
rspecis a dictionary corresponding to the request at that stage. If you are writing a HTTP plugin, the dictionary will contain the keys as described in the http request documentation. If it is an MQTT plugin, it will contain keys described in the MQTT publish documentation.
test_block_configis the global configuration for that test. At a minimum it will contain a key called
variables, which contains all of the current variables that are available for formatting.
In the constructor, this request type should validate the input data and format the request variables given the test block config.
The class should also have a
run method, which takes no arguments and is
called to run the test. This should return some kind of class encapsulating
response data which can be verified by your plugin’s response verifier class.
Tavern knows which request keyword (eg
mqtt_publish) corresponds to
your plugin by matching it to the plugin’s
request_block_name. For the moment,
this should be hardcoded to
request for HTTP tests.
- The base
request object formats the keys and does some extra verification, such as
logging a warning if a user tries to send a body with a
- The paho-mqtt request formats the input data and just makes sure that a user is not trying to send two kinds of payloads at a time.
- tavern-flask reuses functionality from Tavern to format the keys and do extra verification.
Getting the expected response¶
get_expected_from_request should be a function that takes 3 arguments:
stageis the entire test stage (ie, including the request block, test name, response block, etc) as a dictionary
test_block_configis as above
sessionis as above
This function should use this input data to calculate the expected response and perform any extra things that need doing based on the request or expected response. This will normally just be formatting the response block based on the variables in the test block config, but you may need to do extra things (such as subscribing to an MQTT topic).
verifier_type is a class that encapsulate the concept of verifying a response
for your plugin. It should inherit from
take 4 arguments:
sessionis as above
nameis the name of the test stage currently being run. This can be used for logging debug information.
expectedis the return value from
test_block_configis as above.
It should also define a couple of methods:
verifytakes one argument, which is the return value from the
runmethod on your request class. It should read whatever information is relevant from this response object and verify that it is as expected, then return any values from the response which should be saved into the test block config. A plugin does not need to save anything - just return an empty dictionary if you don’t want to save anything. There are some utilities on
BaseResponseto help with this, including printing errors and checking return values. This should raise a
tavern.exceptions.TestFailErrorif verification fails. The easiest way to verify the response is to call
self._adderrwith a string to a list called
self.errorsfor every error encountered. If there is anything in this dictionary at the end of
verify, raise an exception.
__str__should return a human-readable string describing the response. This is mainly for debugging, and should only give as much information as you think is required. For example, a HTTP response might be printed as (
Like with a request, Tavern knows which verifier to use by looking at the
- The base requests verifier Checks a variety of things like the expected headers, expected redirect locations, cookies, etc.
plugin needs to wait for the specified timeout to see if a message was
received on a given topic. Note that there does not need to be a response for
an MQTT request - a stage might consist of just an
mqtt_publishblock with no expected response.
just reuses functionality from the base verifier again. Because the flask
Responseobject is slightly different from the requests one, some conversion has to be done on the data.