Model store

Hmile provides the ability to store and retrieve torch models. To do that we have built three base classes :

MetaModel : model informations like name, description, performance, tags, etc.
ModelStore : a store for torch models

MetaModel

class Hmile.ModelStore.MetaModel(model: Union[Module, str], performance: float, description: str, columns_list: List[str], tags: List[str], creation_date: Optional[datetime] = None, meta: dict = {})

This objet is used to store the model and to get it back from a model store. It also stores meta informations about the model.

Variables

model – the pytorch model. Can be None if no model is associated with the MetaModel
performance – an arbitrary number between 0 and 1 to describe the performance of the model
description – a concise description of the model
columns_list – an ordered list of the columns used to train the model
tags – keywords to describe the model. This is the key value to search a MetaModel from a MetaModelStore
creation_date – the datetime object when the model was created
meta – a dict object to store any other information about the model

ModelStore

class Hmile.ModelStore.ElasticModelStore(es_url: str, es_user: str, es_pass: str)

Store meta models information in ElasticSearch in the index ‘models’

get(limit=10000, **kwargs) → List[MetaModel]

Return a list of MetaModel objects, corresponding to the keyword arguments. Each meta model which match to each keyword arguments will be returned.

Arguments can be :

performance : float
description : str
columns_list : List[str]
tags : List[str]
creation_date : datetime

Parameters: limit (int, optional) – number of Elasticsearch results per pagination. Defaults to 10000.
Raises: Exception – if the argument is not in the list of allowed arguments
Returns: the requested list of MetaModel objects
Return type: List[MetaModel]

store(meta_model: MetaModel)

Store a MetaModel object in elasticsearch in the index ‘models’. If the index does not exist, it will be created.

Parameters: meta_model (MetaModel) – MetaModel object to store

Example

This can be used to store a model an its meta model as follow :

from torch import nn
from Hmile.ModelStore import MetaModel, ElasticModelStore

ELASTIC_URL = "https://myelastic.com:9200"
ELASTIC_USER = "myuser"
ELASTIC_PASSWORD = "mypassword"

# we create an example model
model = nn.Sequential(
    nn.Linear(10 , 20)
)

# we create a MetaModel object
meta = MetaModel(
    model, # the torch model
    0.5, # the model performance
    'test', # the model description
    ['a', 'b', 'c'], # the columns list used to train the model
    ['test'] # the tags
)

# we create a MetaModelStore object
meta_model_store = ElasticModelStore(
    ELASTIC_URL,
    ELASTIC_USER,
    ELASTIC_PASSWORD)

# then we store our mode_model in it
meta_model_store.store(meta)

Then we can retrieve our model :

from Hmile.ModelStore import ElasticModelStore

ELASTIC_URL = "https://myelastic.com:9200"
ELASTIC_USER = "myuser"
ELASTIC_PASSWORD = "mypassword"

# we create a MetaModelStore object
meta_model_store = ElasticModelStore(
    ELASTIC_URL,
    ELASTIC_USER,
    ELASTIC_PASSWORD)

# we retrieve the list of models with the tag 'test'
meta_model_list = meta_model_store.get(tags='test')

# we now get the model
model = meta_model_list[-1].model

# we can now use our model
model(torch.rand(10))