API examples in perl/python?

J

Jfs

Dabbler
Joined
Feb 23, 2023
Messages
23
I've been looking at the API examples showin in https://www.truenas.com/docs/api/rest.html but I don't see any good examples of setting up the environment for the perl/python examples. It would be great if someone had some good examples to learn from. And I've looked at various API threads in the forums, but none of them are any use. I looked at:

Official Python SDK
Swagger codegen of the v2 api
TrueCommand API
API problems


I've tried setting up a python virtualenv and installing modules, but where do I get the 'swagger_client' module from? It's not available in 'pip3 install swagger_client' and it sorta looks like you need to build it yourself, or use the exposed API to build this module? I'd love to see some updated, simple exampls for creating a dataset then adding an NFS share to it.

Of course a discussion of the security implications would also be useful, since I don't want to send my root user password through http connection if I can help it.

And the curl examples are fairly useless too, since it's not explained how to use the API key (which I did generate) to do stuff. And of course exposing the API key is a bad thing as well when using http, so only using https would be better, right?

Please assume I'm a semi-idiot when it comes to python. I've used it, but not alot. So start me with the virtualenv setup and installing the modules needed, then onto the swagger_client modeule setup.

Thanks,
 
J

Jfs

Dabbler
Joined
Feb 23, 2023
Messages
23
So I did a bunch of digging and I've tried downloading the swagger-codegen.jar file and then running it against the system to try and build a python client. Here's my output:
:~/work/TrueNAS$ java -jar swagger-codegen-cli.jar generate -i http://truenas.example.com/api/v2.0 -l python -o test-client
[main] INFO io.swagger.parser.Swagger20Parser - reading from http://truenas.example.com/api/v2.0
[main] INFO io.swagger.parser.Swagger20Parser - reading from http://truenas.example.com/api/v2.0
[main] INFO io.swagger.codegen.ignore.CodegenIgnoreProcessor - No .swagger-codegen-ignore file found.
Exception in thread "main" java.lang.RuntimeException: missing swagger input or config!
at io.swagger.codegen.DefaultGenerator.generate(DefaultGenerator.java:766)
at io.swagger.codegen.cmd.Generate.run(Generate.java:307)
at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:35)
Click to expand...

Which doesn't really help me.


But I went back to just using curl with an API key and I can now create volumes. This took a crap load of trial and error, and the docs are worse than useless that the API gives back to you. It doesn't list all the data it expects, etc. So here's a stupid simple curl script to create a volume:

Code:
#!/bin/bash

KEY="long-string-of-characters-you-got-from-API-key"

url="http://192.168.1.100/api/v2.0"
pool="tank"
vol="test"

json="{ \"type\": \"FILESYSTEM\", \"name\": \"${pool}/${vol}\" }"

#curl -s -X GET -H "Authorization: Bearer ${KEY}" ${url}/pool/dataset

curl -X POST "${url}/pool/dataset" -H "Content-Type: application/json" -H "Authorization: Bearer $KEY"  -d "$json"


Now to try and figure out how to make this all work in a python script, and to create NFS Shares using the API as well. God what a pain to learn! Hopefully this will help someone else down the line.
 
tprelog

tprelog

Patron
Joined
Mar 2, 2016
Messages
297
  • Like
Reactions: Jfs
J

Jfs

Dabbler
Joined
Feb 23, 2023
Messages
23
tprelog said:
I know of one example written in python - A TrueNAS integration for Home Assistant.

github.com

GitHub - tomaae/homeassistant-truenas: TrueNAS integration for Home Assistant

TrueNAS integration for Home Assistant . Contribute to tomaae/homeassistant-truenas development by creating an account on GitHub.
github.com github.com


I'm unsure how useful that is, but it's something nonetheless.
Click to expand...

This is a good set of clues on what needs to be done. I've got some stupid curl bash script to create a volume, but it was going to get really hairy to pass in the exports for the various networks and hosts that I needed to do, so I'll be moving over to python this week to make it work better.

Thanks!
John
 
J

Jfs

Dabbler
Joined
Feb 23, 2023
Messages
23
So I've grabbed the Java based tool called "swagger-codegen-cli.jar" but I can't figure out how to make it work with the TrueNAS Swagger setup. Anyone have any hints? I think I mostly need to pull down the API in JSON format, but for the life of me I can't make it work.

$ java -jar swagger-codegen-cli.jar generate -o cli -i http://209.243.139
.60/api/v2.0 -l python
[main] INFO io.swagger.parser.Swagger20Parser - reading from http://192.168.139.60/api/v2.0
[main] INFO io.swagger.parser.Swagger20Parser - reading from http://192.168.139.60/api/v2.0
[main] INFO io.swagger.codegen.ignore.CodegenIgnoreProcessor - No .swagger-codegen-ignore file found
.
Exception in thread "main" java.lang.RuntimeException: missing swagger input or config!
at io.swagger.codegen.DefaultGenerator.generate(DefaultGenerator.java:766)
at io.swagger.codegen.cmd.Generate.run(Generate.java:307)
at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:35)
 
M

MisterE2002

Patron
Joined
Sep 5, 2015
Messages
211
Why are you fumbling with swagger jar if i may ask?
You answered before but i do not really understand the issue.

As stated, TrueNAS internal contains a Swagger web client. You can test everything (even with login). You can also use PostMan to do testing.
The http://<host>/api/docs/ has also documentation. But not sure if it is complete.

After that you can figure out what to Post. (i also use my browser debugger to figure out their websocket calls)

Then add those to the Python scripts. It would be easy to adapt my scripts to your needs.
I use Linux so i can run them directly. I would assume this can also with Windows.
 
M

MisterE2002

Patron
Joined
Sep 5, 2015
Messages
211
1677698189332.png



and all parameters. Not sure how to figure out which are required though.

1677698261099.png
 
M

MisterE2002

Patron
Joined
Sep 5, 2015
Messages
211
and using python. Still contains non relevant crap

Desktop/mkdataset/backups/mkdataset.py
Code:
import requests
import json
import yaml
import os
import logging
import logging.config
import time
import csv
import sys
import io


#http://<host>/api/docs/


failure_count = 0
successful_count = 0


script_folder = os.path.dirname(os.path.realpath(__file__))
log_config_path = os.path.abspath(os.path.join(script_folder, '..', 'backups_var', 'truenas_log_settings.yaml'))
config_path = os.path.abspath(os.path.join(script_folder, '..', 'backups_var', 'truenas_credentials.json'))
logfile_path = os.path.abspath(os.path.join(script_folder, '..', 'backups_var', 'truenas.log'))
rsync_lst_path = os.path.abspath(os.path.join(script_folder, '..', 'backups_var', 'rsync.lst'))

config = yaml.safe_load(open(log_config_path, 'r'))
# overrule logging location with relative path
config['handlers']['file_handler']['filename'] = logfile_path


logging.config.dictConfig(config)
logger = logging.getLogger()

memory_stream = io.StringIO()
memory_handler = logging.StreamHandler(memory_stream)
memory_formatter = logging.Formatter('[%(levelname)8s] %(message)s')
memory_handler.setFormatter(memory_formatter)
memory_handler.setLevel(logging.INFO)
logger.addHandler(memory_handler)


with open(config_path) as json_data_file:
  config = json.load(json_data_file)






def make_dataset(host, api_key, dataset):
  api_call = 'http://{}/api/v2.0/pool/dataset'.format(host)
  headers = {"Content-Type":"application/json", "Authorization": "Bearer " + api_key}
 
  json_obj = {
    "name": dataset
  }
  logger.info(json_obj)
  resp = requests.post(api_call, json=json_obj, headers=headers)

  if resp.status_code != 200:
    errmsg = 'Failed to create "{}"'.format(dataset)
    logger.error(errmsg)
    raise Exception(errmsg)
  else:
    logger.info('Created dataset "{}"'.format(dataset))



host = config["truenas"]["host"]
api_key = config["truenas"]["api_key"]
make_dataset(host, api_key, "tank/make_dataset_using_python")



Desktop/mkdataset/backups_var/truenas_credentials.json

Code:
{
    "truenas": {
        "host":"192.168.1.210",
        "api_key":"2-ax6Ow6AmUqmldCqGMf6OH1YWhQyq4g3lhKqcsyvBjFZcdG4dRWaMYu2NAbE4GGG0"
    }
}



Desktop/mkdataset/backups_var/truenas_log_settings.yaml

Code:
version: 1
disable_existing_loggers: False
formatters:
    console_formatter:
        format: "%(levelname)s - %(message)s"
      
    file_formatter:
        format: "[%(asctime)s] [%(levelname)s] [%(funcName)s] - %(message)s"
      
filters: []
handlers:
    console:
        class: logging.StreamHandler
        level: INFO
        formatter: console_formatter
        stream: ext://sys.stdout

    file_handler:
        class: logging.FileHandler
        level: DEBUG
        formatter: file_formatter
        # Please overrule default filename location using python script!
        filename: "/tmp/logging.log"
        

        
root:
    level: DEBUG
    handlers: [console, file_handler]
 
J

Jfs

Dabbler
Joined
Feb 23, 2023
Messages
23
I've been fumbling with Swagger because I'm new to hacking with RestAPIs, and the docs on the TrueNAS site talk about using Swagger to manager things. I've basically just been writing my own python scripts to do various things like:

1. create dataset
2. create nfs shares for the data set:
a. RO share for a set of networks.
b. RW share for a set of hosts.

Then some scripts to do reporting, checking of datasets, etc. All simple stuff to make my life simpler so I don't have to use the GUI for everything.

And I appreciate all the stuff you've posted, I'll look it over tomorrow and do more hacking and try to include this into my code base. It's also a good way for me to learn Python better.
 
danb35

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
J

jose-pr

Dabbler
Joined
Apr 3, 2022
Messages
10
Old post but I was looking around for info on this also , posting an example based on some code I used recently: Motivated me to see if I could add some typing info and helper methods around the WebSocket client available in middlewared
github.com

pytruenas/lib/example.py at main · jose-pr/pytruenas

Wrapper around TrueNAS middlewared client. Contribute to jose-pr/pytruenas development by creating an account on GitHub.
github.com github.com
It was easier that I expected, looked at /api/docs/ for the websocket and when in doubt looked with vscode around /lib/python3/dist-packages/middlewared/ , especially the plugins directory to see how the calls are used internally. Was able to provision a bunch of storage servers with the Middleware client.

  • Configure create admin users
  • Join Domain
  • Setup IDMAPs
  • Create datasets
  • Create smb/nfs shares
  • Configure SSH
  • ...
The calls are really simple and align with you can do from the GUI. (The GUI is using the same api to make all the changes)
 
Last edited:
You must log in or register to reply here.

Similar threads

K
Swagger codegen of the v2 api
Replies
1
Views
1K
rmblr
rmblr
D
How to create jail through API. TrueNAS-13.0-U3.1
Replies
5
Views
1K
deamonmv
D
Basil Hendroff
Guided examples on using the API
Replies
5
Views
2K
aervin
aervin
seeN
  • Locked
Plugins/Jails restart via API (+after API unlock)
Replies
4
Views
1K
seeN
seeN
O
Inquiry about Managing SMB Shares through the TrueNAS API
Replies
1
Views
672
jose-pr
J
Top