Swagger

[magnusviri]

The Jamf Pro Swagger file is located here:

https://github.com/jamf/Classic-API-Swagger-Specification/blob/master/swagger.yaml

Loading

I used the following to convert it to json.

https://www.convertjson.com/yaml-to-json.htm

And I'm loading it that way. I couldn't see anything in that swagger parse module that does anything we want anyway. And it's one less requirement.

Using

So, it's kind of complex and I'm still trying to wrap my head around how to actually use it. So that's my next step.

[uurazzle]

Hi @magnusviri,

According to my source, it could be Brad Becker or James Felton of Jamf we can ping about how Jamf uses swagger or suggestions for other route to go with swagger integration & usage.

[magnusviri]

I got the swagger file to work. I converted it to json and I can import that easily. I also updated the records.py file to use the swagger file.

The reason I went down this rabbit hole was because I was hoping that instead of writing code that traverses the output of the Jamf API, I'm hoping we can write code that reads the swagger file and uses that to traverse the output of the Jamf API.

We started this because we were trying to read policy scopes. I looked at all of the things that had scopes and we thought it was a generic thing, but it's not. Everything that has a scope has different parts, so scope isn't generic at all. They each used many similar things, but again, there were many different looking scopes.

It will take longer to write code that reads the swagger file than to write code that deals with the policy scope, but if we do it right, using the swagger file will save us a ton of time and work will make finishing this project a possibility.

[Honestpuck]

I like that records.py a lot. I can clearly see what's what.

[magnusviri]

Ok, I'm not suppose to be working on this right now, but I just wanted to try something real quick. I had never heard of swagger before last week, but when I saw it, I had a hunch that it had some serious potential. Hearing that Kinobi used it as well made me want to look into it just a little bit more. So I did. And well... This site has a "generate client" menu item. I tried it and I got 15 MB worth of python code. I think this thing just wrote the entire python-jamf code we need... I'm sure we'll have to do a lot of work with it still, but we might be closer than we thought to finishing this whole thing (seems like we just barely started). My head is spinning a little... I don't really have time to check it out until Friday. But I'll include a zip if you guys want to look at it.
python-client-generated.zip

[Honestpuck]

That's a fairly complex code base. One of my complaints about python-jss was the code base was too complex and it made supporting it a nightmare. That's sort of how I ended up here 😺

[magnusviri]

We looked at this today. I hadn't heard of Swagger before 2 weeks ago. It looks like Jamf had adopted the OpenAPI (aka Swagger) spec.. The code that I generated is called Swagger CodeGen. I used the online editor to generate the client API that I uploaded in my comment above.

So, is Jamf following the Swagger spec completely? Because first, I can't see anywhere that they've told anyone, "Hey, this is OpenAPI". If they were following it, how come they haven't told anyone? If they had, I'm sure someone would've done what we are doing now a long time ago.

And second, something in the "python-client-generated" project isn't working. Specifically, when it tries to deserialize the json into an object, the json from the jamf server has an extra key that stops it from working. For example, if I pprint the json right on say line 616 in api_client.py I get this.

{'policy': {
     'account_maintenance': {
          'accounts': [],
          'directory_bindings': [],
          'management_account': {'action': 'doNotChange'},
...

However, it looks to me like the code, mainly line 621 in api_client.py expects the data to look like this:

{'account_maintenance': {
     'accounts': [],
     'directory_bindings': [],
     'management_account': {'action': 'doNotChange'},
...

So, is this a bug in the codegen client sdk? Or is Jamf returning json that isn't following the specification.

So what does this all mean? First, the CodeGen client sdk is not done. Because deserialization isn't working. I spent several hours trying to figure this out today...

Second, the funny thing is, before today, I didn't even know that we needed to be deserializing, but now I know, because as I've tried to figure out how this thing works, I've realized that there is so much here we should be doing that these guys are doing. People have obviously thought this all out and figured out how to do this all. The biggest example of this is that they've distinguished between the "api" and the models/objects. So a Policy doesn't get, put, update, or delete it's own data, the PoliciesAPI does.

Third, now I'm not sure what to do for the future. My main motivation for telling Sam to make the API as light as possible was because there's no way any of us had the time to maintain a full blown API that could communicate to every endpoint in the Jamf API. The best we could do was make a bunch of generic functions that make it easier to interact with it, but the side effect of this is that it put the burden on the API user to have to navigate the Jamf JSON. And as I started to use the API in the jctl scripts, I realized that this was a lot more work than I had anticipated. I was trying to come up with ways of making it easier, and my whole motivation for making the Records.py file was because it is easier to deal with classes and methods than straight json. But, like the python_jss project, we were only ever going to do part of Jamf's API. I couldn't see a way to cover all of the Jamf API generically.

That is the advantage of the CodeGen Client SDK. If we can get it working, it will cover the entire Jamf API. It's a lot of code though, 11 MB of code. That's a lot. And there's 39 model files just for policy stuff. That's a lot to wade through. But once you figure out how it all works, it's pretty easy because it's all very repetitive and predictable. In fact, because of that I'm surprised the CodeGen doesn't make more compact code and use more DRY principles. Considering the backing behind OpenAPI, there's probably a good reason it's the way it is. I'm sure smarter people than me are working on this.

And of course, the other advantage of the CodeGen Client SDK is that if we can get it working, it's done. My motivation behind a simple API library was because we would never be done.

I'm not sure where to go from here except that I can learn a lot by trying to get the CodeGen Client SDK to work. So that's what I'm going to work on. I can't say we will adopt it in the future, but I'm sure that we are going to adopt the principles simply because it looks like Jamf Pro is mostly following the spec and if we work with the spec ourselves this project will be much simpler. My guess is that we will probably use it because being done is too appealing. Our goal is to manage our servers and the faster we can start tweaking the server, the happier we are going to be.

But I'm thinking we can still keep the old way way we have been doing it part of the API. I'm just not sure how. Maybe we'll mix the 2. I don't know.

Anyway, like I said, I'm going to study this Client SDK because I have a lot to learn about how to work with API's.

[Honestpuck]

I'd bet that Jamf don't have perfect Open API Swagger. I'd also be reluctant to use the JSON from the server rather than the XML as I know of at least two spots where the JSON is broken. The one I can remember off the top of my head is in getting the version list for a patch the JSON only returns one version rather than the entire list. That's why I liked the way the API took the XML and then turned it right into a dict.

I'm the outsider here so I will let you to have the debate about how to proceed.

[uurazzle]

Hi @Honestpuck:

Feedback from Jamf contact...

The Classic API and the Jamf Pro API are two different beasts when it comes to JSON. The former, does have several documented product issues around inconsistent XML vs JSON responses. Whereas the Jamf Pro API only returns JSON and will always do it as documented.

That about wraps up my responses to those. I’m happy to answer any further questions

[uurazzle]

Hi @magnusviri & @Honestpuck:

I found a contact at Jamf that provided this feedback about swagger specification...

So, is Jamf following the Swagger spec completely? Because first, I can’t see anywhere that they’ve told anyone

In the /api/doc page of your instance you can see the OAS3 tag at the very top next to the Jamf Pro API. this is used to indicate that this page was generated via OAS3 (the successor to swagger) document(s).

I don’t have personal experience with that library and have not used to to create a client library myself. That being said your comment about the JSON being different is something I will investigate. What API endpoint was that model associated with?

There are a lot of API endpoints with more each release. While a client library is very convenient to use, knowing where and how to read the documentation may be a more time-efficient procedure vs maintaining a library that is hand crafted.

A lot of generated code does not follow common programming principals due to complexity of defining those relations and are not meant to be read or changed by humans. It’s an understood part of the trade off between writing code yourself and having the computer do it for you.

[uurazzle]

He said he is willing to setup a date/time to a video chat about it and the project if the group wants?

I also have time if you or your team want to video chat.

Let me know if there is interest.

I’m free all day Wednesday 9am-5pm CST, and after 12pm CST on Friday.

[magnusviri]

So I think this is what we kind of decided with a collective shrug. Most of this is based off of what we heard from the meeting with the Jamf developer.

The new API can't do what we want, so we are have to use the Classic API whether we like it or not. However, the Classic API has problems, isn't really Swagger compliant, it's not going to be fixed, it's probably going to be deprecated, but that it wont happen for a long time. Jamf is putting their efforts into the new API, it should be Swagger compliant, but that it isn't ready yet and full of a lot of "preview" stuff. The only endpoints that can really be depended on in the new API start with /v1, /v2, etc.

So, with all of this in mind, here is what we decided. We are going to stick with what we've done. The Swagger stuff was informative and we might even borrow some things from it. But we aren't going to switch to the CodeGen Client SDK (especially since it doesn't even work without tweaking). We also aren't going to flush out what we've done by deserializing all of the data from the server into Python objects. And we aren't going to try to make sure everything in records.py works. I'm going to create a new discussion listing our priorities.

Also, I'm not sure I like the way Github's discussions are organized. It's hard to see what has had new replies. There's also several discussions about the same topic (e.g. swagger and records.py). I don't know if they can be merged. But anyway, I just thought I'd say that. It seems too easy for things to get buried and lost.

[Honestpuck]

@magnusviri I follow your logic.

I also agree about discussions. I missed this until I got an email. Perhaps a (closed?) channel on MacAdmins Slack would work better. #python-jamf-dev

[uurazzle]

Ok, created the private channel and sent out invites to the group. Thanks: Richard Glaser (he/him/his) University of Utah, Marriott Library ITS richard.glaser@utah.edu<mailto:richard@scl.utah.edu> On Feb 6, 2021, at 9:06 PM, Tony Williams <notifications@github.com<mailto:notifications@github.com>> wrote: @magnusviri<https://github.com/magnusviri> I follow your logic. I also agree about discussions. I missed this until I got an email. Perhaps a (closed?) channel on MacAdmins Slack would work better. #python-jamf-dev — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#20 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABPQEGSWUDUNIUIW3IFL2Q3S5YGV7ANCNFSM4WFBAQOQ>.