If your team is using Jira then at some point you’re going to want to look at some flow metrics to see how you can improve. You’ll very quickly discover that Jira only provides two charts out of the box and that neither one is terribly useful. The cumulative flow chart has known bugs and the control chart is both difficult to read and has problems of it’s own.

The good news is that Jira has all the data you want, it’s just hard to get at it. If you have the budget for it, purchasing a Jira plug-in like Actionable Agile will make getting that data easy.

For the rest of this article, I’m going to assume that you don’t have budget or approval to get one of the metrics plugins and that you’re going to have to get data through the Jira API.

See also the Jira-Export tool that we built to solve this same problem.

Pulling the data is fairly easy. The complexity comes when we want to walk through it. Most of the complexity isn’t in the API itself but rather in the decisions we have to make about when to start and stop the clock when calculating cycle times. These decisions are often unique to specific teams and make it almost impossible to build a generic script to do this.

Getting the data

Note that there are multiple different authentication approaches and you’ll have to find the right one for you. What we’re describing here is the API-Key approach supported by Jira Cloud. If you’re using Jira Server then you’ll have different authentication approaches to consider. Basic auth is probably the easiest but you’ll have to check the Jira documentation to see what’s best for you.

The first step is to get an API key and the instructions to do that are at Atlassian Support

curl --request GET \
  --url "https://improvingflow.atlassian.net/rest/api/2/search?jql=project=SP&maxResults=100&startAt=0&expand=changelog" \
  --user email@example.com:ApiToken \
  --header "Accept: application/json"

You’re going to have to replace a couple of pieces in here:

Replace email@example.com and ApiToken with your email and Jira API tokens respectively. The token you got in the step above and the same email you used when getting the token.
Replace improvingflow.atlassian.net with your own domain. If you’re hosted on Atlassians cloud then it will still be something.atlassian.net. If you’re self hosted then the domain will be the same one that jira normally loads from when you hit it from a web browser.
Replace project=SP with some JQL query. In this case, I’m returning everything from the project ‘SP’ but you could use a filter or any other valid JQL query. Note that if you use any interesting characters then this will have to be URL encoded.
Replace startAt=0 with the record number that you want to start with. This is part of the pagination support in Jira so your first request will always be 0. maxResults is also part of pagination.

Note that expand=changelog is the magic that will return all the change history for the issues. Without that, we have almost none of the information we might want to mine from here.

Running this curl command will bring down the first page of results. Now we can start to unpack it.

Note that the API won’t return any data that you don’t have permission to see through a web browser. If you aren’t seeing what you expect from the API results then first verify that you have permission to see that same data when you go in through a regular browser.

Note that from some Windows systems, I’ve had to include the –ssl-no-revoke flag to the curl command. I don’t really understand why but it fixed a problem at one client so I’m including it here.

Unpacking the data

What we have in the resulting JSON file gives us the full change log for every issue returned. This includes the timestamp of every status transition, every time the flag was added or removed, the priority was changed or any time any other field was modified.

Most likely the first thing you’ll want to do is determine the start and end times for the issue and this is more complicated than you’d expect because of all the various edge cases. See this article for a discussion of the factors you’ll want to consider.

With end times, we can now determine throughputs. With both start and end, we can calculate cycle/lead times.

We’re going to ignore the vast majority of the data in the JSON file. When you open it up, you’ll see far more fields that I have in this sample - I’ve stripped out everything except the fields I care about.

{
  "key": "SP-1",
  "changelog": {
    "startAt": 0,
    "maxResults": 2,
    "total": 2,
    "histories": [
      {
        "created": "2021-06-18T18:44:21.050+0000",
        "items": [
          {
            "field": "status",
            "toString": "In Progress"
          }
        ]
      }
    ],
  "fields": {
    "issuetype": {
      "name": "Story",
    },
    "created": "2021-06-18T18:41:29.398+0000",
    "priority": {
      "name": "Medium",
    },
    "status": {
      "name": "In Progress",
      "statusCategory": {
        "name": "In Progress"
      }
    },
  }
}

The bits between changelog and histories are relevant for pagination.

The data inside the histories block is the full changelog for the issue. There may be multiple items in the items list and they all share the same timestamp at the top.

Inside the item itself, the field tells us what kind of change it is. This could be status, priority, custom fields, flag, etc. The value for that change is in the poorly named toString field.

Inside the fields section are all those things that are at the issue level. The type (Story, Bug, etc), status, priority and creation timestamp. Note that the creation timestamp isn’t considered a change and therefore doesn’t show up in the history.

How do I know what columns are on the board?

If you want to know the columns and their positions then you’ll need a second API call.

curl --request GET
  --url https://improvingflow.atlassian.net/rest/agile/1.0/board/BOARD_ID/configuration
  --user email@example.com:ApiToken \
  --header Accept: application/json

You’ll need to replace email@example.com and ApiToken just as with the curl command above.

You’ll also need to replace BOARD_ID with the id number of the board. When you’re looking at that board in a web browser then the URL you are on will look something like the following:

https://improvingflow.atlassian.net/secure/RapidBoard.jspa?rapidView=1

The ID is the rapidView parameter. In this example, it’s 1.

You may also need –ssl-no-revoke as mentioned above.

The JSON returned by this call will look something like this. Again note that I’ve removed all the fields I don’t care about.

{
  "filter": { "id": "10000" },
  "columnConfig": {
    "columns": [
      {
        "name": "Backlog",
        "statuses": [ { "id": "10000" } ]
      },
      {
        "name": "Ready",
        "statuses": [ { "id": "10001" } ],
        "max": 4
      },
      {
        "name": "In Progress",
        "statuses": [ { "id": "3" } ],
        "max": 3
      },
      {
        "name": "Done",
        "statuses": [ { "id": "10002" } ]
      }
    ]
  }
}

Things to note:

The filter may be useful as a query parameter to the first curl command if you want.
Statuses are a list because it’s possible to have multiple statuses mapped to a single column.
max is the WIP limit for the column. min is also possible if you have minimums set.
name is the name assigned to the column, not the name of a status.

Conclusion

Now you have the ability to pull the data out of Jira and pull the pieces you need out of that data. The harder part is deciding how we’ll calculate start and stop and that’s highly dependent on your company and your team and so it’s almost impossible to create a “standard” script for it.

These instructions have worked for me at many different clients with a mix of Jira Cloud and Jira Server. If your configuration requires something different then let me know and I’ll add it.