GET request issue for 3000 co-ordinates

[Kumarsnks]

Hi all, Good day

Issue

I have an issue with the GET request made to an OSRM for the 3000 co-ordinates to get duration and distance. I am using table/v1/driving endpoint to get the response.

The server(8GB) is crashing for the large amount of data(3000) has been sent for the request.

Is there any alternative HTTP requests method for the large amount of data(>3000)?
Is there any way to get the distance and duration to each P2P location?

Kindly help me with the above mentioned problem.

[danpat]

If you're using osrm-routed, then there is no limit for HTTP GET - I've successfully sent 10k+ coordinates using a client that supports it.

You don't say what kind of crash you're experiencing - is it an OOM error (typically, you'll see a message like Killed on the console if this happens). If it is OOM, then you need to either (a) make morememory available (use a bigger server), or (b) split the request into multiple smaller /table requests, then stitch the results back together. You can use the sources= and destinations= to construct smaller /table requests to fill out parts of a larger matrix.

For distance and duration, look at the annotations= parameter in the documentation at http://project-osrm.org/docs/v5.24.0/api/#table-service

[Kumarsnks]

@danpat Thanks for the solution. The server was crashing because of memory issue, but I need to overcome this by making multiple smaller /table request.

This the code which I am using for construction of smaller /table request.

longLat_list_3000=  "lat-longs......."

lat-long = longLat_list_3000.split(";")

res_list = []
for i in lat-long:
    table = []
    for j in range(0, len(lat-long), 500):
        batch = result1[j:j + 500]
        batch.insert(0, i)
        table.append(batch)
    res_list.append(table)

def osrm(lat_long):
      result2 = []
      result1 = []
      for value in lat_long:
          data = ";".join(value)
          url = "{}{}?sources=0&annotations=distance".format("http://server-ip/table/v1/driving/", data)
          response = requests.get(url)
          res1 = response.json()
          result1.append(res1['distances'][0])
       res2= []
      for j in result1:
          res2.extend(j[1:])
      result2.append(d2)

I need more clarification on the code side using python. Can you share me the code for achieving the above issue? I am using the below URL to get the durations and distances.

url = "{}{}?sources=0&annotations=distance".format("http://server-ip/table/v1/driving/", latitude_longitude)
I am sending 500 lat-long in batch-wise for 3k latitude-longitude I am getting response in 20min.

Kindly suggest that what I am doing wrong in code.

[danpat]

Years ago, I wrote this in Javascript: https://gist.github.com/danpat/83eaed80b6241310ebdf1c0bef04a524

The Mapbox Matrix API has the same API interface as OSRM, so you can adapt that.