REST API with Quarkus, Logstash and MongoDB.

This api was developed for the final work of the subject Databases 2 of the Informatics Faculty - UNLP. This api receives a set of several million traffic accidents and formats them using logstash to convert the coordinates into geoJSON format, then writes it to a MongoDB database, for later analysis and consultation through polygons, radii or analyzing conditions more common in which these occur. Remember that you have a sample database along with a conf file for logstash in the sampledb folder of this repository. Logstash detects files in a directory and will take care of formatting them and send to mongo database.

This project uses Quarkus, the Supersonic Subatomic Java Framework.

If you want to learn more about Quarkus, please visit its website: https://quarkus.io/

git clone https://github.com/nexun/quarkus-mongodb-restapi.git

https://www.elastic.co/es/downloads/logstash

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-mongodb.html

bin/logstash-plugin install logstash-output-mongodb

Important: Set Input path and Output format, also specify your Mongo database

input {
    file {
         path => "C:\AccidentsFileHere.csv"
         start_position => beginning
    }
}

filter{
    csv{
        columns => ["ID","Source","TMC","Severity","Start_Time","End_Time","Start_Lat","Start_Lng","End_Lat","End_Lng","Distance(mi)","Description","Number","Street","Side","City","County","State","Zipcode","Country","Timezone","Airport_Code","Weather_Timestamp","Temperature(F)","Wind_Chill(F)","Humidity(%)","Pressure(in)","Visibility(mi)","Wind_Direction","Wind_Speed(mph)","Precipitation(in)","Weather_Condition","Amenity","Bump","Crossing","Give_Way","Junction","No_Exit","Railway","Roundabout","Station","Stop","Traffic_Calming","Traffic_Signal","Turning_Loop","Sunrise_Sunset","Civil_Twilight","Nautical_Twilight","Astronomical_Twilight"]
        
               }
                mutate{
                    add_field => [ "[geometry]", "%{Start_Lng}" ]
                    add_field => [ "[geometry]", "%{Start_Lat}" ]
                    remove_field => ["message","path","host","Start_Time","End_Time","Start_Lat","Start_Lng","End_Lat","End_Lng"]
                    }
                mutate{
                    convert => [ "[geometry]", "float" ] 
                }    
    
    }

output{
    mongodb{
         uri => "mongodb://127.0.0.1:27017" //Your database here.
         database => "converts"
         collection => "accidents"
    }
}

logstash -f pipeline.conf

You can run your application in dev mode that enables live coding using:

./mvnw compile quarkus:dev

You have 4 endpoints

1)List all accidents

/accidents/{start}-{end}
/accidents/1-100

In this path you should specify GET method, the starting and the end value, the returning values are the accidents contained in the interval.

2)Accidents inside a polygon

/accidents/polygon
[
   {
      "long":-122.66328563750216,
      "lat":38.332236587402136
   },
   {
      "long":-122.36538569651253,
      "lat":37.95741805152669
   },
   {
      "long":-121.9878627298206,
      "lat":38.35485829578574
   },
   {
      "long":-122.66328563750216,
      "lat":38.332236587402136
   }
]
Code snippet to test the endpoint

In this path, you should specify POST method and send the body in Json format with the coordinates and it will return the accidents within.

3)Accidents inside radius

/accidents/radio
[
   {
      "long":-82.3508587993436,
      "lat":34.6372515048226,
      "distanceInKm":100
   }
]
Code snippet to test the endpoint

In this path, you should specify POST method and send the body in Json format with the coordinates of a point and radius, it will return the accidents within.

4)Accidents Statistics

/accidents/analyze/{atribute}/{start}-{end}

/accidents/analyze/Airport_Code/100000-300000
Code snippet to test the endpoint

In this path, you should specify POST method and send the atribute which you would like to get statistics, with the the range of elements to check, it will return statistics of the atribute.

• Civil_Twilight
• Zipcode
• Crossing
• Traffic_Calming
• Bump
• Junction
• Astronomical_Twilight
• Description
• Visibility(mi)
• Country
• Source
• Traffic_Signal
• Street
• Wind_Speed(mph)
• Number
• Amenity
• Distance(mi)
• Railway
• Give_Way
• Airport_Code
• Stop
• City
• Severity
• No_Exit
• County
• Turning_Loop
• Roundabout
• Precipitation(in)
• Pressure(in)
• Side
• Wind_Direction
• Station
• Nautical_Twilight
• Sunrise_Sunset
• Wind_Chill(F)
• TMC
• Timezone
• Temperature(F)
• Weather_Condition
• Humidity(%)
• State