New project time!

Well its not really a new project. I’ve been thinking about this for a while. I want to use Mozilla’s TensorFlow implementation of Baidu’s DeepSpeech architecture combined with a TensorFlow based classification system to map police/EMS dispatch events on google maps. The end result will be a map from the last 24 hours for  the county with pins in every location that police/EMS were called to with each pin having the transcribed summary of the mapped call.

The police/EMS scanner data will stream in real time from broadcastify.com who provides this service for free for non commercial purposes.

The trickiest part of this project will be training a TF model to recognize garbled radio traffic. In some ways this will be easier than having a model that can understand ALL speech. Dispatches use a standard protocol to make information easier to understand over noisy radio transmissions. A specific example of how data will be tagged is the NATO phonetic alphabet. A dispatcher is very unlikely to talk about an Echo or an Xray, so transcriptions used for training will train the model to transcribe “ECHO” as “E”.

Amazon’s cloud based transcription service will be the starting point for creating training data. Mozilla’s Common Voice datasets are made up of sentence long audio clips of human speech paired with transcriptions of each clip. It would be an unreasonable amount of manual work to transcribe hours and hours of scanner traffic so splitting up several days worth of scanner archives and feeding them into Amazon Transcribe may be a a good starting point for training data. Even if I have to do some manual QA, its going to save a lot of time.

Lets see what we get out of Amazon transcribe from this audio clip:

Twenty Alfa. Lots of yelling and screaming Clayton have, according the noise, it can’t be contacted. This needed silver of Fifty Five Alfa returns to Peter Howard weren’t as license native. One Four Forty Five Twenty Alfa the number of Five, Seven One, Four, Three Eight Nine Five Nine zero Forty Five Alfa Know which felt special man was like Five Alfa average welfare. One, Two Six, Seven Five. Taylor’s Town Road and be Five Twenty At the request of Pittsburgh P. D in Berkeley County, West Virginia. There, agency had reported to a domestic and an address in Martinsburg upon arrivals. There were signs of a struggle inside. Men forced entry into the residence. Officers did not locate anyone inside, but did see multiple signs of a struggle. The female victim in her vehicle are not at the residence. They were able to paint in the victim’s cell phone, which is dropping within a mile of the severance. Seen it for further double. Zero. Fifty Seven Four Three Forty Five. You can put Three 15 Alfa and Service and put me on Three Forty hours. Call him right down the street from him. Three 15 Outside, you can go in service. Three. 15 of us Were you dressed? Traffic down under Fifty. Enough. A little Zulu Fifty Burn. Always complete. Seven. Onside itself conquered Port Three Two Seventy. Music for several hours. Seven. Fifty Nine. Well, the son of the protective order, right Would place Fifty Nine Three. Forty Alfa Direct Three. Forty Five Alfa. You can go into service, right Three Forty Five out through 100 Two. Forty Reckoned Eight Seventy Two. Seventy He called himself Alfa Two Sixty Five Alfa for suspicious vehicle Two One Two Five Bureau hunting and square Be Two Eighty, Uh, but also Two Sixty Five further on your cold. No one is in the car in the parking lot and lot of five minutes Is a blue van or truck because it’s to keep on the hunt and is open in the back. There’s a black wrap I don’t Inside the open area that appears to be a body. You’re One 11 Three. I’m happy. Verbal altercations for you. Response to the area of Eight Oh, Nine beliefs of square northeast color of eyes in here, eh? Young woman and a male believes the immediate arguing back and forth cannot see them can only hear them from out Five Three. Here’s the second everything. Four. Twenty Oscar. He really from the call that I was put on, the One give her everything One Fifty.

I did a spot check for locations in the transcript that were mostly correct or totally correct which I have bolded. Amazon gives us the option to use what they call custom vocabularies which are either lists or tables of words that are likely to be in a transcript that makes transcription more accurate. As a test, before starting the transcription job for the audio file above, I created a custom vocabulary with the following words:

Alfa,Bravo,Charlie,Delta,Echo,Foxtrot,Golf,Hotel,India,Juliett,Kilo,Lima,Mike,November,Oscar,Papa,Quebec,Romeo,Sierra,Tango,Uniform,Victor,Whiskey,X-ray,Yankee,Zulu,One,Two,Three,Four,Five,Six,Seven,Eight,Nine,Twenty,Thiry,Forty,Fifty,Sixty,Seventy,Eighty,Ninety

20 hours. Lots of yelling and screaming Clayton have, according the noise. It can’t be contacted. This needed silver of Disney because all the returns to Peter Howard weren’t as license. Negative. One 4 45 20. The number of 571 43 9590 Prefer to talk about it? No one felt special Man was, like five out of welfare. 12675 Taylor’s Town Road and be 5 20 at the request of Pittsburgh P. D in Berkeley County, West Virginia. There, agency had reported to a domestic and an address in Martinsburg upon arrivals. There were signs of a struggle inside. Men forced entry into the residence. Officers did not locate anyone inside, but did see multiple signs of a struggle. The female victim in her vehicle are not at the residence. They were able to paint in the victim’s cell phone, which is dropping within a mile of the severance. Seen it for further double drove 57. Unit 3 45 You can put 3 15 outfit and service and put me on 3 40 hours. Call him right down the street from him. Get it 3 15 Outside you can go in service. 3 15 of us. Were you dressed? Traffic down Under is enough. A little busy burn. Always complete. Put it on. Site himself conquered port to 70 music for several hours. Similar 59. Well, the son of the protective order right would place with 59 3 40 Directory. 45 Alfa You can go into service right? 3 45,100 reported. Reckon David 72. 70. He called himself Salva to 65 Alfa for suspicious vehicle to 1 to 5 euro hunting and square beat to 80 uh, but also to 65 Further on your cold. No one is in the car in the parking lot, and lot of five minutes is a blue van or truck because it’s to keep on the hunt and is open in the back. There’s a black wrap I don’t inside the open area that appears to be a body. You’re 1 11 I’m happy. Verbal altercations for you. Response to the area of eight or nine Police of square northeast color of eyes in here, eh? Young woman and a male believes the immediate arguing back and forth cannot see them can only hear them from outside country. Here’s the second everything. You two for 20? He really from the call that I was put the other one. Give her everything. Yeah.

• Police cruisers are identified by three numbers followed by a letter.
• The dispatcher calls out the current time in a 24 hour format as appropriate
• In service means “ready to take a call”
• “Direct” means “copy”
• “Welfare” means the police need to check if someone is safe

So here it is in a more human readable format:

Dispatcher: 345A Welfare 12675 Taylor’sTown Road and be 5-20 at the request of Martinsburg P. D in Berkeley County, West Virginia. There, agency had reported to a domestic and an address in Martinsburg upon arrivals. There were signs of a struggle inside. and forced entry into the residence. Officers did not locate anyone inside, but did see multiple signs of a struggle. The female victim in her vehicle are not at the residence. They were able to ping in the victim’s cell phone, which is dropping within a mile of this address. Seen notes for further 00:57

345A: 345A

DIspatcher: Unit 345A?

345A: You can put 315A in service and put me on 340A’s Call I’m right down the street from him.

Dispatcher: Unit 315A you can go in service. 315A, Were you Direct?

315A: 315A I’m direct 

Dispatcher: 315A 00:58

Here’s how the data would look on a map:

When inspected:

Further work needs to be done to refine the custom vocabulary that Amazon transcription uses to create good training data. For cost reasons I want my own Tensorflow model but having well tagged data from Amazon is a first step. Once I have a working model I can tackle other challenges like training Tensorflow to classify events like I did manually on the section of audio above to make it easier to map and qualify.