FMCSA

Software Architect, .NET Developer (2023)

Overview

The United States Department of Transportation (www.dot.gov/about) includes the FMCSA (Federal Motor Carrier Safety Administration), which regularly updates 2 files: the registered device list and the revoked device list. Both lists are similar by structure, but have some differences (and, of course, list are consisted different devices). These lists can be accessed via URL through a simple GET request.

Goals

Retrieve device lists from FMCSA after each update, parse them considering all described nuances; reorder local lists if necessary and save them. Subsequently via web part or via web API we should showcase list of devices in correct order, with filtering capabilities and a history of changes (showing additions, reorderings, etc., in each new file). Additionally, provide the ability to locally extend information in our database.

Idiosyncrasies

As is typical for any government agency, they update it not always accurately :) They may mix up records (and typically correct them later, in the next file)

They can update files at any time

Tasks that I, as a developer and project architect, have faced:

Because they may mix up records (and typically correct them later in subsequent files), we need to identify devices that are the same as those in previous versions of the file (list)

There is no way to identify a device by a specific field alone, we have to analyze and compare a set of fields and consider the device's position in previous lists

We need to correctly identify when a device has been modified and reordered, ensuring it is recognized as the same device rather than mistakenly treated as one deleted and another added

Implementation

Technologies:

.NET Core
Azure SQL
Azure Functions
Entity Framework Core
ASP.NET Core & Web API
KendoUI, jQuery

The main efforts in implementing this project were focused on identifying devices. Since there are no concrete rules for identification, we had to analyze the entire new set, compare it with the previously downloaded data, and determine whether the device is new or possibly modified and/or reordered.

Because the two types of files are similar across the entire solution, abstract classes are employed for each responsibility, providing foundational functionality that are inherited by specific classes.

The device ordering is implemented as a singly linked list (each device stores the ID of its predecessor), allowing for easy and quick reordering when necessary (simply by changing predecessor references).

Responsibilities

Design Project Architecture: Develop a scalable and reliable architecture for retrieving, parsing, and managing FMCSA device lists, ensuring integration with both backend and frontend components

Backend Development: Lead the backend development to handle the retrieval of device lists from FMCSA, including parsing the lists, updating local data, and ensuring that the system correctly reflects changes

Frontend Development: Create and maintain the frontend interface using Kendo UI, including active use of Kendo Grid and Charts to showcase the device lists in the correct order, with filtering capabilities, and a history of changes, including additions and reorderings

Code Quality: Produce clean, efficient, and maintainable code that adheres to project specifications and best practices

Testing and Debugging: Conduct thorough testing, debugging, and troubleshooting to ensure the backend and frontend components work correctly and efficiently

Data Management: Implement solutions for local data storage, allowing for the extension of information in the database and managing updates to the device lists

Change History: Develop and maintain functionality to track and display changes to the device lists, including additions, reorderings, and other modifications

Web API: Implement a Web API for all features of the application, supporting RESTful principles

Documentation: Create and maintain documentation for backend components, including architecture, code, UML diagrams, and Swagger for Web API documentation