Fast Parsing HTTP User Agents with .NET

While analyzing nearly 120 million requests, we noticed that one thing was dragging down our performance: parsing user agents for detecting bots and search engines.

For this task, we used the UA Parser library, which is quite widely used. Unfortunately, it is quite outdated, generates a lot of allocations and lowers our performance in its implementation.

|              Method |        Mean |     Error |    StdDev |   Gen 0 |  Gen 1 | Gen 2 | Allocated |
|-------------------- |-------------|-----------|-----------|---------|--------|-------|-----------|
|         'UA Parser' | 3,238.59 us | 27.435 us | 25.663 us |  7.8125 |      - |     - |  168225 B |

Since the alternatives are not really better, we decided to build our own parser: mycsharp/HttpUserAgentParser

During implementation, we paid special attention to the fact that this component could be part of a hot path of the application, and therefore invested a lot in optimized performance. And our implementation lives up to this expectation in comparison!

|              Method |        Mean |     Error |    StdDev |   Gen 0 |  Gen 1 | Gen 2 | Allocated |
|-------------------- |-------------|-----------|-----------|---------|--------|-------|-----------|
|         'UA Parser' | 3,238.59 us | 27.435 us | 25.663 us |  7.8125 |      - |     - |  168225 B |
|    UserAgentService |   391.11 us |  5.126 us |  4.795 us | 35.1563 | 3.4180 |     - |  589664 B |
| HttpUserAgentParser |    67.07 us |  0.740 us |  0.693 us |       - |      - |     - |     848 B |

As can be seen, we generate minimal allocations, but as the only implementation no GenX load at all!

Besides the implementation itself, we also created a caching component that checks respective browser agent strings for equality and thus very widely used agents only need to be parsed once.

We also provide a Dependency Injection registration for ASP.NET Core directly.

• HttpUserAgentParser on GitHub
• 
• 
•