Minimalism, Linux, Self-Hosting, .NET, Go, Python, Philosophy, Psychology, Privacy, Security.
nginx mirroring
nginx mirroring
So you might want to implement some basic realtime functionality on your website. Think of the Hit Counter for example, or maybe you want to do some realtime logging. What if you wanted to do something with requests but you don’t want to use Javascript? What if you on’t want to rewrite your site with some kind of dynamic backend? You can use Nginx to do that!
When I deployed OpenFaaS (Functions as a Service), something identical to AWS’s Lambda, I wanted to write a small function that would log the requests to my site. The easiest way would be to have a small <script> tag in the <head> of my site, but I didn’t want to do that. I like the idea of being free of Javascript, and I don’t want to have to rewrite my site to use a dynamic backend.
The basic idea is to multicast the http requests to a function so the flow looks something like this:
┌──────────┐
┌───►│ Function │
| └──────────┘
┌────────┐ ┌───────┐ ┌────────┤
│ Client ├────| Nginx ├───| MIRROR │
└────────┘ └───────┘ └────────┤
│ ┌─────────┐
└───►│ Content │
└─────────┘
Which can be implemented like this:
server {
listen 80;
listen 443 ssl http2;
server_name her.st;
# .. cert stuff omitted
location / {
mirror /log_function; # mirror requests to /log_function
mirror_request_body off; # if you want to mirror the payload change this to 'on'
root /srv/http/her.st; # your content root
}
location /log_function {
internal; # don't allow public access
proxy_pass https://f.her.st/log_function; # your function
}
}
Now we need to write a function that will log the requests. I wrote a simple function in C# that will log the requests to a Redis database.
The Function
First we have to create a new openfaas function. I’m using the csharp-httprequest template
faas-cli template store pull csharp-httprequest
faas-cli new nginxlog --lang csharp-httprequest --gateway https://f.her.st
this will create a new folder called log_function with the following files:
├── log_function
│ ├── log_function.csproj
│ └── log_function.cs
└── log_function.yml
For more information about the function template, see the openfaas docs.
Inside log_function.cs you’ll see something like
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
namespace Function
{
public class FunctionHandler
{
public async Task<(int, string)> Handle(HttpRequest request)
{
var reader = new StreamReader(request.Body);
var input = await reader.ReadToEndAsync();
return (200, $"Hello! Your input was {input}");
}
}
}
Now we need to add StackExchange.Redis to the project.
dotnet add package StackExchange.Redis
Now we can add the code to log the requests to Redis.
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using StackExchange.Redis;
namespace Function
{
public class FunctionHandler
{
public const int FunctionId = 1;
public const string RedisHost = "10.1.0.105:6379";
public async Task<(int, string)> Handle(HttpRequest request)
{
var connectTask = ConnectionMultiplexer.ConnectAsync(RedisHost);
var funcInput = new FunctionInput
(
request.Path,
request.Headers
);
var json = System.Text.Json.JsonSerializer.Serialize(funcInput);
var redis = await connectTask;
var db = redis.GetDatabase(FunctionId);
var incrTask = db.StringIncrementAsync("requests:count");
var addTask = db.SetAddAsync("requests", json);
await Task.WhenAll(incrTask, addTask);
return (200, json);
}
}
public record FunctionInput(string Url, IHeaderDictionary Headers);
}
Now we can build the function and deploy it to our OpenFaaS instance
faas-cli build -f log_function.yml
faas-cli push -f log_function.yml
faas-cli deploy -f log_function.yml
The Result
Let’s send a request to our function and see what happens.
curl https://f.her.st/nginxlog | jq
{
"Url": "/",
"Headers": {
"Accept": [
"*/*"
],
"Accept-Encoding": [
"gzip"
],
"Host": [
"10.62.0.37:8080"
],
"User-Agent": [
"curl/7.86.0"
],
"X-Call-Id": [
"2fea566b-2684-46d7-9af3-98d3b57b158d"
],
"X-Forwarded-For": [
"42.0.69.69"
],
"X-Forwarded-Host": [
"f.her.st"
],
"X-Forwarded-Proto": [
"https"
],
"X-Forwarded-Scheme": [
"https"
],
"X-Real-Ip": [
"42.0.69.69"
],
"X-Start-Time": [
"1670663513325430714"
]
}
}
now let’s send a request to the blog
curl https://her.st/
Now let’s check the Redis database to see if the request was logged.
$ redis-cli -n 1 GET requests:count
"2"
Works like a charm!
