her.st - Do you see it yet?

symbolism, meditation, prayer, psychedelics

2024-04-05T00:00:00+00:00

Jung’s Mist

There’s a time in your life - if you can think back far enough - where you did not know that you were separate from everything else. When you did not discriminate between you and an ant crawling through the dirt. Everyhting is One and One is All. Carl Jung called the moment of realization the ‘stepping out of the mist’.

“[..] That was in my eleventh year. There I suddenly was on my way to school I stepped out of a mist. It was just as if I had been in a mist, walking in a mist, and I stepped out of it and I knew, “I am.” “I am what I am.” And then I thought, “But what have I been before?” And then I found that I had been in a mist, not knowing how to differentiate my self from things. I was just one thing among other things.” – C. G. Jung

Once you ‘step out of the mist’, you become like god. And you - like him - are alone.

Adam and the Apple

The word fruit appears in Hebrew as פֶּ֫רִי (pərî). As to which fruit may have been the forbidden fruit of the Garden of Eden, possibilities include apple, grape, pomegranate, fig, carob, etrog or citron, pear, quince, and mushrooms.

Adam was not conscious before eating the forbidden fruit. He had awareness but He was not conscious.

“And the serpent said unto the woman, Ye shall not surely die: For God doth know that in the day ye eat thereof, then your eyes shall be opened, and ye shall be as gods, knowing good and evil.”

Good and evil have no objective existance, you can only be taught what good and evil are. Without betraying god, loyalty to god was meaningless. Eating the forbidden fruit gave him the experience of evil and provided the contrast to good.

Yin Yang

This is usually oversimplified as “the bad contains a bit of good” and “the good contains some bad”. This hides the truth. If you deny one side, you never experience the other side to its fullest. You lack the contrast. This is conveniently left out from the movies.

“One quickly learns that the Jedi Code does not give all the answers. If you are to truly understand, then you will need the contrast, not adherence to a single ideal.” – Kreia

There is no virtue in a harmless persion for not harming.

Are you a nice person who could never hurt anyone? Are you unable to fight? Are you… harmless? Harmless people are not good. If you can’t harmm there is no virtue in not harming.

“[..] fear and cowardice, dressed up as morality.” – Nietzsche

If you are not capable of cruelty, you are the victim to someone who is. You have to be capable of it and then learn to control it. How else are you supposed to respect yourself and demand to be respected?

Weak people are dangerous because they deny their potential for it.

“if you are going to be a figher, you have to want to win and you have to want to hurt people. But not for the sake of hurting them, thats what makes you different than an evil person. But you have to have that capacity. You have to develop that. And you know, thats a step to the way of enlightenment, weirdly enough, because that isn’t what people think.” – J. B. Peterson

In Kung Fu Training you will learn how to break the arm, ribcage and nose of an attacker in a single fluid motion on your first day of training and practice it on your peers and your peers on you. Injuries during training are common. You might just get a bit too carried away or not have paid sufficient attention and injure or get injured. You learn quickly how much harm you are capable of inflicting unintentionally, and it will dawn on you how much more you could do intentionally.

With time you become very dangerous, but at the same time, you learn discipline and mental strength. You learn how to be focused, and confident. When somoene tries to provoke a fight, you can respond with confidence to make the aggressor rethink his choice - and if he doesn’t, he will regret it.

In the end, learning to fight is the best way not to get into fights in the future. Strong people know how dangerous they are and learn how control it.

The Shadow

You have to understand that you are no saint, in fact you are much closer to being satan himself. Jung said your shadow reaches right down into the depths of hell. Given the right circumstances, you’d be exactly like Hitler, Stalin, Mao, … And if you don’t know that, you know nothing about people or yourself.

Learning that about yourself is not fun. That’s why you didn’t do it. You have to be willing to descend into hell to learn things like that. If you want to find wisdom, the answer is behind the door you do not want to go through at all. That’s where you kept it all this time.

Integrating the shadow means to step out of the moral framework you have built around you. All the things you denied yourself to feel or experience. Whatever you supress has to be brought into your life and integrated into it.

By denying those things, you also deny feeling the opposites to their full extent.

Spiritual Disciplines

They work. But not how they are practiced today. They also don’t require supernatural beings or explanations. They work even if you do not believe in them. I’ll tell you how.

Prayers

Especially in the western/christian world, prayer has been bastardized into begging “God” for handouts. People think praying for strength will magically make them stronger. The Clergy will tel you that if you pray for strenght, god will put you into situations that will give you a chance to practice strength.

Cute, but doesnt work like that.

Prayer was taken alot more serious in the past. Why do you think prayer is still a thing today? Do you think the people who started your religion were praying like you and had the same answer-ratio that you got? No. Their prayer worked 9 times out of 10. Even though there is no god.

You see, 10min of prayer is not enough. You want to pray for a prolongued amount of time. You will know when you are doing it long enough and right. Repeat the prayer over and over and over…

Mantra

Mantras are phrases or sounds you repeat at a certain rythm, almost like a song, over extended periods of time. The human brain is quite bad at multitasking, so, by repeating words or sounds, your brains language region is occupied and can’t function in autopilot. That basically means, you deny your brain to create and think thoughts for extended periods of time. Your brain does not like that. It will try its best to make you stop. First you will get bored, then you will slur the words or sounds and break the rythm. Those little errors will kick your mind back into autopilot and you will have chatter in your skull. If you persist, eventually you will start to hallucinate as your brains visual region takes over the task of thinking thoughts.

Meditation

People do it completely backwards. Meditation is not about saying “Om” and focusing your attention on your breath or heartbeat. Its not about controlling your mind. You are not supposed to stop thinking. What you want to do is to let the thoughts flow freely and stop passing judgement on them. The goal is observe them come and go and to practice mindfulness. You should be able to be consciously aware of thoughts before they are completely ‘prepared’ and your inner voice starts to speak them to you. You should observe and take mental notes on what sensations you feel in your body when you experience certain thoughts. You do this until you dissociate from your ego and watch your thoughts like an external observer.

Fasting

Once again, fasting has been washed down and turned on easy-mode for todays population. Intermittend fasting as is now being practiced does bring some health benefits, but it will not get you closer to “god” or “enlightenment”. You want to starve yourself for a week or two. Some monks sat in a temple and meditated without food until they died sitting. Some were even turned into statues and preserved forever. Don’t you think they’d have gotten up if nothing happened to them in their mind as they were approaching death? The closer you get to death, the more you will learn. Clearly one of the worst ways to affect your mind. Not very healthy. But there’s always a time and place for everything.

Sacrifice

Imagine standing in front of an altar in a dark, candle lit underground room. Surrounded by other people in robes, chanting prayers. On the altar, there is a young woman, naked, blindfolded, breathing slowly and deeply. The ritual master slowly raising a dagger above his head as the people chant louder and louder, until he brings the dagger down with force, stabbing the woman in the chest and her screams hall through the dungeon as she fights a losing battle for her life.

Just readng this should have made you feel something. Imagine experiencing something like that for real. Do you understand? Traumatic experiences alter your state of concsiousness. I think I’d rather do a few weeks of fasting.

Psychedelics

The fastest way. Most religions originally had their special potion. Clearly, a psychedelic potion. If I were to start a religon, I’d ‘prove’ it right by giving my acolytes a potion laced with psilocybe mushrooms and set up an appointment with god or satan for them.

Ego Death is is the key. Given a high enough dose of psilocybin for example, you will be convinced that you are dying. No matter how rational you think you are, even if you remind yourself that you are just on psychedelics and everything is fine, you will still believe that you have done it this time and taken too much. For all you know, you are literally dying. But your body survives.

“If you die before you die, then you won’t die when you die.” – Inscription over a door at St. Paul’s Monastery on Mt. Athos

The most astounding property of psilocybin and possibly other psychedelic agents - however, I only know this about psilocybin for a fact - is the ability to re-mount your subconscious from readonly to read/write.

You can put a spotlight on any automatic behavior, any impulse response, and then just “will” it into a new behavior. It’s as easy as saying “this behavior no longer serves me, i will not do this anymore” and the next day you wake up a changed man.

For reference: I’ve spent ~9 years in PTSD therapy, stationary for significant amounts of time, been on all kinds of drugs which get prescribed randomly and switched for others in the same random manner, countless hours of therapy sessions and talking about everything. This yielded zero improvements. One heavy dose of psilocybin with the conscious goal of improving my life and 8 hours later, I was virtually symptom free and started functioning. But not only that. I did a career speedrun the likes of which nobody would believe.

You will never be able to talk me out of advocating psychedelics. They are a great tool and they have saved my life.

Zen and Desire

The root teaching of Buddhism is the following: You suffer because you desire

As soon as you start practicing zen, your spiritual teacher will use mindgames on you to break through your mental conditioning. He will guide you towards enlightenment like a prankster. First he will make you give up all your possessions - often by donating it to him. Then he will start giving you more and more chores and excercises until you are completely overwhlemed and have a mental breakdown. Here’s an example:

Student: I am suffering.
Master: You suffer because you desire.
Student: What shall I do?
Master: Try not to desire.
Student: I have tried not to desire for many months now, I still find myself to desire
Master: Try not to desire less than you can.

[often never or after a long time]

Student: I now desire not to desire

The more you desire enlightenment, the longer it will take you to accept your inevitable and unavoidable defeat. You will be trapped in a loop, a catch-22 - like the Oroburos eating its own tail. Eventually you will have a breakdown and accept your inability to do what is asked from you, and during this painful emotional resignation, you may truly understand:

If you seek it, you can not find it.
If you dont want it, you have it.

The Buddha himself sat in the mountains under a tree for weeks trying to achieve enlightenment, and only at the last moment when he accepted defeat and got up, it suddenly happened to him.

Englightenment

The experience of enlightenment is indescribable and incomprehensible until you get a taste first hand.

You experience three ‘people’ inside you.
- one that is your identity, the ego
- one that creates thought, the thinker
- one that processes thought, the prover
The thinker finally shuts up
- no more chatter in the skull, no compulsive thinking
- silence in your mind is the new normal mode
A feeling of peace at an unimaginable magnitude
- calmness, serenity, hearing birds chirp and truly appreciating it
- no more anger, more patience
- seeing through peope playing out the act that is their ego
- no worries, no anxiety, just acceptance
- understanding why things are the way they are
A boost of Confidence, you no longer identify with your identity
- your identity is put in the background
- you stop caring about status, fame or power
- you start to see whats really important
- public speaking is suddenly easy
A feeling of connectedness with everything
- feeling reborn
- no longer fear death
- highly enhanced sense of empathy

Those effects last forever as far as I can tell and are the new default.

You will still get sad, worried, angry, start obsessively thinking thoughts and get worked up by all of it - but as soon as you notice, you can just stop.

You will find a kind of enjoyment in negative emotions. YOu can face your fears, like the fear of heights by sitting at the edge of a cliff in absolute terror, then take a step back - in your mind - and observe your thoughts, all the sensations you feel in your body and take mental notes as if you were an observer. This works for everything.

You can speak your mind without fear of being ridiculed, you no longer worry about what people think about you. That’s why you do not lie. You don’t work for money, you just work for knowledge. You get a minimalist life and home without much stuff because you don’t need many things and dont desire things you dont need.

You go to bed and your mind is no longer reminding you of all the fuck-ups you have (or surely will) commit in the future. You’re fine with fucking up every now and then. Nightmares disappear.

Edit: After the death of my Dog and weeks of grief and sadness on a level I’ve not experienced before, not even for a human, I am still standing by what I’ve said above. I was able to embrace the grief and sadness but also put it aside when I had to.

R.I.P Inu.

– Nov 28th 2020 - Oct 21st 2024

footnotes

‘the thinker’ & ‘the prover’ borrowed from “Prometheus Rising” by Robert Anton Wilson

freedom

2024-04-05T00:00:00+00:00

rule of thumb

The majority is not very intelligent -> The majoriy does certain things -> Doing those things is not very intelligent.

obvious examples

Stupid Majority	Intelligent Minority
they want more money	they want to need less of it
they go into debt for liabilities	they go into debt for assets
they need full time jobs	they work for fun
they have university degrees	they have real skills
they believe others	they question themselves
they do what authority commands	they do, given reason to
they are on social media	they are not
they watch mainstream media	they do not
they parrot what they are being told	they have original thoughts
they vote	they abstain

The majority of people will even stop at a red pedestrian light and not cross an empty road until the light turns green. They are barely autonomous, and just like sheep, as soon as you walk across the red light, they will follow behind you.

why not to vote

Elections no longer matter (if they ever did is another subject entirely) but all over europe right wing parties win 2024 and are not allowed to rule.

how to be free

First of all you need to understand how and why you were programmed.

You were programmed to want money. Everyone has a natural tendency to be greedy. Whatever you have will become boring and you will want more. You want that new iPhone every year because its better, or a new computer or a bigger house, a better fucking fridge, whatever you have, eventually something better will catch your eye and you will want it. For -some- reason. That’s how humanity made it this far. Survivial is enough for animals and natural selection, but not for us. We want to perpetually improve things and make them better. Capitalism is an ingenius system to encourage that behavior and steers whole societies into that direction by exploiting this innate desire.

Obviously, businesses exploit that in ways more insidious than just making better products. They use all kinds of dirty tricks to keep you consooming. From psyops to planned obsolencence. They will get you to consume one way or the other. After all, they are in the business of making money. And you are working 40h/week to spend it with them.

pressure

Everyone wants you to “get a job” and not be “lazy” or a “freeloader”. Usually those people are the most vocal supporters of politicians who do no useful work whatsoever.

Being a politician and being on welfare is almost exactly the same thing: both are paid by the tax payer, but only one of them lies to us on TV. Get what job exactly? During the Covid19 Pandemic most people lost their jobs because they were deemed non-essential and rightfully so. Most jobs are not required to be done, they just exist to keep the system going.

Small businesses are obsolete, the megacorps have taken over. You can get everything delivered from amazon, your groceries get delivered by the big supermarket chains, you can order everything that’s essential from some big company. It doesn’t matter what your opinion on those big corporations is, they provide convenience no small business can provide and the majority of people want that convenience.

They won. And the sooner people accept that, the sooner they can stop wasting their time and energy doing useless jobs that are reminiscent of occupational therapy, just keeping people busy so they don’t think about how fucked up the state of affairs is.

Now, you could get a non-essential job, like software development. What are you going to build? Most likely something a marketing agency convinced enough people they need and not something useful. If you’re fine wasting your life on something like that be my guest.

You see, at some point you realize that there’s no real demand for any low-tech entrepreneurship anymore. The companies that do those things nowdays have been founded 100 years ago and live off of government contracts, own or still rent their stores and workshops at pre-war prices. If they had to rent or pay off a mortgage nowdays, they’d simply not be able to stay open.

And if you’re not in the top ~5% of the IQ distribution, you have no chance of competing in any high-tech field. So you’re stuck with the low-tech stuff, and there’s no demand for that.

breaking the loop

Step 1: Cut all your costs

Buy food in bulk. Buy everything you can 2nd-hand. Do not rely on any services like plumbers or hairdressers, instead learn how to do that stuff yourself. Pick up a new hobby like soldering or woodworking, get a toolbox. Become a man that will just solve the task at hand without relying on a better man.

If you want to invest in something and you take a loan or credit, you are not investing, you are making the bank invest and the bank will make you work for decades to pay it back. If you currently have debt, pay it off as soon as possible. You can not be free if you are in debt. You are a slave to the bank. Do not go into debt before you actually know the difference between an asset and a liability.

If you think a house you live in is an asset you do not understand the difference.

Cancel all subscriptions. Cancel all contracts. Get a cheaper phone plan, get a cheaper internet plan. You don’t need cable or Netflix. Do not pay for anything you can get for free. Learn how to get things for free. Before you make any purchase that isn’t necessary to survive, wait 14 days. After 14 days you will realize that you don’t need it. If you still want it, buy it. From my experience, you won’t buy most things because you forget you wanted them.

Step 2: move to a cheap place

Get a cheaper apartment. Embrace complete minimalism. Get a Bed, Chair, Table and rice cooker. That’s all you need to survive, you’d be surprised how versatile a rice cooker is. You don’t even need a fridge. Turns out half of the year its cold enough outside to just put stuff outside your windowbed to refrigerate it. Don’t bring all of your old stuff, sell it all. You don’t need it.

Steo 3: quit wage slavery

At this point your cost of living should be so low, even government welfare is enough to sustain you. You can now quit your job and live off of welfare. You can now do whatever you want. You can work on your own projects and learn new skills. You are almost free. Take small jobs on demand, whatever you are qualified for, get full time jobs, quit them after a month or two, once you made enough money to do what you want to do and spend the rest of the time on learning and practicing new skills.

Avoid working for money. Work for experience or for the joy of doing, learning, creating or helping others. You never have to worry about if the job pays enough now. You can do whatever job you want to do. The money you do get, you invest into assets and/or new skills that will save you money in the long run.

Step 4: freedom

Once you’ve learned enough skills to make money on your own, or get any job you want, you can basically pack your shit whenever you want and leave. That’s freedom. You can go anywhere you want, you can do whatever you want. The job you accepted turns out to be shit? Just quit. You don’t need the money anyway. You just took the job to learn something or to get money for an investment. Just get a different job.

Book

And if you want to learn more about how the system works, I recommend reading “DIE SHEEPLE DIE” by The Honorable DJ Hives. It’s a free ebook and I’ve attached it below. It’s written in a way the stupid majority can not read. It is FNORD.

“Every appearance of the [misspelled] word subconsciously generates a feeling of uneasiness and confusion, and prevents rational consideration of the subject. This results in a perpetual low-grade state of fear in the populace. This in turn perpetuates the need for Government, because without fear, people don’t need Government.” – The Illuminatus Trilogy (Novel, 1974).

DIE SHEEPLE DIE! by The Benevolent DJ Hives

LLMs solved the search problem

2023-03-23T00:00:00+00:00

Google is dead

Google has emerged as the predominant source of information, offering answers to a wide range of queries, from trivial to scholarly. Its vast resources, including billions of web pages, articles, books, images, and videos, have replaced libraries as the go-to places for knowledge and culture. Google’s success can be attributed to its convenience, enabling users to access information 24/7 from anywhere in the world. Google’s personalization features, which tailor results to users’ preferences, behavior, and relevant factors, have also played a crucial role in its widespread adoption.

Google’s commercial nature means that it collects and analyzes users’ data to improve its algorithms and provide relevant results, making users the product, not the customer. While other companies also collect data, Google’s dominance and value lie in its data collection. Nonetheless, Google’s algorithms are not neutral or objective, and they can be influenced by censorship laws or political agendas. The quality of Google’s results can be inconsistent, depending on the sources it indexes and ranks.

Google’s dominance may be coming to an end as it has not released any new products in years, and its acquisition of other companies has not resulted in significant innovation. Its size is now limiting its ability to move quickly in a fast-changing landscape, and it risks being left behind.

The Search Problem

The search problem refers to the challenge of efficiently and accurately retrieving relevant information from vast amounts of data. This problem is particularly relevant in the context of search engines like Google and Bing, which index and search through billions of web pages.

In order to solve the search problem, search engines like Google and Bing use complex algorithms to analyze the content of web pages and determine their relevance to specific search queries. These algorithms take into account a wide range of factors, such as the frequency and location of keywords on a web page, the popularity of the page, and the credibility of the source.

Google and Bing systematically download and analyze web pages and index the content of those pages into a searchable database. Ranking involves determining the relevance and importance of each page in relation to specific search queries.

The Solution: LLMs

Storing massive amounts of text in a searchable format requires an index, similar to Google’s. However, even small-scale, self-hosted Full-Text-Search solutions like ElasticSearch require insane amounts of RAM, often in the hundreds of gigabytes, for comparatively benign datasets. Google, on the other hand, requires entire data centers to store everything.

Literally tens of thousands of servers spread across multiple data centers. To give you an idea of the scale, according to a report from Statista, as of 2021, Google’s search index contained over 130 trillion web pages, and the company processes over 3.5 billion searches per day. This level of scale requires an enormous amount of hardware resources to maintain and operate.

In contrast, Large Language Models (LLMs) like LLaMA’s 65 billion parameter model, trained on over 1 trillion tokens, require only about 10GB of disk/ram and can be queried easily with 100 tokens per second on a consumer GPU. This is an incredible level of compression and performance for searching vast amounts of information.

GPT4 by OpenAI, a model trained on over 100 times more tokens than LLaMA, yet with only marginally more parameters and hence disk/ram requirements pretty much knows everything and fits into a single server.

With LLaMA, parameter count can be trimmed without losing much accuracy. You can shave off half the size and still have the same quality for a given language, like English. Furthermore, LLMs are currently unoptimized for space/resource usage, and techniques such as trimming and quantization are improving every month. Quantization algorithms can currently quantize a 16-bit model down to 4 bits, saving three-quarters of the RAM/DISK requirements while still delivering almost identical results when queried. The potential for further optimization in this area is truly mind-blowing.

nginx mirroring

2022-12-10T00:00:00+00:00

nginx mirroring

So you might want to implement some basic realtime functionality on your website. Think of the Hit Counter for example, or maybe you want to do some realtime logging. What if you wanted to do something with requests but you don’t want to use Javascript? What if you on’t want to rewrite your site with some kind of dynamic backend? You can use Nginx to do that!

When I deployed OpenFaaS (Functions as a Service), something identical to AWS’s Lambda, I wanted to write a small function that would log the requests to my site. The easiest way would be to have a small <script> tag in the <head> of my site, but I didn’t want to do that. I like the idea of being free of Javascript, and I don’t want to have to rewrite my site to use a dynamic backend.

The basic idea is to multicast the http requests to a function so the flow looks something like this:

                                            ┌──────────┐
                                       ┌───►│ Function │
                                       |    └──────────┘
    ┌────────┐    ┌───────┐   ┌────────┤
    │ Client ├────| Nginx ├───| MIRROR │ 
    └────────┘    └───────┘   └────────┤
                                       │    ┌─────────┐
                                       └───►│ Content │
                                            └─────────┘

Which can be implemented like this:

server {
    listen 80;
    listen 443 ssl http2;
    server_name her.st;

    # .. cert stuff omitted
    
    location / {
        mirror /log_function;    # mirror requests to /log_function
        mirror_request_body off; # if you want to mirror the payload change this to 'on'
        root /srv/http/her.st;   # your content root
    }

    location /log_function {
        internal;                                 # don't allow public access
        proxy_pass https://f.her.st/log_function; # your function
    }
}

Now we need to write a function that will log the requests. I wrote a simple function in C# that will log the requests to a Redis database.

The Function

First we have to create a new openfaas function. I’m using the csharp-httprequest template

faas-cli template store pull csharp-httprequest
faas-cli new nginxlog --lang csharp-httprequest --gateway https://f.her.st

this will create a new folder called log_function with the following files:

├── log_function
│   ├── log_function.csproj
│   └── log_function.cs
└── log_function.yml

For more information about the function template, see the openfaas docs.

Inside log_function.cs you’ll see something like

using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;

namespace Function
{
    public class FunctionHandler
    {
        public async Task<(int, string)> Handle(HttpRequest request)
        {
            var reader = new StreamReader(request.Body);
            var input = await reader.ReadToEndAsync();

            return (200, $"Hello! Your input was {input}");
        }
    }
}

Now we need to add StackExchange.Redis to the project.

dotnet add package StackExchange.Redis

Now we can add the code to log the requests to Redis.

using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using StackExchange.Redis;

namespace Function
{
    public class FunctionHandler
    {
        public const int FunctionId = 1;
        public const string RedisHost = "10.1.0.105:6379";

        public async Task<(int, string)> Handle(HttpRequest request)
        {
            var connectTask = ConnectionMultiplexer.ConnectAsync(RedisHost);
            var funcInput = new FunctionInput
            (
                request.Path,
                request.Headers
            );
            var json = System.Text.Json.JsonSerializer.Serialize(funcInput);

            var redis = await connectTask;
            var db = redis.GetDatabase(FunctionId);

            var incrTask = db.StringIncrementAsync("requests:count");
            var addTask = db.SetAddAsync("requests", json);

            await Task.WhenAll(incrTask, addTask);

            return (200, json);
        }
    }
    public record FunctionInput(string Url, IHeaderDictionary Headers);
}

Now we can build the function and deploy it to our OpenFaaS instance

faas-cli build -f log_function.yml
faas-cli push -f log_function.yml
faas-cli deploy -f log_function.yml

The Result

Let’s send a request to our function and see what happens.

curl https://f.her.st/nginxlog | jq

{
  "Url": "/",
  "Headers": {
    "Accept": [
      "*/*"
    ],
    "Accept-Encoding": [
      "gzip"
    ],
    "Host": [
      "10.62.0.37:8080"
    ],
    "User-Agent": [
      "curl/7.86.0"
    ],
    "X-Call-Id": [
      "2fea566b-2684-46d7-9af3-98d3b57b158d"
    ],
    "X-Forwarded-For": [
      "42.0.69.69"
    ],
    "X-Forwarded-Host": [
      "f.her.st"
    ],
    "X-Forwarded-Proto": [
      "https"
    ],
    "X-Forwarded-Scheme": [
      "https"
    ],
    "X-Real-Ip": [
      "42.0.69.69"
    ],
    "X-Start-Time": [
      "1670663513325430714"
    ]
  }
}

now let’s send a request to the blog

curl https://her.st/

Now let’s check the Redis database to see if the request was logged.

$ redis-cli -n 1 GET requests:count
"2"

Works like a charm!

coding is dead

2022-12-01T00:00:00+00:00

AI is better than you

AIs like Github Copilot will replace the need for general purpose software developers very soon. Already Copilot is alot more useful to me than a junior. It writes entire methods from a tiny comment and usually writes decent code at that. As soon as you figure out how to talk to it, it will tell you pretty much anything you want to know. This is just the beginning. Copilot is the first AI to write code. Other AIs already create Art that wins competitions, we have Video generation AI, we have speech synthesis, even AIs that write entire blog posts on major news sites. The rate of improvement is ridiculous. All of those AIs created utter shit two years ago and today they are almost on the level of a human.

In the future, people will just descibe what software they want and the AI will write it for them. For free, in a few seconds. We won’t be able to compete with that. That future is maybe - optimistically - 5 years away.

frontend development

Webdevs will be the first to fall victim to it I predict. No matter how highly you think about yourself as a webdev, especially the frontend, its doesnt take more than a few weeks to learn the basics and more than room-temperature IQ. Already most companies outsource frontend stuff to 1$/hr indians. There’s already some AIs that can create entire websites from a few screenshots and prompts. They existed before Copilot and they will be better than you in a few years.

mobile apps

Next in line will be mobile app developers who are not much more than glorified webdevs. They will be replaced by AIs that can write apps for all platforms at the same time in a few seconds.

backend developers

REST APIs are very trivial too, especially if you use microservices or something else with a very strict architecture and flow. AI will write better SQL queries than you, too.

general purpose software developers

Unit Testing will be done by AIs too. They will write the tests for you and run them. You will just have to tell them what to test. UIs will be generated by AIs too. You will just have to tell them what to show and how to show it. They will even write the code for you. Utilities, Scripts, DevOps, everything will be done by AIs. Those things are trivial and usually passed to juniors anyway.

specialized software developers

If you’re working on something that is very specific and non trivial, like low level networking code, optimization, … you’re safe for a couple more years. But even then, AIs will be able to write code that is better than you in a few years.

copilot & others

They do not sleep. They are not limited by typing speed, they dont take breaks and they are extremely cheap to run. Yes, you might need a 100$/m subscription for some serious stuff, or run it on a server that costs you a few 1000$ a month, but that’s still orders of magnitude cheaper than a team of developers.

Hardware acceleration of AI just started a few years ago. Initially we had GPUs do the heavy lifting, but now we have FPGAs, ASICs, TPUs, … that are orders of magnitude faster than GPUs. They are also orders of magnitude cheaper and more power efficient. Supercomputer clusters have been built with AI accelerators for years now and they train AIs 24/7/365. They are already better than humans at many things. You are training AIs everytime you solve captchas, play games, use google, … and you are training them to be better than you. Well, you are classifying training data for them, but you get the point.

computer-linguists

Eventually I think people will get really good at describing things to AIs and get hired just for that. You won’t hire a programmer but an computer-linguist to “develop” your software. A person that is intrinsically good at describing things to AIs, with a good understanding of the domain, the problem, and maybe a bit above Junior level programming skills to be able to write the glue code if needed.

conclusion

We took our jobs.

update 7th Dec 2022

https://youtu.be/vnzAY1SUFSA

education

2022-11-08T00:00:00+00:00

I’m going to write from the perspective of a high iq autodidactic individual who values self determination and autonomy above anything else.

public schools

Designed to break your autonomy and resistance to authority, from the first day you’re forced to sit still for hours and do as you are told, when you are told and how you are told. If you resist, you get sanctioned. Either you will be told to do more stuff (eg. repeatedly write a sentence or copy a paragraph) or you will be punished (eg. detention, suspension, expulsion). The goal is to break your autonomy and make you a good worker. A good worker is someone who waits for instruction and does as he is told without question.

Education boils down to obedience training - as you would train a dog. A dog works for treats and his training is basically just conditioning his behavior to get treats by following instruction. Children do not get treats, they get grades. Grades are a form of social currency. They are a way to measure your worth in society, your obedience to authority, your ability to follow instruction. The child’s parents believe in this reward because they were taught to and their parents believed in it for the same reason. If you follow this back a few generations, you arrive at a point in time when education actually did result in better lives, but that has been out of date for at least 3 generations at the time of writing. If you go back to the 1950s it still does not work. The world has changed and the system has not.

university

You pay twice for your education: once with money and once with time. – AI generated

A waste of time and money for most professions. If there is no danger to human life you can learn everything you need on your own. I might not want a surgeon to trial and error his skills on life human beings, but if you’re in a theoretical field there’s no reason not to. Most practical fields are of no danger to anyone but yourself either. Additionally, every degree is bundled together with subjects that have no relevance to what you want to do. Even if university access is free in your country, it is still too expensive to attend.

University standardizes the knowledge of different fields. It is a way to make sure that everyone has the same knowledge. – AI generated

Let’s take computer science as an example here, as I had to interview my fair share of graduates.

I’m a senior now and throughout the years I’ve had to hold the hands of alot of juniors and get them started. On average, compsci graduates have all the same skills. You have 50 graduates who have learned the identical stuff applying for the job, interview one of them and you know all of them. They are useless. You have to spell out every step of the way for them to know what they are supposed to do and then hold their hand till the end. If you get a CV without a portfolio of projects they did on their own time that resume goes into the trash. It takes way less time to teach a self-taught programmer the obscure but useful things compsci graduates understand than teaching those graduates how to be a competent programmer.

This is not my opinion, this is first-hand real-world experience.

Nowdays you’re lucky if you can dump a folder of creds and profiles on a new hire and say “there you go, that has your wiregurad profile, your company password manager and your creds for internal services, please setup your laptop and clone this repo”. Most won’t be able to figure it out on their own, which is quite an achievement if you consider they have a compsci degree.

testing

Testing measures your short-term memory first and foremost. That’s why testing has been dumbed down over the years, lowering the passing scores, simplifying the language of the questions. As soon as there’s complex syntax used and variables hidden in the query, the conditioned child will no longer be able to process the information and draw conclusions. Children have been conditioned to follow clear instruction, with every step of the way being verbosely defined. Nothing is left to the imagination. The child was not allowed to think for himself, be creative or autonomous. And now it can’t be.

The purpose of exams is not to measure your knowledge, it is to measure your obedience to authority. – AI generated

jobs

Now you’re prepared to join the workforce. You will be doing as you are told for the next 50 years until retirement. For a fraction of the money you could have made if you had been self-employed. Well, at least that’s what I’d have said 5 years ago, now post-covid, all the small businesses went bankrupt and the big guys bought all of it up. So now you’re stuck in the workforce.

retirement & beyond

Your body is tired now and will cease to function shortly after. This is the time when you won’t have regrets like “Oh I wish i studied harder, worked harder, spent more time in the office”. You will be in denial. You will be happy that you can finally rest and live your life, not allowing thoughts like “I could have done that from the beginning”, “I’ve wasted my life playing a game that wasnt worth playing”. You will have health issues from age and exhaustion and won’t be able to really live your life now.

The energy you once had will have faded and as the darkness approaches, you will finally have the courage to admit to yourself and your children that you’ve wasted your life. That you should have spent more time with your family and doing things you wanted to do. You will tell your children to take it easy and follow their dreams, to not worry about what everyone else thinks about you. Like your grandparents told you, but you didn’t believe. But it’s too late for your children now. They’re adults and in the middle of the same game you’ve lost, and they will lose too, because like yourself, they were conditioned to follow instruction and do as they are told, just not your last instruction, but the instruction of the system.

conclusion

The educational system works as intended but not for the benefit of the students. It is a system that is designed to produce workers for the economy. The system is not designed to produce critical thinkers, innovators, or entrepreneurs but to produce good consumers who will buy the products and services of the economy, citizens who will pay taxes and obey the laws, soldiers who will fight for the country, drones who will do as they are told and not question authority. Slaves.

introducing: atlas - my gemini server

2022-08-26T00:00:00+00:00

Atlas is a gemini server without any dependencies. It works on x86, x64 and ARM64. Linux is supported and Windows/MacOS should work fine too - but I don’t know since I don’t use either. The name refers to the launch rocket used to launch the gemini capsules into space.

Features

C#/net7
gemini://
titan://
spartan://
CGI
VHOST
Simple configuration
Docker

gemini://

After a week of trial and error with the occasional look at the documentation, I’ve finished the first version of atlas. The gemini protocol is super easy to understand and implement but the other features I wanted were not :D

There’s no way to upload anything using gemini. The best you can do is to use the INPUT request to pass query strings to the server.

titan://

So there’s a companion protocol called Titan which allows you to upload files. The Titan protocol uses the same status codes as gemini and works just fine in-band.

gemini:// and titan:// are pretty much the same, titan just adds a filename and size parameter inside the META section of the request. Very easy to parse.

spartan://

Spartan was a bit more involved, as it’s not compatible with gemini. It uses a different default port and no encryption. Also the status codes are not identical to those of gemini. I’ve had to open another socket for spartan and write custom handlers for all the different requests. Although the protocol itself is very similar to gemini - simpler really - it is different enough to require a second codepath for parsing requests and sending responses.

vhosts

Gemini Virtual-Hosts can be easily implemented by looking at the SNI parameter in the TLS handshake, spartan doesn’t use TLS, so they put the hostname into the request header - no problems there.

cgi-bin

CGI scripting was super easy too, just a bunch of different environment variables that get passed to the script and its output redirected to the user’s browser. Authorisation and authentication can be done using the client certificate when the request comes from gemini. If the request comes from spartan however, the script has to do the heavy lifting. Maybe there shouldn’t be any authorization or authentication with spartan since its a plaintext protocol and passwords or tokens would be suseptible to MITM attacks.

directory index

Directory Listing was implemented directly in the server and creates a nicely spaced list of files. I’ve not done directory navigation yet though. That shouldn’t be too hard though.

config

For configuring atlas, i’ve settled on a simple JSON file.

{
  "Port": 1965,
  "Capsules": {
    "sallsave.net": {
      "AbsoluteRootPath": "/srv/gemini/sallsave.net/",
      "AbsoluteTlsCertPath": "/srv/gemini/sallsave.net/sallsave.net.pfx",
      "FQDN": "sallsave.net",
      "Index": "index.gmi",
      "Locations": [
        {
          "AbsoluteRootPath": "/srv/gemini/sallsave.net/",
          "Index": "index.gmi",
        }
      ]
    },
    "evilcorp.net": {
      "AbsoluteRootPath": "/srv/gemini/evilcorp.net/",
      "AbsoluteTlsCertPath": "/srv/gemini/evilcorp.net/evilcorp.net.pfx",
      "FQDN": "evilcorp.net",
      "Index": "index.gmi",
      "MaxUploadSize": 4194304,
      "Locations": [
        {
          "AbsoluteRootPath": "/srv/gemini/evilcorp.net/",
          "Index": "index.gmi",
        },
        {
          "AbsoluteRootPath": "/srv/gemini/evilcorp.net/cgi/",
          "Index": "script.csx",
          "CGI": true,
          "RequireClientCert": true,
        },
        {
          "AbsoluteRootPath": "/srv/gemini/evilcorp.net/files/",
          "Index": "index.gmi",
          "DirectoryListing": true,
          "AllowFileUploads": true,
          "AllowedMimeTypes": {
            "text/*": {
              "MaxSizeBytes": 1048576
            },
            "image/*": {},
            "audio/mpeg": {},
            "audio/ogg": {},
            "audio/wave": {}
          }
        }
      ]
    }
  }
}

I took some inspiration from nginx there and since gemini, spartan and titan rely heavily on mimetypes, I’ve added a TSV file of mimetypes that can be easily edited by hand.

docker

To make deployment easy, I’ve added docker support. you can pull it from dockerhub

docker pull herstfortress/atlas:latest

make sure you mount two volumes, one for /srv and one for /etc the config belongs in /etc/atlas/config.json and the capsules in /srv/gemini/my.capsule.com/

code

The code can be found on github

That’s basically it for now.

You can check out a demo at:

gemini://ec.her.st/

spartan://ec.her.st/

the http web is a lost cause

2022-08-20T00:00:00+00:00

I’m going to refer to the HTTP/S-BASED internet as ‘web’ from here on out

Back in the 80s, the web was a new medium for communication, used by real people to communicate with each other. You’d find a website of someone who wrote about something he was interested in and nothing more. No ads, no tracking, no “call to action” buttons, no JavaScript. There were no newsletters or sign-up forms - just a simple website with a couple of pages that linked to other websites with interesting content. That was the time of truly original content. People would make their “WebRings” and have a page dedicated to linking to other people’s websites.

That’s how you found ‘content’ on the web before search engines like Google ruined it all by incentivizing people to embedd ads, tracking code and keywords into every single page and do shit like “Search Engine Optimization” to get more traffic. In the 90s, more and more people started to get on it and businesses started to get really interested too. It was all about email, and how to use it to market your business and get more people to buy your stuff.

Today, it’s a big mess. The web is mostly used by businesses. People don’t create websites anymore, they sign up at a couple ‘Social-Media’ platforms and use it to post pictures of food and to repeat what media outlets say to virtue-signal and get a lot of ‘likes’ and ‘shares’ on their posts. It’s used to crowdsource ‘clout’ and dopamine kicks. They know what they are doing. Everyone on the right side of the bell-curve knows what they are doing.

There are no real people anymore, just a bunch of algorithms that have an Instagram account. I came to the conclusion that the web is not an actual medium that allows you to reach an audience or to share your ideas with the world. It’s an illusion. It’s a bunch of websites that people create for the sole purpose of making money from advertisers. The people running them spend more time on SEO and automation than they do on the actual content.

all browsers are spyware/malware by Google

I’m going to keep this section short, you can read my browser post and dig deeper yourself.

Mozilla gets the majority of its budget from Google and Firefox just exists as a legal shield for Google against an anticompetitive/monopoly lawsuit. Firefox will die as soon as Google pulls the funding from Mozilla.

BUT WHAT ABOUT [[PRIVACY BROWSER NAME]]???!?

They are based on firefox or chromium. They exist at the mercy of Google.

BUT [[BROWSER NAME]] HAS ITS OWN ENGINE!

Yes and that browser can’t display half of the websites correctly.

burn it all down and rebuild it from the ashes

gemini

You may think of Gemini as “the web, stripped right back to its essence” or as “Gopher, souped up and modernised just a little”, depending upon your perspective (the latter view is probably more accurate) – Project Gemini FAQ

Gemini is a new protocol (june 2019) that explicitly states that it opposes the use of the web for anything other than displaying content. No ads, no tracking, no “call to action” buttons, no JavaScript. Just a lightweight browser that can display .gmi files.

as of early 2021 there were about 200,000 known Gemini URLs, spread across about 750 “capsules” (the Gemini community’s term for “sites”), 500 domains and 600 IP addresses. – Project Gemini FAQ

The Protocol is so simple that it’s possible to implement it and create a basic browser in a single weekend. I’m going to attempt this myself - stay tuned for that - it’s a fun project. Also there seems to be a lot of interest in the project.

According to the statistics (gemini uri) there’s 2622 of them, totaling about half a million of URIs.

There’s quite a bit of content on gemini!

no server-side styling

There’s no css support. There’s no styling support what-so-ever on the server. The rationale is, the CLIENT is the one who decides how to display the content. Finally we can have unified styling across all sites! No need for ‘dark mode’ extensions that make the content look like shit.

the protocol will never change

If you write your own client for fun, you can use the protocol today, tomorrow and in 10 years, without having to update your code.

the markup language will never change

The markup language is Gemtext. It’s a simple markup language that will not change. Unlike Markup (and Markdown…) which has different flavours, Gemini Markup is consistent.

gemini markup

Long lines get wrapped by the client to fit the screen
Short lines don’t get joined together unlike Markdown
Write paragraphs as single long lines
Blank lines are rendered as blank lines

Headings
# Heading
## Sub-heading
### Sub-Sub-heading

List
* Mercury
* Gemini
* Apollo

Quote
> I contend that text-based websites should not exceed in size the major works of Russian literature.

Code
    ```
        echo "Hello World"
    ```

And that’s it. Easy to learn, easy to style locally, easy to parse and render. There’s no special syntax.

quickstart

I’m currently using lagrange, a gui client until my own one is done. The website looks exactly like how the client renders gemini sites out of the box. geminiquickst.art is a great place to start exploring.

gemini sites (capsules)

gemini.bortzmeyer.org - Crawler, Statistics about Capsules, Gemini Software

heavysquare.com - A personal blog about tech, coding and random thoughts

astrobotany.mozz.us - Virtual Garden Game

mozz.us - cool collection of projects like geminichat

tilde.pink - tilde.pink is an open tilde server, providing shell, email, and gopher/gemini accounts.

konpeito.media - Low-Fi Music

rawtext.club - Resist the dazzling spectacle of mainstream social and news media.

have fun

I’ll be writing my own gemini server called Atlas and my own client called EVA now - following the gemini naming convention.

Peace!

the RG351MP

2022-03-13T00:00:00+00:00

the software

I have no idea what the factory shipped, I was afraid of anbernic-poisoning so I immediately went to flash ArkOS … 3 times… unsuccessfully and then settled for 351elec @ https://351elec.de/.

elec was a new experience for me. Linux, yes, but everything runs as root, there are no users, no homes, not even a package manager. Getting dotnet and monogame up and running was quite the adventure.

Now, what can this thing do? Let’s look at the hardware so we can get a rough idea about its capabilities.

CPU: RK3326

The official Anbernic Store says its a 1.5ghz CPU, that however is a lie. It runs at 1.3ghz. I’ll be taking a look at overclocking later, maybe I can make it true.

Quad-core ARM Cortex-A35 CPU
ARM Neon Advanced SIMD
ARMv8 Cryptography Extensions
256KB unified system L2 cache
VFPv3 single and double-precision

Alright, so we’re dealing with an Ultra-Low-Power CPU from long-long-ago. The L2 cache is hilariously small, there’s no L3 cache at all. 1GB of DDR3L RAM is not fast either and the GPU will share it with the CPU, making it even slower. Small cache, slow RAM, I’m predicting a memory bandwidth bottleneck. At least it can do SIMD, so we can vectorize our code. Now let’s look at the GPU.

GPU: Mali G31 MP2

Further research shows it’s running at 650 MHz, has two cores and can push 1,3 GPix/s, 980 MT/s or 21.4 GFLOPs. Interestingly, ARM claims 4x MSAA has almost no performance overhead - we’ll see about that! Also surprisingly, it supports Vulkan!

APIs

OpenGL ES 3.2
Vulkan 1.1
OpenCL 2.0
h264,h265,VP9 - Decode & Encode

handheld xbox

So we’re basically working with an original xbox in terms of performance - with some extra CPU cores and newer feature-sets. Sweet.

Anbernic was smart enough to use a low resolution screen, namely a 3.5”, 640x480 @ 60hz IPS screen.
Most manufacturers would have slapped a 720p or 1080p screen on it just for marketing and cripple the rest of the hardware, but at 640x480, there aren’t many pixels to push.

I want to add that the screen is super crisp. The resolution is more than enough for the size.

getting distracted

Suikoden, Pokemon, Earthbound, Batsugun, Alien Soldier, Advance Wars 2, Tekken 3…. I’ve already wasted a couple of hours writing this post, even at this moment, it’s next to me playing some demo videos of random games as screensaver (which you can jump into by pressing start, launching the game that’s currently playing, super cool!) begging me to pick it up or at least look at it.

Can’t wait to play some more Destruction Derby later, but the dog will want out before I’m done writing. Ugh.

game development

The primary reason for me to even buy this device has been to give me some additional motivation for creating games. I want to target the RG351 directly and make it my primary platform. I figured becoming a console game developer would be cool as it would impose a lot of hard limitations on me - not only in terms of performance, but also input methods and screen size.

elec is a readonly OS. Every reboot - it resets. Persistent storage is only possible in /storage. Which is the 2nd uSD card in the RG351MP. That’s fine by me.

Since elec has no real shell, ships almost no libraries and has no terminal, we need to setup a couple of things like a script to launch our game and find/ship all the files required for the game to function. On MonoGame you need to ship libGL.so.1 and libEGL.so.1.

351elec is a 64bit OS and the RK3326 is an ARM CPU, that makes our dotnet command

dotnet publish -c release -r linux-arm64

but first we have to enable single file output.

csproj

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6</TargetFramework>
    <MonoGamePlatform>DesktopGL</MonoGamePlatform>
    <TieredCompilation>false</TieredCompilation>
    <PublishSingleFile>true</PublishSingleFile>
    <SelfContained>true</SelfContained>
  </PropertyGroup>

now we scp the contents of the publish folder and create our little launcher script

#!/bin/bash

gamedir="/storage/roms/ports/test/"

if [ -d "/opt/system/Tools/PortMaster/" ]; then
  controlfolder="/opt/system/Tools/PortMaster"
elif [ -d "/opt/tools/PortMaster/" ]; then
  controlfolder="/opt/tools/PortMaster"
elif [ -d "/roms/ports" ]; then
  controlfolder="/roms/ports/PortMaster"
 elif [ -d "/roms2/ports" ]; then
  controlfolder="/roms2/ports/PortMaster"
else
  controlfolder="/storage/roms/ports/PortMaster"
fi

echo "Starting Pixel Engine (.net6, C#10, ECS) by https://her.st/" > /dev/tty0
source $controlfolder/control.txt
source $controlfolder/tasksetter.sh

export LIBGL_ES=31
export LIBGL_FB=4
export SDL_VIDEO_GL_DRIVER="$gamedir/libs/libGL.so.1"

cd $gamedir
$GPTOKEYB "Pixel" -c "./test.gptk" &
$TASKSET ./Pixel > /dev/tty0
kill -9 $(pidof gptokeyb)
printf "\033c" >> /dev/tty0

map.gptk:

back = esc
start = enter
a = enter
b = up
x = \"
y = \"
l1 = mouse_right
l2 = mouse_right
r1 = mouse_left
r2 = mouse_left
up = w
down = s
left = a
right = d
left_analog_up = up
left_analog_down = down
left_analog_left = left
left_analog_right = right
right_analog_up = mouse_movement_up
right_analog_down = mouse_movement_down
right_analog_left = mouse_movement_left
right_analog_right = mouse_movement_right
deadzone_triggers = 3000
mouse_scale = 8192
mouse_delay = 16

The “game” will now show up in 351elec and you’ll be able to launch it.

That’s as far as I got in a day. Documentation didn’t exist so I had to reverse engineer other ports to figure out how to port >.> I got my PixelGlue engine running, tomorrow I’ll get inputs working properly…

conclusion

For game development on the RG351MP, the architectural choice seems obvious -> ECS

We need to hit the cache or we cripple the GPU with increased memory bandwidth requirements from the CPU side. It will be essential to use all four cores as they’re rather weak.

Vulkan goes without saying, however that’s out of reach for me at this point. I lack the required skills.

It’s smaller yet heavier than expected (i didn’t check the measurements)
The D-PAD lacks a center pin, it can press all directions at once - and often does so
The screen isn’t very vibrant and quite dim - no usage outdoors if sun
The sound is meh. NO bass, muddy mids, exaggerated highs.
The battery life seems to be 5 hours, not 8
You can’t really use both analog sticks at once due to ergonomics
Too slow for N64, Dreamcast, PSP.

stay tuned for part 2

how VPNs are not helping you

2021-08-10T00:00:00+00:00

First of all, a VPN will only hide information from your ISP as you are encrypting everything on your computer and then send it to the VPN provider which then decrypts it and passes it on to the open internet. This means the VPN provider itself has the ability to see exactly what you’re doing and keep records of every packet you send through it.

what if vpns are illegal in your country

Then you’re fucked as soon as you connect.

The popular VPN protocols like OpenVPN and WireGuard are easily detected on the network. Even if your government doesn’t know what sites you access, they see that you are using a VPN and the IP of the VPN server you’re connected to. Now they can call your ISP, get your address and come arrest you. They don’t even have to go to the VPN provider in that situation.

SSH is usually not monitored though, and supports tunneling. If you happen to own a vps somewhere that isn’t directly connected to you, you might create a socks5 tunnel quite easily

ssh -ND 8080 user@ho.st # Creates the socks5 proxy
curl -x socks5h://localhost:8080 https://wtfismyip.com/text # uses the proxy

but make no mistake, ssh tunnels have a considerable impact on your net speed, especially ping times.

compared to without the tunnel

That’s just a socks proxy though, not a VPN. DNS for example will still bypass the tunnel.

vpn’s are legal and my vpn provider doesn’t keep logs

Let’s give them the benefit of doubt and say they really don’t. Who’s to say the Datacenter they reside in doesn’t? After all, packets from protocols like OpenVPN are easily identified on their infrastructure - as I’ve made evident above.

Additionally, if the VPN you’re using is in any of the following countries

Australia
British Overseas territories (e.g. British virgin islands)
Canada
Israel
Japan
New Zealand
United Kingdom
United States
Denmark
Netherlands
France
Norway
Germany
Belgium
Spain
South Korea
Singapore
Sweden
Italy

They are required by law to have logs and share them with the authorities. Even if they claim not to keep logs. If you live in one of them, your government can spy on you by proxy - simply report you to a member-state and request the data from them.

Additionally, in order to pay the VPN provider, you have to give them your payment details which are directly linked to you. The only VPN provider on the clear-net that allows cash payments and no sharing of personal details - as far as I know - is mullvad

Do you think your VPN provider will defend you when they get pressured/threatened by a government to surrender the data they have about you? Risking their business? Their money? Don’t be naive.

which browser for privacy

2021-08-03T00:00:00+00:00

A common misconception is that cookies are used for tracking and cleaning cookies or pre-emptively blocking some or even all 3rd party cookies will make you untraceable.

That might have been the case 10 years ago.

Today they are tracking you by identifying your browser’s and by extension your system’s unique characteristics. That’s what people mean when they talk about ‘fingerprinting’ and - contrary to what some cybersecurity ‘experts’ tell you in their YouTube videos or blogs - there’s nothing you can do against it.

but some browsers prevent fingerprinting

Yeah so they claim but they don’t really prevent it. Let’s take Brave Browser for example. They have a unique set of measures they take against fingerprinting you, like for example blocking canvas read access or changing the list of fonts it reports to be installed on the host. You might already noticed the issue: a unique set of measures. Already they can deduce that the browser in use is Brave, can simply fingerprint based on other characteristics and so you become - yet again - traceable.

who are ‘they’ and what do they use to fingerprint you

Primarily Google and Facebook.

Google Analytics is on 99.9% of websites that show up on a google search. It doesn’t stop with google analytics though, they also own YouTube and many websites embed YouTube videos. Facebook has the infamous Like button and embedded content, like Photos, Videos, Comments … Both also happily let you sign up with their accounts elsewhere - “Register using Google/Facebook” or “Link your Social Account”.

It gets far more insidious when you consider all the spin-offs they own and you probably use as well, like Facebook Messenger, Google Hangouts, WhatsApp, Gmail,…

If you use any of that, it now has a name to your fingerprint, a phone number, emails and all their contents which tell them which other websites you have an account on, where you work and which jobs you applied to among a plethora of other more or less sensitive data.

If you pool all that information together (they do that completely automated) they pretty much know exactly who you are, what you do, where you are and what you like/dislike.

so what is the solution

There’s only one real solution. Get rid of Facebook and google. Close your accounts, never go back to them. Don’t use google to search the internet (use Searx, Startpage), don’t use YouTube to watch videos (use PeerTube, Odysee/LBRY), don’t use Facebook to stay in touch with your friends and family (meet, chat using XMPP or signal) and instead of GMail, host your own or find yourself an alternative.

what if i can’t get rid of it

There’s still a way but it’s tedious and easy to fuck up.

Isolate your browsing habits to different browsers and create new accounts wherever possible.
Have one browser for Facebook.
Have one browser for Google Services and
and another for everything else that’s in no way connected to your Google/Facebook accounts.

AND PAY ATTENTION

you probably created a bunch of accounts in other places with Facebook/Google Social Login OR YOUR GMAIL ADDRESS. Some accounts might even be connected to both. Logging into any account that’s connected to either outside of the designated browser will set you back to square one. Create new accounts if you ever linked them to your social accounts.

browser recommendations

While avoiding Google Chrome and Microsoft Edge might be obvious and needs no explanation, I also avoid Firefox and Brave.

Firefox blogs about pro-cancel culture stuff and uses Google Search by default - which is their biggest source of income.

Brave has a history of rewriting urls you visit with their affiliate ones.

In the past I’ve recommended browsers like Dissenter, WaterFox… all of them turned out to be evil. Might as well just abandon hope and give up on the Web.

Gopher could be an alternative.

is that enough?

No.

imposter-syndrome as competitive edge

2020-08-27T00:00:00+00:00

my aquired imposter-syndrome

no, this isn’t clickbait and I’ll make a case for it

When I started getting into programming almost 14 years ago - when I was 14 - i was immersed in a developer community with a handful of very gifted programmers but also hundreds of copy/paste kids from the first minute. I had joined an english-only private server community that wrote MMORPG servers from scratch.. as someone who could barely order a drink in english and didn’t even know how to pronounce ‘C#’. Thankfully, there was a guide on how to download some source code, compile it and how to connect your game client to it, it required zero programming knowledge and was basically unzip and F5 in visual studio. Half an hour later I was writing an advertisement about my new server.

They tore me apart like fucking vultures.

When reality comes crashing down on you, you have a binary choice: accept it or go into denial. I had no idea what I was doing and people were quick to point that out. My first choice was denial. I defended myself saying most other servers are using the same source code and have the same problems and my Hamachi room was better and my uptime was from after-school hours till the morning before I had to turn off my computer (ahh what i’d give to be 14 again). I felt like i was competing nicely with the majority but people immediately pointed out that ‘just because the majority is a bit more shit it doesn’t meant my shit is good’

I felt horrible - because they started making sense.

Having players play on my shitty server - talking with me, encouraging me, telling me about all the known bugs and exploits,.. - was a source of good feels though, so I couldn’t just quit. I had found something that truly gave me joy in life, i was passionate and driven. I’d just scroll through the code files and read everything trying very hard to understand at least a single line. I’d read the threads on the forum, 200 pages in one go if it was that long, to get some insight into programming; what were people talking about? Is anyone posting code? Is there something I could copy paste?

There was a lot of discourse, my english was too bad to make sense of most of it but i was persistent! Failure was not an option, I’d prove all those cunts wrong and learn how to fix the problems and create something they couldn’t criticize anymore, force them to see me as an equal! How they saw me instead was a kiddy who couldn’t do anything bot leech off of others and copy paste together some shit others had to write.

Back to feeling terrible I guess.

After some weeks of copy paste I had the skill of a GOD… in copy paste … still couldn’t write any code myself but at least I somehow figured out how scopes work and where I have to paste stuff and that I sometimes had to rename a variable. Most of the other people leeching code couldn’t figure out where to paste stuff and if there was a typo somewhere they’d complain on the forums about “shit releases” because the “code doesn’t work”. That gave me some joy. Additionally I had picked up a couple terms and keywords, i had a rough idea what an integer and a string was, i knew about semicolons and curly braces and one guide that showed how to add a new NPC to the game was easily deconstructed into the knowledge of how to edit existing NPCs. That opened up a lot of new terrain for me. Now I’d be able to create some basic quests! I’d spend days to build a trivial ‘talk to A then talk to B then get item C’ quests. My players liked what I was doing and the american / british players I had would improve the grammar, spelling and wording and I’d update the code. I felt good again! Posted my first ‘guides’ on how to make those quests on the forums, .. and get torn into pieces by people shitting all over my code.

Hello darkness my old friend…

Months later, I’d have hardly any issues reading english forum posts, i’d read parts of code and understand them. I finally discovered StackOverflow! Now I could ask my questions there and get answers from professionals! Wrong. I got shamed and banned within a week. There’s been some leaks of new source code with much more features, it wasn’t as easy to setup, required MySQL and Apache, config files and it was unstable as fuck. But I knew i needed to make it work to stay competitive. I’d spend the first day trying to figure out what MySql was, what apache was, how i’d get the database imported and the server connected to it, how to create a SignUp page in php, ended up paying someone 10$ for a blank page with a username and password box and a submit button. That’s when I created my first paypal account at the age of 15 and spending half my monthly pocket money. Paypal was supposed to be 18+ but in the early days they didn’t do any verification. Sadly that ruined the end of the month. I had to pay 20$ a month for 1GB of 3G internet and I just spent half of it. So a couple days later internet was gone for the rest of the month. Without any internet all I had was trial and error, changing random stuff and see what happens, do more NPCs and quests, write a list of things to google when I had internet again,.. and fearing that all my players would bail on me for the long downtime.

and i couldn’t blame anyone but me

Two years later, I’d be comfortable writing code, implement custom features nobody else had ever even thought about. I’d stay in the top 3 for months, have the most popular private servers for a year while the big guys took a break working on their next version. Those big guys still saw me as the same Leecher I had been in the beginning. They’d always talk shit about me, when I posted code they’d still heavily scrutinize it and find 500 things wrong in a 100 character line of code. It got to a point where I thought i was just a complete failure. I felt like I improved A LOT but the people i looked up to and wanted recognition from wouldn’t give it to me, just the polar opposite. Another problem was that I had reached the stage where I could actually understand their criticism and see the truth in it. There was no denial anymore, I knew they were right. I was proud of how good I had become on my own but the people I looked up to would always prove that I hadn’t become good at all.

I felt like a complete scam.

I took a break from the private server scene and bought a book on C#, read it, did all the practices, it was horribly boring. I stated writing some random utility tools for my personal use, made some tools for friends, dabbled with GUI programming and chat applications. My coding abilities were very much constrained to the MMORPG servers I was working on. I had no idea how to do anything else but I learned quickly after a slow start. Writing my own chat client/server to replace MSN was the first project i got really into since taking a break from the private server development. Eventually I realized that I could probably write my own private server from scratch as its just like a chat server, just with different logic and more packet types - which were already available in the community.

So I started writing my own. I’d have two, three sometimes four reference projects open in visual studio, initially copying a lot of parts without modifying them in any meaningful way, primarily the cryptographer, the packets, the packet parsers, and the calculations and dice-rolls. After 5 months I had something that was almost playable. You could login, kill monsters, get experience, use skills, talk to NPCs,.. all of the basic stuff worked - more or less. Surely now they couldn’t call me a leech anymore, right? “Half of the code is leeched and the other half is even worse”

I’d never get a job if i didn’t start learning how to write proper code. If I want to turn my passion into my job I have to become as good as them. I started pestering the guys i idolized. Please teach me how I can improve, please show me your code so I can learn. No fuck off! Please! NO! PLEASE SELL ME YOUR CODE, ILL GIVE U 100$! 100$? Hahaha I spent months on this code its worth at least 300$! OK ILL GIVE YOU 300$!

I probably spent 600$ on private server source codes in total. I bought some off of the guys directly, others I bought from people they trusted, others I manipulated into giving me their for free and even others I’d pester into letting me watch them code on TeamViewer. I’d ask for control to check out files and make sure I got every file from top to bottom for at least a moment. They didn’t know that I ran screen-recording software so I could slow it down later and type it all out. Which I did. It took hours and hours.

Often times I’d look at their code and find myself dumbstruck by never having seen certain classes or attributes or unsafe code in unconventional places. I’d take parts of that code to StackOverflow or other people i knew and ask for explanation if google didn’t yield anything.

I got quite good at this point. I was more than comfortable writing code, I even wrote another server from scratch, this time with minimal “inspiration”. And I made sure it used less RAM than any other public server, was more CPU efficient than any other public server, would never crash and would still have features nobody else had. I’d get sooo obsessed about optimizing code because I knew people would just shit on it again if it wasn’t perfect. I’d take courses about low level C# programming with unsafe code, I’d take courses on Threading and read CodeProject articles on “High Performance Sockets” and “Memory Allocations” all of that. Eventually I made my server run on a raspberry Pi - i realized that if I set my target hardware to something as powerful as a retarded potato, I’d have no problems with performance when I actually put it on a VPS for production.

I’d get great results, everything would be fast enough, use weird memory optimizations like packing things in BitArrays, stackalloc, pointers, threads,.. and my code would look absolutely horrible every time i got half way through to completion… I always felt like a scam again. I mean after all, if I hated my code and could see 100 things that are wrong, im sure they could see 500 things that are wrong with it. I was too afraid to post anything anymore because I couldn’t take the shame.

the revelation

One of my idols had sold me his code and even after all my pestering, still talked to me and occasionally even answered some questions (though I’ll never forget his default response to 90% of my questions: “I forget D:”). He was a couple years older than me and had a job as a programmer when I sent him my latest server I had written from scratch. It was the first time he said “its not bad”.

I said that one day I’ll become a professional software developer too and get a job in the industry and that I’ll keep learning until I’m useful and try to get into uni for a computer science degree.

He laughed and told me that I’m clearly better than most of his colleagues

Telling me that some computer science graduates struggle with reversing an array and most juniors need someone to hold their hand just to write code that works - let alone efficiently or beautifully.

He said that almost all code is shit code, that his code is shit code in his eyes, that it keeps him on his toes, pushing the limits, getting better, that if you are your own critic and take your criticism to heart and work with it, success is inevitable and you’ve earned the the right to call yourself a programmer.

He said as soon as you become complacent you stagnate and become a fraud.

today

I’m convinced, Imposter-syndrome is just as bad as pride. Good servants but bad masters. The key is to find balance. Know what you can do, know what you can’t and be honest about both.

simplifying our image hosting service

2020-08-25T00:00:00+00:00

Let’s cut out the middle-man

$ fuck FTP

Last time we’ve setup vsftp to handle our image uploads. This foolishness ends now. We’ve already got our SSH Server up and we can easily transfer files through it. Let’s do that, shall we?

apt remove vsftp -y

$ fuck Apache

Apache configs suck. Hello nginx.

apt remove apache2 -y && apt install nginx -y

$ setting up nginx

Let’s also change the domain from h.img.alumni.re to cdn.her.st as I’m not going to be using my homeserver anymore.

We’re going to use threaded async IO and sendfile for maximum throughput on potentially big files like binary data, audio and video but not for small things like images and assets (which are mostly css and js text files). There’s some added latency involved when using threaded async IO so its only worth paying that overhead if the file tansfer will usually take more than a couple seconds. We really don’t want to add any additional latency to image requests as that would unnecessarily slow down page loading speed.

/etc/nginx/sites-enabled/default

server {
    listen 80;
    root /srv/http/cdn.her.st;
    index index.html;
    server_name cdn.her.st;

    location / {
     try_files $uri $uri/ =404; 
    } 
    location /bin {
     sendfile on;
     aio  threads;
     try_files $uri $uri/ =404; 
    }
    location /videos {
     sendfile on;
     aio  threads;
     try_files $uri $uri/ =404; 
    }
    location /audio {
     sendfile on;
     aio  threads;
     try_files $uri $uri/ =404; 
    }
    location /assets {
     sendfile on;
     try_files $uri $uri/ =404; 
    }
}

Now let’s spin up the service and we’re done!

systemctl enable nginx --now

$ porting ImgUp to bash

#!/bin/bash
path=$1
filename=$2
if [ -z $filename ]; then 
    filename=$(basename $path)
fi

scp -q $path trbl@cdn.her.st:/srv/http/cdn.her.st/images/$filename
echo "/assets/images/$filename" | xclip -sel clip
echo "/assets/images/$filename (added to clipboard)"

usage:

imgup /path/to/image.jpg [Optional remote filename]

Much better than the C# clusterfuck with multiple classes.

we did it again! :D

The first iteration brought down complexity and increased performance! Can’t ask for a better outcome considering how little effort this took.

whats left

imgup argument parsing
make universal script that can upload any kind of file
create multiple versions of the image (server sided?)
- Raw image
- Resized to Thumbnail Size

ssh is power

2020-02-03T00:00:00+00:00

let’s just jump right in.

port forwarding

Did you know, you can fuckin port forward with fuckin ssh?! You might want to host a gameserver so you can play with your friend, or show a client your progress by giving him access to your local webserver - but you are behind a firewall and don’t want to or simply can’t forward the ports? SSH can do it. All you need is a cheap server, like the cheapest of my Micro plans and you can use it to forward ports in a single line.

remote forwarding

For said scenarios above, where you run a server locally and want to expose it to the internet, like a webserver or gameserver running on port 8080

ssh -R 8080:127.0.0.1:8080 user@hostname.her.st

now you can give hostname.her.st:8080 (or the server IP) to your friend or client, and ssh will tunnel everything to your local machine.

local forwarding

now let’s say you want to access a mysql server that’s running on your server. You can’t connect to it over the internet MySQL will only listen on 3306 locally, and be firewalled off. Still want to connect your dev environment to it? SSH can do it.

ssh -L 3306:127.0.0.1:3306 user@hostname.her.st

if you now connect your mysql client to 127.0.0.1:3306, you will actually connect to hostname.her.st:3306 (and you get encryption for free)

appendix

The format is defined like that:

8080:127.0.0.1:8080

RemotePort (on the server):Destination (your computer):LocalPort(on your computer)

disconnected - no problem

if you have access to screen on the server, use ssh -t user@hostname.her.st screen -RR and you will create a screen immediately upon login, or reattach to the last one if it exists. Similar behavior can be achieved with tmux. if your internet connection goes down or indeed even if you close the terminal, your stuff will keep running.

mount remote filesystem locally

tired of scp yet? try sshfs

run a speedtest

yes | pv | ssh remote_host "cat >/dev/null"

and it it’s good enough (internet: >= 5mb/s, local: 35mb/s)

mount it

sshfs -o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3,idmap=user user@hostname.her.st:/ /mnt/server-root

now you can just cp and mv to /mnt/server-root to upload files. you can treat it like any folder.

oh and visual studio code

You can literally let vscode ssh into your server, install itself there, copy all your local settings and extensions over and have vsc run on your server instead so your development environment is always the same, no matter what device you use. All your files will be stored on the server too, so no more forgetting your files either. I’ve been using it for months and love it. https://code.visualstudio.com/docs/remote/ssh

unlimited free proxies

2019-11-20T00:00:00+00:00

c# proxy checker

With the shit that’s going down in Hong Kong, Iran and dozens of other places around the world, VPN’s and Proxies are in high demand. Wouldn’t it be great if we had a free unlimited supply of web proxies? Googling around for some free ones, I mostly found sketchy websites with sketchy paid plans, and a couple Pastebin links with massive lists of IP’s and ports…

…massive lists with IP’s and ports…

But before we even try to automatically search Pastebin for new proxy lists, we have to make sure that we can test proxies in the first place; Thankfully, .net comes with full proxy support and you can use it in a single line!

I’ve made one and put it on github, I call it SockPuppet because it tests SOCKS web proxies.

arch

My version is a simple Producer-Consumer setup that only keeps twice as many proxies in memory as it has worker threads, in case the lists get really big after running the scraper for a few weeks or months. I’ve annotated the code and encourage you to quickly read over it.

BlockingCollection

Proxies here is a BlockingCollection<Proxy>. If you never heard about BlockingCollection<T>, it’s kinda like a threadsafe List<T> that behaves like a ConcurrentQueue<T>. You foreach it just like a List and every iteration the current item is removed from the collection. That being said, a foreach loop on a BlockingCollection will never return. The Collection is going to start Blocking when it’s empty - which is perfect for the consumer workloop of our Producer/Consumer setup.

In addition, it can also be made to block on Add(T item)! Usually we’d need a lock of some kind to keep the ‘queue’ from growing too big. Simply instantiate it with the overload that accepts an int for boundCapacity like so:

var blockingColl = new BlockingCollection<T>(10);

and bockingColl.Add() will block as soon as there’s 10 elements in it.

private static void WorkLoop()
{
    // threads will be blocked at Proxies.GetConsumingEnumerable()
    // as long as everything INSIDE the foreach loop is threadsafe
    // we can throw as many threads on it as our network can handle
    foreach (var proxy in Proxies.GetConsumingEnumerable())
    {
        proxy.Test();                 // connect to proxy, download website
        if (proxy.Alive && proxy.Safe)// Safe = Identical response as without proxy
            Writer?.WriteLine(proxy); // write to output file
        // write to stdout, IP starts at position 8
        Console.WriteLine($"{(proxy.Alive ? "[ Up! ]" : "[Down!]")}{proxy}");
        Trace(proxy);
    }
}

Please note that Writer is a StreamWriter and is not threadsafe, also Console’s threadsafety will break if you start changing colors.

private static void StartThreads(int threadCount)
{
    Threads = new Thread[threadCount];
    for (int i = 0; i < threadCount; i++)
    {
        Threads[i] = new Thread(WorkLoop); // Target the method above

        // IsBackground = true
        // if the main thread exits kill this thread,
        // don't wait for it to exit and keep a zombie process running
        Threads[i].IsBackground = true; 
        Threads[i].Start();
    }
}

parsing

We’re having an easy time again, the parser will be a breeze!

The go-to format you find on Pastebin is ‘IP:PORT’ like so:

252.25.168:8080
200.170.41:80
197.38.230:60724
...

Let’s write a parser that will only read a sane amount of lines while checking them in the WorkLoop on N threads, great performance, great resource utilization. We don’t waste RAM or sacrifice startup time by reading everything and instead do things on demand with a healthy buffer.

while (!Reader.EndOfStream) // while there is shit to read
{
    //always trim your lines!
    var line = Reader.ReadLine().Trim();
    var parts = line.Split(':');//naive implementation
    var ip = parts[0]; // expecting only valid data
    var port = ushort.Parse(parts[1]); // (╯°□°）╯︵ ┻━┻ 

    var proxy = new Proxy(ip, port, timeout);
    Proxies.Add(proxy); // BlockingCollection<Proxy>
}

testing

Now, testing the Proxy is super easy. WebProxy is a built-in class, just like HttpWebRequest. Both together and we’re virtually done.

public void Test()
{
    try
    {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://her.st");

        var proxy = new WebProxy(IP, Port);
        proxy.BypassProxyOnLocal = false;
        
        request.Proxy = proxy;
        request.UserAgent = "ProxyTester Version: 1";
        request.Timeout = Timeout;

        WebResponse webResponse = request.GetResponse();
        var reader = new StreamReader(webResponse.GetResponseStream());
        _response = reader.ReadToEnd().Trim();

        Alive = true; // proxy is working
    }
    catch
    {
        Alive = false; // proxy is not working
    }
}

sorting

It would make sense to do multiple rounds of sorting, by latency, by throughput, by country, we will focus on sorting by country first. There’s 100’s of ‘free’ APIs for ‘Geo-IP’ lookups, but there’s always one catch: after a certain amount of queries, they ask for your credit card.

How do those services do it? I’ve done some googling and as it turns out, there’s free databases available, most notably the IP2Location Db’s - we will use LITE-DB5 which has a C# parser by the Taiwanese Sky Land Universal Corporation licensed under the Unlicense. Perfect. Let’s legally steal their code and hook it up.

I will dive deeper into how this db was created in my next networking article

private static void Trace(Proxy proxy)
{
    var location = Locator.Locate(IPAddress.Parse(proxy.IP));

    if (!OrderedProxies.ContainsKey(location.Country))
        OrderedProxies.Add(location.Country,new List<Proxy>());
    OrderedProxies[location.Country].Add(proxy);
}

OrderedProxies?

As you can see in the code above, I created a Dictionary<string Country, List<Proxy>> namely OrderedProxies which lets me group the proxies by country very easily. In the next part, I hope to have had time to refactor and rename most of the worst variable and class names. I’ve been facepalming too much while writing this article. I am fully aware that the Trace method makes the entire producer/consumer setup pointless.

what’s next?

There’s still a lot to be added to this little service. Off the top of my head:

Ping proxies to find the best latency
run speedtest to find the best throughput
automate Pastebin harvesting
database
web api with json responses - REST like.

see you soon

creating an image hosting web service

2019-07-11T00:00:00+00:00

We’ve got work to do

$ needs

Easy to use service that lets us host images and other static data and allows direct linking and embedding. Any webserver will do. I have a virtual server with apache2, so I’ll use that. Considering that box also runs a FTP server, we shall upload our files using FTP. We also need an app that can take an image path as argument, upload it to our service and copies the direct link to it straight into the clipboard.

The app should be a command line tool and do nothing without arguments.

$ requirements

A tiny private server should be able to host everything we need.

Web Server
FTP Server
SSL
UploaderApp
Possibly a Browser / Management app in the future

$ setting up the webserver

First we have to install a webserver. Usually I’d use Nginx, but since I have an apache server already up, I’ll go with that.

I’ll use a custom subdomain for this service. https://cdn.her.st/ Here’s a visual representation why I chose this domain in case you are wondering:

Server	Type	Domain	TLD
h	img	alumni	re

the h symbolizes my Homeserver.

install apache

# Updating the repos and installing everything we need
> sudo apt update && sudo apt install apache2

Apache is installed and ready, you can verify that by going to http://0.0.0.0/

setup VHOST

Now its time to configure the virtual host. To do that, we have to create a text file in

/etc/apache2/sites-enabled/ called cdn.her.st.conf (replace the name with your domain) and put some text inside of it.

> sudo nano /etc/apache2/sites-enabled/cdn.her.st.conf

<VirtualHost *:80>
        # ServerName is supposed to be DOMAIN . TLD
        ServerName alumni.re
        # ServerAlias is supposed to be the entire (sub)domain
        ServerAlias cdn.her.st
        # Document root is where your files will be stored
        DocumentRoot /var/www/cdn.her.st/html

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

And for my final trick, let’s create the directory structure for the webserver.

# We've set this path above as DocumentRoot
> mkdir /var/www/cdn.her.st
> mkdir /var/www/cdn.her.st/html
> mkdir /var/www/cdn.her.st/html/img

After setting the DNS records of your subdomain at your provider, your site should be ready, you can verify that by going to it in a browser: http://cdn.her.st/

$ setting up Certbot for free SSL

Pretty important to set up SSL nowdays as every browser will freak the fuck out if your images come from an ‘insecure source’ - basically over http instead of https … There’s no real benefit in encrypting static files, but if it makes the browsers happy..

# Updating the repos and installing everything we need
> sudo apt update && sudo apt install apache2 certbot python-certbot-apache
# let it do it's magic
> sudo certbot --apache

When you’re asked how you want to enable SSL, chose “redirect”, wait for certbot to finish writing new configs and pulling the certs, then run

> sudo certbot renew --dry-run

If that command fails, check Certbot Help otherwise, let’s automate the renewal process by adding

0 1 * * * /usr/bin/certbot renew & > /dev/null

to your crontab (> crontab -e) - and dont forget the new line at the end or it will complain..

$ setup FTP

I regret using FTP for this already, but let’s get it installed and setup. In theory, you could skip this step and use http uploads, but I rather have something isolated from the webserver, as I plan on writing my own webserver for static content later, and I probably won’t implement anything but GET. Enough rambling, let’s punch in some commands to install it and enable it on boot.

> sudo apt-get install vsftpd -y
> sudo systemctl enable vsftpd

Now let’s add our FTP user

#ftp boi gonna live in the webroot (and get jailed into it)
> useradd ftp -m -d /var/www/cdn.her.st/html
> sudo passwd ftp

Now we have to make sure our server is setup properly. We will need to configure write permissions, ports, directories and authentication. I’ve attached my configuration file which should work for you too.

> cat /etc/vsftpd.conf 
listen_ipv6=YES
anonymous_enable=NO
anon_upload_enable=NO
local_enable=YES
write_enable=YES
dirmessage_enable=YES
use_localtime=YES
xferlog_enable=YES
connect_from_port_20=NO
secure_chroot_dir=/var/run/vsftpd/empty
pam_service_name=ftp
rsa_cert_file=/etc/letsencrypt/live/cdn.her.st/cert.pem
rsa_private_key_file=/etc/letsencrypt/live/cdn.her.st/privkey.pem
ssl_enable=YES
require_ssl_reuse=NO
pasv_enable=YES
pasv_min_port=1024
pasv_max_port=1025
allow_writeable_chroot=YES
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list
file_open_mode=0777
local_umask=022

/etc/vsftpd.chroot_list is an empty file. Just run

> sudo touch /etc/vsftpd.chroot_list

to create it, also notice I’m re-using our SSL cert we got for our webserver earlier. I’ve taken the default path here, so when certbot renews it, vsftpd will always use the newest one.

Now let’s start the ftp server.

> sudo systemctl start vsftpd

bonus step: disallow ssh for ftp user

> sudo nano /etc/ssh/sshd_config

Add/edit the following line

DenyUsers ftp # ftp is my user

> sudo systemctl restart vsftpd

$ bringing it all together with our c# netcore app

review

Let’s quickly review what we got setup so far so we can start thinking about how we are going to implement that in code..

Webserver w/ ssl & domain
FTP Server w/ ssl & domain
FTP User w/ r+w access to domain’s root folder

and here’s what we still need

App that let’s me upload images quickly.

c

Let’s start with the FTP Upload first. That should be the most difficult part here. Thankfully, the .net framework already has pre-made classes to deal with FTP, namely FtpWebRequest. So let’s design our first class, the one responsible for uploading images…

public static class Uploader
{
    // First we will set our root address for the following requests
    private const string FTP_IMG_ROOT = "ftp://cdn.her.st/images/";
    // Next we set our Id file's public HTTP url, we will download this and parse it to set the current Id.
    private const string HTTP_IMG_ID_FILE = "/assets/images/Id.txt";
    // We use the curId as something like a counter, so we don't overwrite old files. I decided to do this on the client since *I'm* the only client.
    // Don't be an idiot.
    private static int nextId;
    private static int curId;

    // Since this is a static class and its initialized only if arguments are passed, its ok to block in the constructor.
    // Don't do this is bigger applications. 
    static Uploader()
    {
        // as stated above, here we download and parse the Id file so we know what the last Id on the server is
        // (it gets worse)
        using (WebClient client = new WebClient())
            client.DownloadFile(HTTP_IMG_ID_FILE, "Id.txt");

        if (File.Exists("Id.txt") && int.TryParse(File.ReadAllText("Id.txt"), out curId))
            nextId = curId + 1;
    }
    
    // Saves a couple of lines of code :D
    private static FtpWebRequest CreateUploadRequest(string file)
    {
        var request = (FtpWebRequest)WebRequest.Create(FTP_IMG_ROOT + $"{file}");
        request.Credentials = new NetworkCredential("ftp", "root");
        request.EnableSsl = true; // this is the reason we can't use WebClient. It won't work with ssl.
        request.Method = WebRequestMethods.Ftp.UploadFile;
         return request;
    }
    
    // not sure why i ended up using tasks... i bet they just slow everything down tbh..
    // you test that and email me the results. blog@her.st ;D
    public static async Task<string> UploadAsync(string path)
    {
        // further attemt at creating a more unique path but still giving it some readability.
        // this would turn File.txt into File_3948.txt
        // I don't even check if a file with the same name exists and just assume so.
        // Asking the server for a file list, looking for it and THEN starting to upload
        // takse too much time. This is single user anyways, I won't run 20 instances of this shit.
        var request = CreateUploadRequest(Path.GetFileNameWithoutExtension(path) + "_" + curId + Path.GetExtension(path));
        using (var fileStream = File.OpenRead(path)) // doing streams like a good boi in case file is biiig
        using (var ftpStream = request.GetRequestStream())
            await fileStream.CopyToAsync(ftpStream); // but in the end I take the lazy route.
        await UpdateId(); // Told you it'd get worse.
        return request.RequestUri.AbsoluteUri.Replace("ftp", "https");
    }
    
    // did you think the server would keep track of the counter? 
    private static async Task UpdateId()
    {
        Interlocked.Increment(ref curId); // atomicly incrementing our counters because by now i have no idea where our methods execute
        Interlocked.Increment(ref nextId); // doing this seems to calm me down, no idea if its snakeoil
        await File.WriteAllTextAsync("Id.txt", $"{curId}"); // we write it so we can read it ...
        var request = CreateUploadRequest("Id.txt"); // another request
        using (var fileStream = File.OpenRead("Id.txt")) // this is a file with a fucking number in it. Number might get big, lets use a stream XDDDDDDD
        using (var ftpStream = request.GetRequestStream())
            await fileStream.CopyToAsync(ftpStream); // another lazy way out
    }
}

Couple of lines of code, nothing too fancy, kept it simple for the most part. I wish I went with HTTP uploads instead, having to deal with the FtpWebRequest directly made this class way bigger than it needed to be. Let’s make our Main() smaller :D

public static async Task Main(string[] args)
{
    if (args.Length == 0) // no args? no bueno.
        return;// seppuku
    var builder = new StringBuilder();
    for (int i = 0; i < args.Length; i++)
    {
      // upload image, get direct link bacl
        var url = await Uploader.UploadAsync(args[i]);
        // if this is the last file, don't add a new line at the end.
        if (i == args.Length-1)
            builder.Append(url); // add to url list
        else
            builder.AppendLine(url); // add line to url list
        Console.WriteLine(url); // optional
    }
    // set clipboard to the url list 
    Clipboard.Set(builder.ToString());
}// another kind of seppuku

Not much to say about that.. I’m using a command line utility called xclip to set the clipboard on linux because I’m too lazy to figure out how to properly do that. Anyways, it works so I’m happy.

public static class Clipboard
{
    public static void Set(string text)
    {
      // gotta write it into a temp file for xclip :D fuck you xclip :D
        var tmpFilePath = Path.GetTempFileName(); 
        File.WriteAllText(tmpFilePath, text);
        try
        {
          // now we cat it and pipe it into xclip..
            var arguments = $"-c \"cat {tmpFilePath} | xclip -i -selection clipboard\""; 
            var process = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = "bash",
                    Arguments = arguments,
                    UseShellExecute = false,
                }
            };
            process.Start();
            // y u no exit u shit? setting the clipboard won't take 5sec on the slowest potato. 
            // Kill after 5 sec, sumting wong.
            process.WaitForExit(1000 * 5); 
        }
        finally
        {
            File.Delete(tmpFilePath); // its not you, its me
        }
    }
}

F5

We did it. The minimal viable product is complete. We will iterate over the code and expand its functionality soon - There’s a couple of things we can put on our todo list now, as we have a working prototype.

refactor and clean up the code
Convert to WebP
create multiple versions of the image
- Raw image
- Resized to Thumbnail Size
Copy all the links into the clipboard with formatting for easy copy/paste into new Article.md

First things first though, here’s how I’ll use this application:

Print + shift
        maim -s ~/upload.png; cwebp ~/upload.png -o ~/upload.webp; imgup ~/upload.webp && play ~/.config/.ding.wav && trash ~/upload.webp && ~/upload.png

maim is a screenshot utility and gives you a selection rectangle you can place with your mouse, then save it to my home directory as upload.png, run cwebp a command line webp converter, saving it as upload.webp next to the source file, invoking our app, I’ve called it ImgUp passing it the upload.webp path, then after it finishes uploading, I play a ding sound and delete the files from my home directory.

The URL to the image is now in my clipboard and I can CTRL+V it here

the birth of a programmer

2019-07-10T00:00:00+00:00

A bit over a decade ago a series of coincidences made me embark on the quest of becoming a programmer. The event that started everything must have been when thirteen year old me with his old Athlon XP 2400, 256 MB DDR RAM, nforce2 board, an (infamous) Geforce4 MX 440 Windows XP Computer, was trying to play Star Wars Knights of the Old Republic in the early 2000’s. it was a very cinematic experience in the menu, and a glorified PowerPoint presentation inside the Endar Spire. What I’m talking about? Horrible performance. Single digit framerates.

I was determined to get to the bottom of this. Why would my computer betray me at a time like this?

Down the Rabbithole

Of course, I immediately tracked down the only Computerstore in town and went there. It was a tiny shop with three or four employees, boss was still working the register and the entire store had a family vibe to it. Not to mention the shelves stacked to the roof with the newest graphics cards, motherboards, CPU’s, networking gear, printers, pre-built Systems, CRTs and a couple of early LCD’s - with the good old 4:3 aspect ratio.

Now, this PC I had gotten when my family upgraded to a Pentium 4 with Hyperthreading and a Geforce 6600 was already obsolete and the store guy told me that there’s not much to be done, but told me to get another 256MB RAM module now and a better graphics card in the future. I don’t know how much I spent on that module but I vividly remember it was a lot for me back then as a 13-year-old.

Turns out that guy knew what he was talking about, doubling up on RAM made a big difference! I’d now have ~20 FPS and could actually play the game I was so excited about.

A month later I found a 35$ (and also infamous) Geforce FX 5500 at an electronics store and bought it immediately. It had twice the VRAM and it was bigger! Naively thinking that I had struck a bargain and would go home, put it in, turn up all the graphical settings on all my games and just go

Yea. It didn’t happen. Still had too little RAM, my CPU was a slow singlecore and I just upgraded to my second mistake.

It was an improvement from the Geforce4 MX but not a whole lot more. Turns out the worst cards Nvidia ever released up to that point were the Geforce 4 series and the FX 5000 series :) A year later Oblivion came out and the FX 5500 didn’t cut it, went back to the computer shop and got a AMD Radeon HD 2600 PRO because it still worked with the AGP bus. Yes, I wasn’t even using PCIe yet…

Tinkering with my computer taught me a lot though, I’d open it up, tear it apart and reassemble it, try to run it without all the components, see what happens, change out parts with stuff I sourced from the dumpsters, breaking Windows XP once a month and reinstalling it, buying books about computers or going to the local community center for internet access as I didn’t have any access to the internet at home until next year, try to squeeze out every single bit of performance from this little shitty slow computer was also highly rewarding. I’d obsess over a point increase in benchmark scores, I’d overvolt and overclock my CPU with a pencil - nowadays people increase a multiplayer in the bios and think they’re overclockers - disgusting XD - and disable windows services, deleting protected system files to free up some space on my 80GB IDE HDD that sounded like a peppermill being thrown down the stairs when it was doing something .. anything..

There was no turning back now.

Modding

If you didn’t have internet or money, you had to spend a lot of your time with the same things. When I got bored, I couldn’t watch a YouTube video - YouTube didn’t exist - so what do you do? Well, for me it was either going out, walking to friends houses and asking them if they wanted to come outside and do something because - you guessed it - we didn’t have phones either. Also most of the time, you’d go from friend to friend to friend to friend, often walking for hours, through the entire city till you find someone who has time and can be fucking bothered to come out. I’d always try to plan my routes before embarking and it never felt like I had taken the best route. I hadn’t heard of the traveling salesman problem yet. The alternative would have been playing old games or making my own maps, items, NPCs, and Quests in the original Neverwinter Nights, the only game I owned that shipped with modding tools.

I thought I could negotiate more computer time with the lifegivers by showing them productivity. A futile endeavour

I spent countless hours with the NWEditor, I still remember vividly, sitting in my room, building my first city, populating it with NPC’s, setting waypoints for each of them, making them have a daily routine, loads of dialog and quests …. with rewards that broke the entire game cause I got to set the stats of the items too… That was the first time power corrupted me and turned me into a maniac. - all while listening to ‘Ridin dirty’ by Chamillionaire :D

Later I’d buy GameMaker and FPS Maker (which is now Open Source!) trying to make something cool I could show off, but that never happened, GameMaker was too complicated and FPS Maker was the most hideous thing i’ve ever seen and felt terrible. I remember building a D-Day map with hordes of enemies that’d never stop, and you’d have to protect your bunkers for as long as you could. It would work for two waves and then nothing would spawn. Never figured out what was wrong.

Turns out making games is hard and a thirteen-year-old without any programming experience can’t do it.

The Internet

Prepaid 3G Internet launched. No contract, no need to be an adult, just walk to the store, get a starter kit with USB Modem and SIM card, plug it in, install the software, dial in and surf. It was amazing. Finally I could play multiplayer games over the internet, finally I could look at websites and see what’s on the internet, download free software, free games, discover piracy, download more cough free stuff,…

Fun fact: my 3G internet was a breathtaking 1mbits download (110kb/s) and I paid 25$ per Gigabyte - exactly how much money I got from my granny a month

Eventually I’ve started playing a random Chinese MMORPG called Conquer Online, my first MMORPG experience. I spent so much time on this grindy free to play & pay to win game. Conquer reminded me of Diablo and Sacred even a tiny bit about Neverwinter, so I was immediately in love with it and ignored all the pay to win stuff. Then one day another one of life’s coincidences: Googling around for Conquer leveling bots and hacks, I came around a game-hacking forum and on it I’d find something that’d change my life forever.

This game actually had a pretty new private server community and there were private server source codes available for download.