Almost two years ago I wrote a blog post about checkpoints in ZFS. I didn’t hide that I was a big fan of them. That said, after those two years, I still feel that there are underappreciated features in the ZFS world, so I decided to do something about that.

We already learned a lot about Express.JS, although, Gynvael surprised us with another exciting challenge. A few more and we may learn JavaScript (just kidding nobody knows it).

Gynvael published another mind-blowing Express.JS challenge last Wednesday. Let's keep our right track and try to solve it. Are you ready?

In the last couple of days, Gynvael Coldwind tweeted out three exciting challenges. All the challenges used the Python and Flask or Express framework implemented in node.js. I decided to accept his challenge, and after struggling a little bit, I was able to solve all of them. In this post (or write-up), we will go over them together.

Some time ago, Michał Bentkowski published an XSS challenge on his twitter. I accepted the challenge, and here is my story.

Some time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features.

One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way.

I don’t use Windows often: mostly only at work to test protocols. However, I have now started playing a little bit more with it for developmental purposes. So, I installed a Visual Studio and I have to admit that this IDE is pretty good. But this isn’t why we are here. Recently, I needed to write a very small debugger for Windows in C language. This post documents how to do that, and a small obnoxious behavior which cost me a lot of time.

Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device.

I recently was looking at some old games. Really old games. Games that were running on the DOS. When I grabbed a random game, it turns out that it was built using the DOS4GW extender.
When I was young (almost 10 years ago) I helped a little bit with porting swars to the modern OS. When I looked around I couldn’t find any free loader for the LE/LX format, so I decided to create one.

Today, this post is more like a very quick note about a funny error message in the ZFS. ZFS is amazing file systems. I often mention why it is so powerful here on the blog. However, even ZFS can sometimes give us unlogic messages. Today we will look into one of them.

A network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump.

… or the story what happens when you forget that this supposed to be a simple task. Last weekend I had some time to look into the Google CTF for Beginners. I solved all the task apart from one in category “sandbox” which went quite interestingly. Basically, I misread a few stuff and went way over the solution. Let’s look into my story…

One of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot?

I use ZFS extensively. ZFS is my favorite file system. I write articles and give lectures about it. I work with it every day. In traditional file systems we use df(1) to determine free space on partitions. We can also use du(1) to count the size of the files in the directory. But it’s different on ZFS and this is the most confusing thing EVER. I always forget which tool reports what disk space usage! Every time somebody asks me, I need to google it. For this reason I decided to document it here - for myself - because if I can’t remember it at least I will not need to google it, as it will be on my blog, but maybe you will also benefit from this blog post if you have the same problem or you are starting your journey with ZFS.

I would like to invite you to our monthly Polish BSD User group meetup. We have had eleven meetings, with the group running for almost a year now! A lot of interesting topics have been covered during that time and I’m sure this will continue in April!

In operating systems there are often long-term problems. One of them was described by lcamtuf in 2002 (this is the earliest mention I was able to find, and this article motivated me to do the work). For many years we didn’t have a way to remove files in race free way. Let’s first look closer at the problem.

You have a few different ways to upgrade your operating system. For example, if you are using ZFS as your root file system you can use boot environments to accomplish this task in a quick, simple and safe way. Boot environments are the new ‘it’ thing in FreeBSD. However, today we will discuss a more old-school approach to this problem using the GPT flags. This approach may be used to upgrade your operating system if you don't use ZFS, and you want your system to be on the read-only partition, for example when you are building an appliance.

In January it’s time for our 9th Polish BSD User Group!

This time I would like to invite you to a special edition in which instead of three different 15 minute talks, we will have an hour-long tutorial. The whole meeting will be about FreeBSD jails, in particular about managing them with iocage.

Wiele lat temu zafascynowałem się systemem plików ZFS. W 2015 roku popełniłem artykuł na łamach świetnego polskiego czasopisma - Programista. Minęło już kilka dobrych lat od publikacji więc czas się nim podzielić na moim blogu :)

Artykuł ma już kilka lat natomiast uważam że wiedza w nim zawarta jest wciąż aktualna. Mój artykuł opisuje podstawy ZFS - czym jest, mechanizmy ZFSa oraz API dostępne dla programistów.

Throughout December I took a break from writing blog posts, but New Year is a good moment to get back to it. Today will be a very personal post in which I will tell you a summary of my accomplishments over the previous year. I think I was quite productive. Do you?

What do all programmers like to do after work? Ok, what do most programers like to do after work? The answer is simple: play a good game! Recently at the Polish BSD User Group meetup mulander was telling us how you can play games on OpenBSD. Today let’s discuss how this looks in the FreeBSD world using the “server only” operating system.

I spent a couple of years evangelizing about Capsicum. I wrote many articles about it. So, it is very natural that I would also like to update you on this blog about the progress of the Capsicum project in FreeBSD, because this is what I’m doing in my free time. That said I feel that this blog wouldn’t be completed without some introduction to what Capsicum is. This post should fill this gap. Over the next weeks and months we will extend this topic and discuss different parts of Capsicum. Without further introduction let’s jump into the topic.

Assertions are very important for developers, because they help developers to find bugs. This concept should be used to check that the internal state of our application is correct. Some developers even measure a quality of the project/code based on the amount of assertions. In many programs assertions are enabled only in developer modes and not in production. Programmers should always remember that assertion can’t be used to validate user input because it can introduce some bugs into the code. Although this topic is very interesting this is not what we will talk about today. Today let’s discuss how to make our assertions be even more useful.

In a couple days there will be another meeting of the Polish BSD User Group! It’s already our 6th meeting, which means that the group has been running for almost half a year. The next meeting will be a special one, because we will be guests at Warsaw University of Technology! Considering all of this I decided to summarize our accomplishments.

It’s amazing what happens when you do any kind of research. Many times, you find a few interesting things around one topic you are looking into. First, I published a post about changing a process name which describes how it can be accomplished on different operating systems. This was the main point of my research. Then I described how to install other operating systems using grub2-bhyve on bhyve hypervisors. Today, the last topic is somewhat related to them. When I was looking at the process title it was easy to notice that the kernel needed to read some piece of memory from it. What would happen if the memory region didn’t exist?

When I was writing a blog post about the process title, I needed a couple of virtual machines with OpenBSD, NetBSD, and Ubuntu. Before that day I mainly used FreeBSD and Windows with bhyve. I spent some time trying to set up an OpenBSD using bhyve and UEFI as described here. I had numerous problems trying to use it, and this was the day I discovered the grub2-bhyve tool, and I love it!

In October there will be two very interesting conferences at which I'm honored to give a talk.

Let's start by defining what the title of a process is - in this article, we will understand the name reported by ps(1)/top(1). When we are creating a new process using fork(2) the process inherits the name from its parent. In the scenario when we call exec(2) function we also pass the list of the arguments for a process, which will be treated as a process title. What if we would like to change the title of the process when it's running? It turns out that many operating systems do it in a different way. In this article we will discuss how open source operating systems like FreeBSD, Linux, OpenBSD, NetBSD, and DragonFlyBSD do it.

Those days locking algorithms are critical for operating systems, especially in a multi-threaded world. With time it turns out that classical locks, like mutex, are performance costly. Even when we are using techniques like having reads and writes mutex, synchronizing the state between CPUs can be costly. To optimize some cases, the lockless algorithm started to be used.

Many years ago, I was very interested in creating games. Unfortunately, I resigned from this path for my career. When I look back, this was one of the best methods for me to learn to programme. After many lines of codes, you are getting something that you can show anybody, and they will understand what you created which was very motivating. There are many, many algorithms which you must understand and implement. In my time, when web games weren’t yet very popular, this was also a great environment for me to learn C++ and C - which I use today. On the other hand, the language doesn't really matter - if you want to learn a new language why not to write a game in it. Today almost every language has graphics libraries you can use. Don't limit yourself and don't be afraid to scratch your own framework. If you are looking for something to implement and you are new in computer science, with a clear conscience, I recommend you create a remake of your favorite game.

When you build a new service (or an appliance) you need your users to be able to configure it from the command line. To accomplish this you can create system accounts for all registered users in your service and assign them a special login shell which provides such limited functionality. This can be painful if you have a dynamic user database.

The nvlist is a very useful container but, sometimes, using the name of an element to fetch it may be less than optimal, or even impossible. Let us consider two scenarios:
- We are traversing a list which was created with an NV_FLAG_NO_UNIQUE flag.
- We are traversing very large list.

In March, to FreeBSD landed a very interesting feature called 'zpool checkpoints'. Before we jump straight into the topic, let's take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data.

Nvlist allows us to keep pairs (name, value). The question for today is what if the name of an element is unknown? Or what if there is a need to traverse over all elements on a list? First of all there is the nvlist_next() function. This function allows to iterate over one single list. The return value is the name of the next element on the list or NULL if the list is ends. The additional information which is returned by this function is the type of the element. Thanks to these two values (the name of the element and it’s type) we can decide which function to use. The last element is secret cookie which refers to the position where we are on the list. At the beginning the cookie needs to be initialized by NULL otherwise the program will behave abnormally.

One of the assumptions of nvlist is that during fetch functions (like get or take) an application knows what elements are on the list, otherwise read operations fail and the application will be aborted. If an element is optional then it is necessary to check if the element exists using the exist functions.

Nvlist is an implementation of a general purpose container based on a list which contains pairs (name, value). The idea behind this library is borrowed from the Solaris library which has the same name. In Solaris nvlist is used for example, by the ZFS. In FreeBSD nvlist exists in kernel and userland. Nvlist was implemented to provide an easy inter-process communication framework for applications that will be privileged separated and later sandboxed, due to this in userland there are functions allowing us to send nvlist over the Unix domain socket, in that case there is also a possibility to send descriptors with nvlist.

Recently one of my friends had a strange problem - he wanted to initialize a few variables with a different type in a single for loop. What was even more strange was that he wanted to declare just one of them and both of them have different type. A declaration of variables is forbidden in the loop for C standard which are older than C99, so let’s assume we are using exactly that standard. One of the questions could be “why is your friend so crazy that he would want to do that” - basically he wanted to create a macro which would declare for him a head of the loop very similar to what STAILQ_FOREACH.

I had the pleasure of giving talks at two great BSD conferences - BSDCan and AsiaBSDcon. The first one was held in June at the University of Ottawa, Canada and the second one was in March at the Tokyo University of Science, Japan.

bhyve is a hypervisor for FreeBSD. A few weeks ago I gave it a try and I must admit I was very impressed, so much that I decided to write a short tutorial on how to configure bhyve to boot from NFS filesystem. So lets start.

I'm a fresh FreeBSD committer who is very interested in security things. I also work for the Wheel Systems company which develops security solutions. So it was natural for me that I should attend Cambridge Developer Summit which, in my opinion, is the most security related event in every committer’s calendar. This was also my third visit to Cambridge. For the first one I also wrote a trip report which you can find here. The conference was held in August 17-20, 2015.

To już ponad miesiąc po IGK'2014. Czas ożywić bloga krótką relacją z tego wydarzenia. Jak co roku w Siedlcach w Instytucie Informatyki na Wydziale Nauk Ścisłych Uniwersytetu Przyrodniczo-Humanistycznego odbyła się konferencja poświęconej "inżynierii gier komputerowych". Miałem okazję być na tej konferencji już po raz siódmy.

Firstly what is FreeBSD diskless? FreeBSD diskless is an operating system that will boot on your computer (or virtual machine) over the network. There are many situations in which you probably want to use such a configuration. For example if you are a computer science teacher you could configure one system in your class and make other computers use it over the network. In another situation you could need some laboratory to play a little with the FreeBSD kernel.This configuration has a lot of benefits. For example you don't have to restart your developer machine every time you compile the kernel or worry that you broke your kernel and you will not have a developer machine any more. If you know me a little you probably know that I'm not a computer science teacher.

I haven't forgotten about this place, but also I didn't think that writing technical post would be so hard in English. I hope I get some practice and will become much better. Today is the perfect day to write a new blog post.

Ugh, zaczyna się robić tłoczno, drugi post w przeciągu dwóch miesięcy, nie jest źle ;](warto dodac że post pojawia się za namową pewnej osoby (Pozdro Gyn ;>))... Dzisiaj opisze projekt nad którym ostatnio pracowałem. Pomysł użycia tej technologi chodził za mną już od igk ale dopiero ostatnio zrobiłem coś co by się nadawało do pokazania publicznie.