More on the UI

Duh, silly me, in my previous post I forgot that my routines in C were actually not in use anymore and was all pure python.

Then, I had some spare time to play around and get some hard data on my experiment, first let's describe a bit how that UI was done: basically win32 api's, it uses a LayeredWindow and AlphaBlend calls, this is because I wanted to see how far I could push it with plain old but well supported api's, also I wanted to have full per pixel alpha support on the root window (not just a global alpha value)
Another thing to consider, everything you saw in the video is a window (not a native Window, but my own defined window, again in python, the only real hwnd is of the root window): the title bar, each button, the background, even the scrollbars are composed of windows: the arrow buttons and the sliding block, all this was intentional as to exercise child windows and not a plain image with trickery.

In my notebook (i7 with a Radeon) the window that appears in the video is able to archive about 60 fps when resizing (just a small change in it to force a full redraw) with a close size to 1366x768, this is with no alpha pixels in the background while there are ones in the scrollbar.
Performance gets tricky here, if most of the background has alpha value (so you can see beneath it) then performance varies depending on how many pixels, it went as low as 45 fps.

Then, I made another 'driver' to suppress the alpha blending at all, so everything is opaque, while still paints over a LayeredWindow all the image ops are FillRect and BitBlt, also the call to UpdateLayeredWindow using ULW_OPAQUE, at this point the frame rate went all the way up to 120 fps, quite a speed boost.

Yet, I should try (whenever I can find the time) to use one driven by directx and possibly opengl too, i'm not too anxious to try the opengl one but I may do so.

Having said all that, it is important to remember that 60 fps is a lot for a commercial application (not for a game) and that 99.9% of the time for an application the redraw is usually a minimal part of the window.

Well then, I'm not sure if / when I'll continue with this experiment, i'm itchy about knowing the potential FPS with directx but no promises, I may or may not try it out

Hope it is of any use to some adventurous soul

-Mat

The (sorry) state of UI and some hope for the future

The user interface, the face we show to the users, our presentation card, nobody would argue that it is important right?
However, if we pay close attention to the state of it we can easily see lots of flaws, over time (in my case since the '80) many of us have even become blind to it's defects.

So, before I continue I should warn you: if you keep reading you may actually end up seeing (or I should rather say 'be more aware') many defects and imperfections in the software you use everyday, so don't blame me later if you feel displeased with your everyday interaction with your computer :)

And also, please keep in mind this applies mostly to Windows, Linux and to some extent Mac, ios and android suffers much less this issues.

Let's start our journey then, just open notepad, yes, you know what i'm talking about, everybody has it (or use gedit in linux), then resize let's say half the size of the screen should suffice.
Now, grab the top-left corner and resize it, moving it constantly to the top left corner of the screen and then to the bottom right corner, the purpose is to continuously resize the window and see how it is drawn, dont release the mouse button, just keep doing the movement and focus your view into the lower right corner of the notepad window.

If you're in windows, you can see the right scrollbar histerically jumping as we resize
If you're in linux (somewhat old versions of gnome) you'll see the same behavior, the drawing quite lagging and at a quite different possition from the mouse
If you're in linux latest gnome, it gets a bit better, yet you can easily see 'tearing' and lagging on the paint and mouse

Now this is not a notepad thing, it happens with all applications I've used, there are even worst offenders than the simple notepad, while one could argue that users are not all the time resizing their windows (or even doing it from the top left corner) it is an easy way to expose a drawing deficiency that affects in other more subtle ways.

If you do the same with the lower right corner you can see the same problem, but it is harder to see because your eyes have to track a moving target, in any case the problem is still there and still visible.

Another relatively easy flickering to spot: put some content in notepad (copy a somewhat long text into notepad), maximize notepad, then open the search dialog (ctrl + f) and close it with the escape key, do that repeatedly and you can see after a few tries that the text in notepad flashes (internally it clears the window and then paint the text back on top of it)

Another easy one? open system properties from control panel, then switch constantly between the tabs of the dialog, sometimes the painting produces a flickering.

More advanced ones? I tried yesterday with google chrome, resizing it from the lower left corner (just grab the lower left corner and bounce it around for a few seconds) and it was freezing the UI for a second or two (if there was some page loading or so)

Ok ok, there are many more presentation / drawing / transitioning glitches around but you get the idea right? so what?

Well one could argue that Mac are better at it, while they're better at UI and usability they still have their pile of UI glitches and they're bad at everything else, while Windows is better at many other things there's clearly a lot of user interface debt they have to pay, on the linux land... well, my experience has been quite bad and varied but there are tons of different windowing engines / managers / drivers and so all in different maturity, havent found yet one that does justice to today machine power capabilities

With all this I started my journey into what can be done about it, focusing mostly on windows I spent a horrible amount of time poking around ways to reduce this visual artifacts with mixed but all unsatisfactory degree of success, the problems in windows lies deep within the operating system, no matter what trick I tried.
Among other things I tried plain old win 32 apis, mfc, windows forms, WPF, java, even as far as to also use QT and GTK, none of them worked as I expected on this regard, it let me wonder why after all this years why Microsoft didn't-doesn't care / fixed (I checked this issues all the way back to windows 95, and yes, even checked it on windows 10)
After all, will you buy a car with the paint scratched or with some rust? or would you rather buy the car next to it with better / shinier paint free of rust? surely the power given by their dominance affect where they spent their efforts but man... I cannot praise or justify sloppy work on the 'presentation'.

Anyway after my long journey (countless hours of testing, prototyping and googling) I ended up going to the very basics: let's see how a modern windowing system could do this better, and did that.

My experiment consisted into relying only in a core, flat window and then do everything else by myself, effectively creating child windows, graphics primitives and all that within  my library, very similar to what Java did internally, it ended up performing like this: demo
Be warned, the quality of the filming is rather poor :)

So what was that? well, a basic windowing system that supports from the very core 32 bits per pixel alpha channel windows with flicker free and tear free drawing - effortlessly, meaning the developer doesn't need to do anything to get it 'right', displaying, moving, re-sizing, panel switching and so are truly rendered like if it is a game, it is of course modeled after what a modern computer is capable of.

Most surprising (for some of you) it is entirely coded in python 2.7 (except for 2 functions coded in c: fill and fill_rect), given the poor performance python has and the high speed archived in the demo you can easily see how feasible is to do this in a high performance language like c / c++, as my purpose was to demonstrate the feasibility it was just a great language to prototype it.

I'll be closing this post as it is getting quite long already, got some interesting insights into how a modern windowing library could be approached and I'd like to write and discuss about some of the aspects of it but that may happen in a future post. for now i'm signing off

-Mat

The poor man's garbage collector

While c++ is not my everyday thing, every time I take on it I miss some high level features like a garbage collector. Indeed it is a very polarizing topic for c++.
Smart pointers do come to give some help but they annoy me as they've their own rules to follow and they re not that simple or straightforward to use as a full-fledge gc

This weekend I had a bit of inspiration on it and come up with something to try, but be warned: I am still validating the concept, looking at different angles and frankly not entirely sure it works ... yet :)

So let's start with the basic: reference counters and the cycle problem, I wont go into details, there's plenty of reading around about how the reference counting work and the cycle problem. In my experiment I take them as the starting point, let's call my smart pointer GcPtr (which is as you guessed a c++ template, very much likely the existing ones in the c++ libraries).

It works as you may expect, so in order to break cycles I introduce the concept of 'Context', what is it? well just the function body where you are, in code a function looks like:

void do_something_interesting()
{
    Context context;
    GcPtr<Node> node(new Node());
        //a reference to itself is the most basic case of a cycle
    node->set_parent(node);
}


Well, that's not enough for showing much, but we see that starting a function with a context object allocated in the stack gives me the possibility to execute something in the destructor of the context, just when the function is going to go out of scope.
The idea here is that the GcPtr gets attached to the active context, hence if the context is destroyed everything attached to it becomes unreachable, if it wasn't claimed due to its reference count getting to 0 then it can still be reclaimed because it goes out of the context where it was allocated.
Yes, that is part of the story but not all, because a pointer can be 'passed down' to another function / level / context, let's see a more complex example:

void create_child(GcPtr<Node>& parent)
    Context context;
    GcPtr<Node> child(new Node());
    child->set_parent(parent);
}

void do_something_interesting()
{
    Context context;
    GcPtr<Node> node(new Node());
        create_child(node);
}
 
Now we got a parent-child circular reference, child was created in context 2, while node was created in context 1.
What happens internally is that inside the node class we have:

void Node::set_parent(GcPtr<Node>& parent)
{
    m_parent = parent
    parent->set_child(GcPtr<Node>(this))
    ...
 
void Node::set_child(GcPtr<Node>& child)
{
    m_child = child;
    ...

What happens here is subtle, m_child was attached to context 1 (it was created in do_something_interesting) so when it is assigned something attached to context 2 it gets promoted to context 1.
Again, child was created in context 2, but at the point we assign something that is reachable from context 1, it gets promoted, meaning context 2 does not know of it anymore and is added to context 1
Because of that, after returning from create_child, the child would still be alive (there is one reference in the node instance that references it)
Finally, after context 1 is destroyed, even as the reference count is 1 for node and 1 for child, it is destroyed as it leaves the context is attached too.

What would be the rules for using this smart pointer? pretty simple:

• keep references inside of GcPtr
• If you're creating objects in a function / method, start the function with a context creation as above
• dont touch GcPtr's in your destructors (as a rule of thumbs, I need to think it more deeply)

Neat? maybe, I did a proof of concept and works like a charm, on the positive side it is very deterministic, it does not rely on background threads neither stop-the-world syndromes as some complex GC do, there's nothing complicated like generations and so, the extra overhead is quite small.
It is still possible if desired to offload the deallocation to a background thread but that is in itself a tradeoff that will benefit some cases while punish others so it is not a win-win thing.
Another beauty is that you can go selectively using it, it is not an all-or-nothing approach.

Yet, as I mentioned before, I still have to look at it from different edges and angles, I'm not personally convinced it will handle all the cases, what if for example in the above example childs was a vector? how would the smart pointer realize the right context? not sure yet, and an elegant solution needs more poking around...

I'll publish the example code I created later on, it is just quite messy at the moment and needs heavy massaging but so far is about 50 lines of code or so

If you happen to have something to point out please do! I'm sure I'm missing stuff out.

-Mat


UPDATE:

Heh, too good to be true, not long after I wrote I realized that a class that holds a GcPtr would have to have the lowest context of any of it's reachable fields, so it still needs to climb up and traverse a chain of references, while it does not invalidate the whole concept it does put more complexity, it starts to look like a class will need a reserved field _base_context or so.
Well then, as I was suspected it was too simple to be fully working, surely I'm still missing something else.