Resizing a DirectX rendering window

Nov 1, 2011 at 6:14 PM

The demos are using a fixed size, not resizable window. This is not what most users are expecting.

Resizing a window in a normal application is easy, just a matter of flags. How is it done in Hieriglyph3 ? I see the the window is encapsulated in Win32RenderWindow class but I don't see anything related to Windows messages processing. The traditional message loop is located in main.cpp, and as usual, the window procedure (WindowProc) is there. It dispatches a few message and call ProcessEvent. I have not found anything handy to extend this to handling more messages such as WM_SIZE.

Once I understand how to let the user resize the window, how do I tell the shader that the window size has changed and the rendered image has to be recalculated (I mean the user see more or less of the initial image. In other words the image is not zoomed in or out but show a larger or smaller part of the whole picture. btw: I'm not building a game but a radiology imaging software).

-- Francois Piette

 
Coordinator
Nov 1, 2011 at 7:40 PM

The handling of a window resize event is actually something I have been considering in the past few days.  It would be trivial to create a new event type for the WM_SIZE message, and then to have the application receive the message to trigger some logic.  The issue is that each application will handle this event in its own way, depending on the rendering configuration that is being used.  Since Hieroglyph 3 allows for very different varieties of rendering setups (i.e. deferred rendering, light prepass rendering, forward rendering...) then there is no one size fits all handling of the event.

However, I am currently planning out how to add a method that will allow an application to handle this on its own.  It will require that the swap chain for the window be resized accordingly, plus any other resources that are dependent on the screen size will need to be resized as well.  That would also mean the viewport sizes need to be updated dynamically too.  This will likely be introduced soon, but I can't make any promises about how long it will take (there are lots of other things already queued up...).

As far as the view that is held within the window, that is all determined by the contents of the scene (which in your case would just be an image) and the view/projection matrices used to put it into the rendered output.  Those can be updated as well in the resizing event handler, but would also be specialized for each rendering scenario.  I think that won't be too hard to update once the framework is in place for handling a resize.

Nov 2, 2011 at 8:50 PM

Thanks Jason.

I don't expect the framework to do automatic action about window resizing. As you mention, most of the work is application specific.

Regarding how Windows messages are handled in Hiergoplyph3, I think you are on the wrong track. Converting each message to a specific class instance maybe good in an OOP point of view, but it is difficult to expand and result in a performance penality.

I would suggest another design: The application would register each Windows message it want to handle (for example WM_SIZE). registering shall be done supplying the message number (WM_SIZE) and a method pointer (a callback function which is a method of an object). This method takes 4 arguments: the window handle, the message number, WParam and LParam, and returns and LResult. In short, it is a WindowProc.

The framework has to maintain a list of message number and corresponding method pointer so that it is able to call the required method when the message is pumped out of the queue. The actual WindowProc in main.cpp will lookup the list of message and call the appropriate method if any, and call DefWindowProc if the message is not registered.

If the framework has something to do with some of the messages, then it is easy to add it before or after the method registered by the developer. We could also imagine a 5th argument to the methode registered by the developer: it would contain the pointer to the method the framework would do if the message has not been registered. The developer has the resposability to call it before or after his own work, or not call it at all because it has replaced the work to be done (this mimics a virtual method overriden in a derived class).

-- Francois Piette

Coordinator
Nov 2, 2011 at 9:49 PM

Hi Francois,

Thank you for the suggestions.  However, I think our two solutions are not too far apart...  The event handling is indeed already done by means of registration for a desired event type.  However, the event system is not only used for windows messages - it also allows for custom messages defined only by and for the framework, such as a 'frame started' event.  To support such a variety of event types, it makes sense to have the events provide their own data, and hence the need for a class to define that data and provide accessors for it.  Other than having a class to hold the data, the EventManager class is performing the registration, lookup, and dispatch activities that you describe above.

One could argue that we could use the WindowProc directly for all event receivers, but then objects that don't need to know anything about a windowing system would be forced to include windows declarations...  Thus I feel that my implementation provides flexibility to define its own events, type safety for the data being passed, and the flexibility to have any class receive any events that it is registered for.  The penalty is that I am allocating a new message object for each event being sent, but in any profiling that I have done the event system is nowhere near a bottleneck.  So for now, I think I will leave the system as it is, and if need be later on I could optimize the system to allow for statically allocated event objects that could be recycled instead of allocating/deallocating them.

Even so, I enjoy discussing the design concepts, so if you have further design ideas or improvements (or think there is something I am overlooking) then please let me know!

- Jason

Coordinator
Nov 2, 2011 at 11:14 PM

One other thing - I am in the process of updating the system to properly respond to window resizing.  This was already in the works to handle a change to full screen mode, and is requiring some refactoring of the application classes.  I'm approximately half way through now, and should be wrapping it up in the next couple of days.  This will allow the swap chains for the windows to be resized, providing pixel perfect rendering, regardless of the window size.  Of course, I will post here once the change has been pushed to the repository!

Nov 3, 2011 at 7:04 AM

Hi Jason,

Translating Windows events to objects is not practical because there are so much Windows messages.  You cannot practically create a class for each one and obviously you can't expand your switch statement to include every possible message. And of course you cannot handle dynamically created messages.

Messages are used everywere in Windows API, not only for UI but also for all kind of notifications. An application can also use messages to have an asynchonous behaviour - event driven - and yet avoid the complexities of multithreading where it is actually not required.

About resizing: I don't meant only going to full screen which is only a special case. Any size sould be supported. If things are automated, the developer should be able to set the min and max sizes of a window (there is a Windows message for that).

Anything that is automated should have hooks so that the developer could override the standard behaviour.

One last thing: The window used for rendering is not necessary the main window. Actually in all my applications, the rendering window is just a child window of the application window. And there could be several rendering windows for image comparison. Basically, the main application screen is divide in regions: the top has a tool bar full of icons, the left side has a pipeline of image thumbs, the bottom has a status panel, the right side has an outlook style tools menu and the center is the rendering region with one, two or four images in varying layout.

-- Francois Piette

 

Coordinator
Nov 3, 2011 at 6:56 PM

Hi Francois,

I still fail to see the large difference - any message that your application is going to utilize has to be handled somewhere.  In Hieroglyph 3, it is handled in the main WindowProc and then converted into an Event so that it can be distributed within the engine along with other events that are only defined for internal use.  In your suggested implementation, there has to be a registration mechanism somewhere in the main WindowProc that checks to see if there are any consumers for a message.  This means that all messages are being treated somehow, and then dispatched or not depending on if a handler is there.  The only difference is that I create a class which interprets the event data for me in a type safe way.

This is actually why I think it is not only convenient to use a class to wrap the events, but it is also safer and easier to maintain.  The message (which consists of very basic data types) data is only processed in one place, and hence allows many classes to interpret the data in the exact same way.  If you don't wrap the message, then there will be lots of places that need to interpret the data which isn't so easy to maintain.  Especially when talking about void* data, this is especially treacherous...

Regarding the number of messages, I don't see this as a problem.  I currently have the messages being handled that I use, and the remainder are passed to the default window processor.  If it turns out that I really need to support all possible messages, I can just create a generic an instance of EvtWindowMsg and use it to pass the parameters that you mentioned (the standard windows data).  Then any event receivers that are interested in seeing all messages can register in the EventManager to receive those messages.  This would also handle the switch statement size issue - although I don't think having a large switch statement is really a problem either... perhaps it isn't so nice to look at, but that is sometimes a concern I suppose.

About resizing: I know what you are talking about - I am implementing a resizable window where the users can drag the edges of the window to make it bigger or smaller.   This is actually implemented now, and I am just working out the specifics of how to update all of the GPU resources to match the new sizes, and then I need to update each of the sample applications to utilize this system.  The mention of full screen is just that it changes the screen resolution, so it has to be handled in the same way.

And finally about the main window vs. child window: there is no concept of a main window in Hieroglyph 3 - there is the ability to make as many windows as you want, and the internal classes essentially just require an HWND to render into.  This means you can use child windows, sister windows, or what you want.  As long as you have an HWND for it, then you should be able to use the rendering system to render into it!

- Jason

Nov 3, 2011 at 10:10 PM

About messages:

What I was trying to tell you is that you cannot create a class for each message (64K messages) and you can't have a switch statement with all the messages. There are a lot (potentially) of messages which are purely generated by the application. All message have always to be handled, if no one is willing to handle it, then DefWindowProc has to be called.

Using a generic EvtWindowMsg is IMO not a good solution. It will considerably slow down processing and won't have any added value because you don't know the structure of the data used in the message.

Don't forget that you also have dynamic messages using RegisterWindowMessage API function. This is very useful in library or component code where you need a unique message number for a given message queue.

It is VERY important to have the fastest possible "message pump" and message handling because you may have several THOUSANDS messages per second because messages are used for notifications. For example you may use messages to handle socket communication, or wave input and many other high speed I/O having notifications.

About resizing: OK, perfect.

About main versus child window: OK. Don't forget that only the main window should call PostQuitMessage on destroy.

Thanks for your patience with me.

-- Francois Piette

 

Dec 3, 2011 at 8:02 PM
jzink wrote:

About resizing: I know what you are talking about - I am implementing a resizable window where the users can drag the edges of the window to make it bigger or smaller.   This is actually implemented now, and I am just working out the specifics of how to update all of the GPU resources to match the new sizes, and then I need to update each of the sample applications to utilize this system.  The mention of full screen is just that it changes the screen resolution, so it has to be handled in the same way.

Hello Jason,

Did you succeeded in implementing this resize feature ? If you did, let me know which one of the samples is demonstrating it.

Thanks

-- Francois Piette

Coordinator
Dec 3, 2011 at 9:23 PM

The structure for resizing has been implemented generically in the RenderApplication base class, so theoretically any of the samples that utilize this base class should support resizing.  However, the resizing relies on the render view being used for the main rendering of the application to resize its resources in order to get an appropriate update in the size of the swap chain, depth/stencil resources, and any additional render targets.  Support has already been added to the renderer in order to do this resizing of the resources in place without the application doing anything special.

Currently only the ViewPerspective render view does this properly.  So any of the RenderApplication based samples that use specialized render views to perform their resizing don't currently handle it properly.  However, you can take a look at the WaterSimulation sample to see one working example of the whole setup.  The others will be updated fairly soon, probably after I wrap up my current commit.

Dec 4, 2011 at 2:10 PM

Hello Jason,

I think I'm starting to grasp the resizing system you've setup.

In my application I have a resource which needs to be somewhat updated when the window is resized since some of it's parameters depends on the window's size. Here is the code I use to create the resource. Actually, it is an image in a special file format irrelevant here. The image is uRawBitmapSizeX x uRawBitmapSizeY (1920 x 1536) pixel which is converted into a float by code before the lines I show. 

	Texture2dConfigDX11 IbnConfig;
	IbnConfig.SetColorBuffer( uWindowWidth, uWindowHeight ); 
	IbnConfig.SetFormat(DXGI_FORMAT_R32G32B32A32_FLOAT);
	IbnConfig.SetBindFlags( D3D11_BIND_UNORDERED_ACCESS | D3D11_BIND_SHADER_RESOURCE );

	D3D11_SUBRESOURCE_DATA SubResData;
	SubResData.SysMemPitch = sizeof(m_IbnImage->RawDataFloat4[0]) * uRawBitmapSizeX; 
	SubResData.pSysMem = m_IbnImage->RawDataFloat4; 
	SubResData.SysMemSlicePitch = 0; // No meaning for 2D

	m_Texture[0] = m_pRenderer11->CreateTexture2D( &IbnConfig, &SubResData );

The resource contain the full image I would like to update the resource "color buffer" without recreating the resource which would involve copying a lot of data from CPU memory to GPU memory (3 million float numbers !). How can I do that ?

Thanks for your help.

-- Francois Piette

 

 

 

Coordinator
Dec 4, 2011 at 2:56 PM

If you have a large resource already residing in GPU memory, and you want to resize the resource without copying the data again from the CPU to the GPU, then there are a couple of ideas that I could suggest.  First, you can create a secondary resource on the GPU, then copy directly from the one to the other.  This would eliminate any CPU traffic, and would allow you to easily resize the buffer as needed.  The second way would be to keep the image at max resolution, then just use mip-maps of the texture to produce smaller versions of it.  Then depending on the size required, you would simply create a separate shader resource view that selects the appropriate mip-map in your shader.

As far as I know, there is no way to resize the texture in place with its contents staying in them.  However, I think either of those solutions should be usable in your use case (the texture being the primarily viewed surface).

Dec 4, 2011 at 3:34 PM

Well, I probably wrongly expressed the situation. The resource I'm talking about (the image) is idenependent of the window size, exactly as in your second way of doing. The proble is that the code creating the resource (remember we discussed about it in another discussion) is dependend on the window size (Call to SetColorBuffer). This is the only things which change when window size change. My question was: how - from the resize event - do I update this information without throwing away the resource and recreating it ?

Probably my question is stupid but I must admit I don't fully graps [yet] the whole process.

Thanks

-- Francois Piette

 

 

Coordinator
Dec 4, 2011 at 6:05 PM

I am still not sure I understand, but if you want to override the resize sizes, then you can do that by overriding the RenderApplication::HandleWindowResize method.  That is where the windows message is actually handled.  If you want to modify the x and y size of the resize, you can do it there.

But still, if your primary resource is independent of the window size, then it should be created somewhere else instead of according to the window size.  If you need to resize a window that is viewing that resource, then you would resize a secondary resource - not the primary one.

If I still haven't answered the question, please let me know...

- Jason

Dec 4, 2011 at 8:18 PM

For your first paragraph, I have understood where to catch resize handling. Already using it for other purposes.

For your second paragraph: My application derive from your ImageProcessingDemo. I replaced one of the PNG images by my image in special format, loaded as a resource (see the source code in previous message). I replaced the HLSL by one fitting my needs but basically it works the same as in your sample:  The OutputMap is computed from the InputMap.

The problem is that both InputMap and OutputMap are created by CPP code and window size is used for SetColorBuffer so it depends on window size. For the OutputMap, I understand, but the InputMap, it is independent of the window size. What is SetColorBuffer supposed to do ? Why doe it needs the window size ? Once a resource has been created with SetColorBuffer called with window size, how to change that without recreating the resource ?

Sorry for my bad english...

-- Francois Piette

 

Coordinator
Dec 5, 2011 at 5:43 PM

Please don't apologize - I am the one not understanding.  Your english is perfectly fine :)

I think the problem here is that the ImageProcessor sample assumes a 640x480 input image.  The size of the window is initialized to that value, but it doesn't mean that it must be that value.  When you create the output and intermediate buffers, just create them in the size that your input image is.  Here is the code in question:

	// Create the texture for output of the compute shader.
	
	Texture2dConfigDX11 FilteredConfig;
	FilteredConfig.SetColorBuffer( 640, 480 ); 
	FilteredConfig.SetBindFlags( D3D11_BIND_UNORDERED_ACCESS | D3D11_BIND_SHADER_RESOURCE );

	m_Intermediate = m_pRenderer11->CreateTexture2D( &FilteredConfig, 0 );
	m_Output = m_pRenderer11->CreateTexture2D( &FilteredConfig, 0 );

In this case, you would simply replace the 640 and 480 by your image size.  The display of the texture in the window will need to be updated according to how you want it to be (i.e. scale the whole image to fit, or have it zoom in or out) but your processing chain should be independent of how the image is being viewed.