qxl: support mono cursors with inverted colors

Monochrome cursors are still used by Windows guests with the
QXL-WDDM-DOD driver. Such cursor types have one odd feature, inversion
of colors. GDK does not seem to support it, so implement an alternative
solution: fill the inverted pixels and add an outline to make the cursor
more visible. Tested with the text cursor in Notepad and Windows 10.

cursor_set_mono is also used by the vmware GPU, so add a special check
to avoid breaking its 32bpp format (tested with Kubuntu 14.04.4). I was
unable to find a guest which supports the 1bpp format with a vmware GPU.

The old implementation was buggy and removed in v2.10.0-108-g79c5a10cdd
("qxl: drop mono cursor support"), this version improves upon that by
adding bounds validation, clarifying the semantics of the two masks and
adds a workaround for inverted colors support.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1611984
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Message-id: 20180903145447.17142-1-peter@lekensteyn.nl

[ kraxel: minor codestyle fix ]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

1 files changed, 16 insertions, 0 deletions

diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c
index 6debe8fc11..14ad2b352d 100644
--- a/hw/display/qxl-render.c
+++ b/hw/display/qxl-render.c
@@ -236,12 +236,28 @@ static QEMUCursor *qxl_cursor(PCIQXLDevice *qxl, QXLCursor *cursor,
                               uint32_t group_id)
 {
     QEMUCursor *c;
+    uint8_t *and_mask, *xor_mask;
     size_t size;
 
     c = cursor_alloc(cursor->header.width, cursor->header.height);
     c->hot_x = cursor->header.hot_spot_x;
     c->hot_y = cursor->header.hot_spot_y;
     switch (cursor->header.type) {
+    case SPICE_CURSOR_TYPE_MONO:
+        /* Assume that the full cursor is available in a single chunk. */
+        size = 2 * cursor_get_mono_bpl(c) * c->height;
+        if (size != cursor->data_size) {
+            fprintf(stderr, "%s: bad monochrome cursor %ux%u with size %u\n",
+                    __func__, c->width, c->height, cursor->data_size);
+            goto fail;
+        }
+        and_mask = cursor->chunk.data;
+        xor_mask = and_mask + cursor_get_mono_bpl(c) * c->height;
+        cursor_set_mono(c, 0xffffff, 0x000000, xor_mask, 1, and_mask);
+        if (qxl->debug > 2) {
+            cursor_print_ascii_art(c, "qxl/mono");
+        }
+        break;
     case SPICE_CURSOR_TYPE_ALPHA:
         size = sizeof(uint32_t) * cursor->header.width * cursor->header.height;
         qxl_unpack_chunks(c->data, size, qxl, &cursor->chunk, group_id);