risc-v: 0.675 user-level: 0.674 mistranslation: 0.615 permissions: 0.595 device: 0.586 arm: 0.573 PID: 0.559 semantic: 0.555 register: 0.553 performance: 0.546 operating system: 0.546 graphic: 0.532 assembly: 0.531 architecture: 0.530 ppc: 0.523 network: 0.518 TCG: 0.517 debug: 0.516 virtual: 0.512 hypervisor: 0.502 socket: 0.501 files: 0.490 boot: 0.486 VMM: 0.473 KVM: 0.470 peripherals: 0.466 vnc: 0.465 kernel: 0.441 x86: 0.402 alpha: 0.375 i386: 0.370 [Qemu-devel] [Bug] Docs build fails at interop.rst https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31 (Rawhide) uname - a Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431 allows for the build to occur Regards Aarushi Mehta On 5/20/19 7:30 AM, Aarushi Mehta wrote: > https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw > running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31 > (Rawhide) > > uname - a > Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32 > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux > > Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431 > allows for the build to occur > > Regards > Aarushi Mehta > > Ah, dang. The blocks aren't strictly conforming json, but the version I tested this under didn't seem to care. Your version is much newer. (I was using 1.7 as provided by Fedora 29.) For now, try reverting 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7 instead, which should at least turn off the "warnings as errors" option, but I don't think that reverting -n will turn off this warning. I'll try to get ahold of this newer version and see if I can't fix it more appropriately. --js On 5/20/19 12:37 PM, John Snow wrote: > > > On 5/20/19 7:30 AM, Aarushi Mehta wrote: > > https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw > > running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31 > > (Rawhide) > > > > uname - a > > Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32 > > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux > > > > Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431 > > allows for the build to occur > > > > Regards > > Aarushi Mehta > > > > > > Ah, dang. The blocks aren't strictly conforming json, but the version I > tested this under didn't seem to care. Your version is much newer. (I > was using 1.7 as provided by Fedora 29.) > > For now, try reverting 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7 instead, > which should at least turn off the "warnings as errors" option, but I > don't think that reverting -n will turn off this warning. > > I'll try to get ahold of this newer version and see if I can't fix it > more appropriately. > > --js > ...Sigh, okay. So, I am still not actually sure what changed from pygments 2.2 and sphinx 1.7 to pygments 2.4 and sphinx 2.0.1, but it appears as if Sphinx by default always tries to do add a filter to the pygments lexer that raises an error on highlighting failure, instead of the default behavior which is to just highlight those errors in the output. There is no option to Sphinx that I am aware of to retain this lexing behavior. (Effectively, it's strict or nothing.) This approach, apparently, is broken in Sphinx 1.7/Pygments 2.2, so the build works with our malformed json. There are a few options: 1. Update conf.py to ignore these warnings (and all future lexing errors), and settle for the fact that there will be no QMP highlighting wherever we use the directionality indicators ('->', '<-'). 2. Update bitmaps.rst to remove the directionality indicators. 3. Update bitmaps.rst to format the QMP blocks as raw text instead of JSON. 4. Update bitmaps.rst to remove the "json" specification from the code block. This will cause sphinx to "guess" the formatting, and the pygments guesser will decide it's Python3. This will parse well enough, but will mis-highlight 'true' and 'false' which are not python keywords. This approach may break in the future if the Python3 lexer is upgraded to be stricter (because '->' and '<-' are still invalid), and leaves us at the mercy of both the guesser and the lexer. I'm not actually sure what I dislike the least; I think I dislike #1 the most. #4 gets us most of what we want but is perhaps porcelain. I suspect if we attempt to move more of our documentation to ReST and Sphinx that we will need to answer for ourselves how we intend to document QMP code flow examples. --js On Mon, May 20, 2019 at 05:25:28PM -0400, John Snow wrote: > > > On 5/20/19 12:37 PM, John Snow wrote: > > > > > > On 5/20/19 7:30 AM, Aarushi Mehta wrote: > >> https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw > >> running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31 > >> (Rawhide) > >> > >> uname - a > >> Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32 > >> UTC 2019 x86_64 x86_64 x86_64 GNU/Linux > >> > >> Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431 > >> allows for the build to occur > >> > >> Regards > >> Aarushi Mehta > >> > >> > > > > Ah, dang. The blocks aren't strictly conforming json, but the version I > > tested this under didn't seem to care. Your version is much newer. (I > > was using 1.7 as provided by Fedora 29.) > > > > For now, try reverting 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7 instead, > > which should at least turn off the "warnings as errors" option, but I > > don't think that reverting -n will turn off this warning. > > > > I'll try to get ahold of this newer version and see if I can't fix it > > more appropriately. > > > > --js > > > > ...Sigh, okay. > > So, I am still not actually sure what changed from pygments 2.2 and > sphinx 1.7 to pygments 2.4 and sphinx 2.0.1, but it appears as if Sphinx > by default always tries to do add a filter to the pygments lexer that > raises an error on highlighting failure, instead of the default behavior > which is to just highlight those errors in the output. There is no > option to Sphinx that I am aware of to retain this lexing behavior. > (Effectively, it's strict or nothing.) > > This approach, apparently, is broken in Sphinx 1.7/Pygments 2.2, so the > build works with our malformed json. > > There are a few options: > > 1. Update conf.py to ignore these warnings (and all future lexing > errors), and settle for the fact that there will be no QMP highlighting > wherever we use the directionality indicators ('->', '<-'). > > 2. Update bitmaps.rst to remove the directionality indicators. > > 3. Update bitmaps.rst to format the QMP blocks as raw text instead of JSON. > > 4. Update bitmaps.rst to remove the "json" specification from the code > block. This will cause sphinx to "guess" the formatting, and the > pygments guesser will decide it's Python3. > > This will parse well enough, but will mis-highlight 'true' and 'false' > which are not python keywords. This approach may break in the future if > the Python3 lexer is upgraded to be stricter (because '->' and '<-' are > still invalid), and leaves us at the mercy of both the guesser and the > lexer. > > I'm not actually sure what I dislike the least; I think I dislike #1 the > most. #4 gets us most of what we want but is perhaps porcelain. > > I suspect if we attempt to move more of our documentation to ReST and > Sphinx that we will need to answer for ourselves how we intend to > document QMP code flow examples. Writing a custom lexer that handles "<-" and "->" was simple (see below). Now, is it possible to convince Sphinx to register and use a custom lexer? $ cat > /tmp/lexer.py < *', Generic.Prompt), (r' *<- *', Generic.Output), ] } class QMPExampleLexer(DelegatingLexer): def __init__(self, **options): super(QMPExampleLexer, self).__init__(JsonLexer, QMPExampleMarkersLexer, Error, **options) EOF $ pygmentize -l /tmp/lexer.py:QMPExampleLexer -x -f html < { "execute": "drive-backup", "arguments": { "device": "drive0", "bitmap": "bitmap0", "target": "drive0.inc0.qcow2", "format": "qcow2", "sync": "incremental", "mode": "existing" } } <- { "return": {} } EOF 
    -> 
{
         "execute": 
"drive-backup",
         "arguments": 
{
           "device": 
"drive0",
           "bitmap": 
"bitmap0",
           "target": 
"drive0.inc0.qcow2",
           "format": 
"qcow2",
           "sync": 
"incremental",
           "mode": 
"existing"
         }
       }

    <- { "return": {} }
$ -- Eduardo On 5/20/19 7:04 PM, Eduardo Habkost wrote: > On Mon, May 20, 2019 at 05:25:28PM -0400, John Snow wrote: > > > > > > On 5/20/19 12:37 PM, John Snow wrote: > >> > >> > >> On 5/20/19 7:30 AM, Aarushi Mehta wrote: > >>> https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw > >>> running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31 > >>> (Rawhide) > >>> > >>> uname - a > >>> Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32 > >>> UTC 2019 x86_64 x86_64 x86_64 GNU/Linux > >>> > >>> Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431 > >>> allows for the build to occur > >>> > >>> Regards > >>> Aarushi Mehta > >>> > >>> > >> > >> Ah, dang. The blocks aren't strictly conforming json, but the version I > >> tested this under didn't seem to care. Your version is much newer. (I > >> was using 1.7 as provided by Fedora 29.) > >> > >> For now, try reverting 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7 instead, > >> which should at least turn off the "warnings as errors" option, but I > >> don't think that reverting -n will turn off this warning. > >> > >> I'll try to get ahold of this newer version and see if I can't fix it > >> more appropriately. > >> > >> --js > >> > > > > ...Sigh, okay. > > > > So, I am still not actually sure what changed from pygments 2.2 and > > sphinx 1.7 to pygments 2.4 and sphinx 2.0.1, but it appears as if Sphinx > > by default always tries to do add a filter to the pygments lexer that > > raises an error on highlighting failure, instead of the default behavior > > which is to just highlight those errors in the output. There is no > > option to Sphinx that I am aware of to retain this lexing behavior. > > (Effectively, it's strict or nothing.) > > > > This approach, apparently, is broken in Sphinx 1.7/Pygments 2.2, so the > > build works with our malformed json. > > > > There are a few options: > > > > 1. Update conf.py to ignore these warnings (and all future lexing > > errors), and settle for the fact that there will be no QMP highlighting > > wherever we use the directionality indicators ('->', '<-'). > > > > 2. Update bitmaps.rst to remove the directionality indicators. > > > > 3. Update bitmaps.rst to format the QMP blocks as raw text instead of JSON. > > > > 4. Update bitmaps.rst to remove the "json" specification from the code > > block. This will cause sphinx to "guess" the formatting, and the > > pygments guesser will decide it's Python3. > > > > This will parse well enough, but will mis-highlight 'true' and 'false' > > which are not python keywords. This approach may break in the future if > > the Python3 lexer is upgraded to be stricter (because '->' and '<-' are > > still invalid), and leaves us at the mercy of both the guesser and the > > lexer. > > > > I'm not actually sure what I dislike the least; I think I dislike #1 the > > most. #4 gets us most of what we want but is perhaps porcelain. > > > > I suspect if we attempt to move more of our documentation to ReST and > > Sphinx that we will need to answer for ourselves how we intend to > > document QMP code flow examples. > > Writing a custom lexer that handles "<-" and "->" was simple (see below). > > Now, is it possible to convince Sphinx to register and use a custom lexer? > Spoilers, yes, and I've sent a patch to list. Thanks for your help!