feature request: XMLLayout (cfr. log4j)

Wim Deblauwe added a comment - 25/Nov/07 8:28 PM I would like to have this too. It would help me to support logback for my log file viewing application (http://vigilog.sf.net)

Thorbjoern Ravn Andersen added a comment - 24/Nov/08 11:36 PM - edited XMLLayout initial implementation. Generates log4j.dtd compliant XML fragments. "properties" and "locationInfo" attributes are enabled per default. XML generated is ASCII only and compatible with UTF-8 and ISO-8859, and has been briefly tested with vigilog. Feedback is appreciated.

Thorbjoern Ravn Andersen added a comment - 24/Nov/08 11:37 PM The following logback.xml works with the attached com.runjva.logback.xml.XMLLayout: <configuration debug="false">     <appender name="STDOUT"     class="ch.qos.logback.core.ConsoleAppender">     <layout class="com.runjva.logback.xml.XMLLayout"> <!-- <locationInfo>true</locationInfo>     <properties>true</properties> --> </layout>   </appender>    <root>     <level value="debug" />     <appender-ref ref="STDOUT" />   </root>    </configuration>

Maarten Bosteels added a comment - 25/Nov/08 9:29 AM Great ! Some questions (after a very quick look at the code, and without actually running it): * why are locationInfo and properties package private, instead of just private ? * why are locationInfo and properties default true (in log4j they are deafult false, because locationInfo does have an impact on performance) * why use ASCII when one of the advantages of XML is that you can specify an encoding ? * I think it would be better to use <![CDATA[ blocks instead of escaping all non-ASCII characters, remember that XML != HTML,   and AFAIK these escaped characters are NOT valid xml.

Thorbjoern Ravn Andersen added a comment - 25/Nov/08 7:49 PM This is an initial release just to see if stuff works, not a final release. I'm still learning how to get Joran configuration working properly, so I decided to leave these on per default for now (the missing private is a typo) The escape mechanism used is valid XML - please see http://www.w3.org/TR/2004/REC-xml-20040204/#dt-charref - and allows a pure ASCII based XML fragment to refer to any Unicode character. In my experience this approach is very robust to file encoding issues and transports between systems. Please try it and let me know how it works, so we can make a better version.

Thorbjørn Ravn Andersen added a comment - 04/Feb/09 1:38 PM - edited revised and commented - generates humanly readable output including a ISO8601 date in the time attribute (which is allowed but unused by the log4j XMLLayout)

Thorbjørn Ravn Andersen added a comment - 04/Feb/09 1:41 PM This logback.xml snippet works with the uploaded XMLLayout.java file <appender name="FILE" class="ch.qos.logback.core.FileAppender"> <layout class="com.runjva.logback.xml.XMLLayout"> <LocationInfo>true</LocationInfo> <Properties>true</Properties> </layout> <File>c:/logback-output.xml</File> <Append>false</Append> </appender> The c:/logback-output.xml file can be read with the "File -> Load Log4j file" in the Chainsaw version which can be invoked with Java Web Start from http://logging.apache.org/log4j/docs/webstart/chainsaw/chainsawWebStart.jnlp

Ceki Gulcu added a comment - 05/Feb/09 10:31 PM I am marking this issue as resolved since the XMLLayout as found in our repository seems to work fine. It differs from the code Thorbjoern in that it does not output a "time" attribute. I wonder why it was introduced. Moreover, XMLLayout in our svn repo handles locationInformaion without creating new *CallerConverter instances. It also relies on character transformation very similar (if not identical) to those in the original XMLLayaout as found in log4j. Your comments are welcome.

Thorbjoern Ravn Andersen added a comment - 05/Feb/09 11:58 PM I do not understand the "as found in our repository" sentence. You added the file yesteday after I uploaded a revised patch, so the file was not there (unless I missed something). The time field was added to make the XML output readable to a human looking at the file, as nobody I know can understand timestamps in his head. This also includes the careful formatting of the output with line breaks. My character transformation is a clean design working well without the disambiguity of the CDATA (is a <tag> a real tag or a part of a CDATA stream) and as an added benefit generates 7-bit clean output. I have produced a neat implementation conforming to logback conventions and tested it with Chainsaw. You haven't even made a single comment on this issue even though its been in here for 14 months, decided to disregard my work and just grab the log4j implementation, add the MDC lines and calling it FIXED. I would have appreciated discussing this on logback-dev before the fact and not in comments after the fact. As it is now I see no reason my name is in the @authors tag in the file, and I would appreciate it if you removed it.

Ceki Gulcu added a comment - 06/Feb/09 5:17 PM The meaning of "as found in our repository" was "as *currently* found in our repository". I was not intimating any precedence with respect to your contribution. > My character transformation is a clean design working well without the > disambiguity of the CDATA (is a <tag> a real tag or a part of a CDATA > stream) and as an added benefit generates 7-bit clean output. There is no need to generate 7-bit clean ouput, as we are generating an XML document which supports UTF-8 encoding by default. In your version, the block relating to locationInfo creates various new caller converters, which are quite unnecessary. > I have produced a neat implementation conforming to logback > conventions and tested it with Chainsaw. You haven't even made a > single comment on this issue even though its been in here for 14 > months, decided to disregard my work and just grab the log4j > implementation, add the MDC lines and calling it FIXED. While I have not made any comments per se, I have assigned the issue to me in the recent weeks, signaling that I intended to work on the topic. Your work has not been discarded. Although it has not been adopted as is, your contribution has given the necessary impetus to get this issue fixed. I have looked at your code and adopted parts of it. Hence the author attribution to you. Not all code contributions get accepted as is. I welcome any any specific comments you might have on the code which is currently in SVN, including comparisons with your contribution. > I would have appreciated discussing this on logback-dev before the > fact and not in comments after the fact. The code in SVN is a basis for discussion. Again, I welcome any comments you might have on this topic. > As it is now I see no reason my name is in the @authors tag in the > file, and I would appreciate it if you removed it. I have already explained why the @author tag was justified. If you still wish to see your name removed, please let me know.

Thorbjoern Ravn Andersen added a comment - 16/Feb/09 10:19 AM After careful consideration I have reached the conclusion that I will not do "after-the-fact"-discussions. I have a need for the functionality provided by my implementation so I will bring it in the as400 incubator (with a revised name). Regarding my name in the @author tag: I have found in general that @author tags are more a hinderance than an advantage in projects where code are considered a common responsibility - if you see a bug, you take responsibility and ensure it is fixed (perhaps by yourself) - and for a SVN-backed project like logback it is redundant and perhaps even outdated. Hence you do not have to add my name to the authors tags in the future and I'd appreciate it if you would remove it from those files which have it.

Ceki Gulcu added a comment - 16/Feb/09 10:56 AM > After careful consideration I have reached the conclusion that I will > not do "after-the-fact"-discussions. I understand that you are pissed that your XMLLayout contribution was not adopted as is. I also understand that you are peeved about the lack of discussion before I committed XMLLayout in SVN. My bad. Can we discuss the functionality missing in XMLLayout compared to your version? > I have a need for the functionality provided by my implementation so I > will bring it in the as400 incubator (with a revised name). Same question as above, which functionality do you have in mind? > Regarding my name in the @author tag: I have found in general that > @author tags are more a hinderance than an advantage in projects where > code are considered a common responsibility - if you see a bug, you > take responsibility and ensure it is fixed (perhaps by yourself) - and > for a SVN-backed project like logback it is redundant and perhaps even > outdated. Hence you do not have to add my name to the authors tags in > the future and I'd appreciate it if you would remove it from those > files which have it. There are different opinions on the subject of author tags. IMO, in oss projects where peer recognition is a big motivation for contributors, author tags matter quite a bit. At this time, there is only one class bearing your name, i.e. XMLLayout. At your request, I'll remove your name from that class asap.

Thorbjoern Ravn Andersen added a comment - 16/Feb/09 2:45 PM I am neither "pissed" nor "peeved", but I was quite astonished. I have found it to be due to the difference in our coding cultures and my subconcious expectations. So be it. There are two key functionalities: 1) "generates humanly readable output" - the output is in a form that is readable to the human eye when browsing the text file containing the output from the XMLLayout. This _INCLUDES_ the time attribute which contains a humanly readable version of the timestamp, put in a location where it is useful in the situation where the human needs to read the logfile. I took great care in putting the linefeeds in the right places while maintaining DTD conformancy. 2) Uses numeric XML entities instead of CDATA. In my experience this is more robust to data transports where the encoding is broken - which we have more often than I like to think about here - as it keeps to 7-bit ascii, AND it is better to the human reader as "<", ">" and "&" are only found in their proper form in tagnames and entities and not as a CDATA stream. Also linefeeds etc are encoded as their numeric XML entity, resultning in each log event generating exactly two lines of output (but can be very wide). The goal I am pursuing is to move from processed logs in flat text files (oh the joy of multi line log messages) into easily reprocessable log files in our production scenario. Hence a XMLLayout class which generates humanly readable output while being log4j DTD conformant so it can be processed with e.g. Chainsaw and hopefully soon Lilith. I understand your desire to be 100% compatible with log4j. I hope you understand my need for a different version which is compatible with the log4j DTD but does not generates 100% identical output. In XML there are many ways to say the same thing.

Ceki Gulcu added a comment - 16/Feb/09 4:23 PM > 1) "generates humanly readable output" - the output is in a form that is readable to the human eye when browsing > the text file containing the output from the XMLLayout. This _INCLUDES_ the time attribute which contains a > humanly readable version of the timestamp, put in a location where it is useful in the situation where the human > needs to read the logfile. I took great care in putting the linefeeds in the right places while maintaining DTD > conformancy. The output generated by XMLLayout_Blue (blue for the version currently shipping in logback 0.9.16) also generates "humanly readable" output. The timestamp attribute is *not* part of log4j.dtd, at least not on any currently shipping version of log4j. I am not sure one can just add unexpected attributes to a given element and still be DTD compliant. How about adding a comment which includes the date? A comment will satisfy the human reader without affecting DTD compliance. > 2) Uses numeric XML entities instead of CDATA. In my experience this is more robust to data transports where the > encoding is broken - which we have more often than I like to think about here - as it keeps to 7-bit ascii, AND it is > better to the human reader as "<", ">" and "&" are only found in their proper form in tagnames and entities and not as > a CDATA stream. Also linefeeds etc are encoded as their numeric XML entity, resultning in each log event > generating exactly two lines of output (but can be very wide). We could add a 7-bit transport property (a boolean) to XMLLayout_Blue and if 7-bit transport property were set, your style of encoding would be used.

Thorbjoern Ravn Andersen added a comment - 16/Feb/09 6:23 PM The log4j.dtd present in the log4j-1.2.15 distribution has this definition of log4j:event <!-- The timestamp format is application dependent. --> <!ATTLIST log4j:event     logger CDATA #REQUIRED     level CDATA #REQUIRED     thread CDATA #REQUIRED     timestamp CDATA #REQUIRED     time CDATA #IMPLIED > The XML specification states in 3.3.2 that "In an attribute declaration, #REQUIRED means that the attribute MUST always be provided, #IMPLIED that no default value is provided." (http://www.w3.org/TR/REC-xml/#sec-attr-defaults).

Ceki Gulcu added a comment - 16/Feb/09 7:14 PM The log4j.dtd in log4j version 1.2.14 reads: <!ATTLIST log4j:event     logger CDATA #REQUIRED     level CDATA #REQUIRED     thread CDATA #REQUIRED     timestamp CDATA #REQUIRED > Thus, my previous remark is still valid. The timestamp attribute is *not* part of log4j.dtd in log4j 1.2.14 and earlier, I don't think we can just add unexpected attributes to a given element and still be DTD compliant. How about adding a comment which includes the date? A comment will satisfy the human reader without affecting DTD compliance.

Thorbjoern Ravn Andersen added a comment - 16/Feb/09 7:29 PM I stand corrected. I did not look further back than the version included with 1.2.15. The XML fragments generated with my XMLLayout will not validate against older versions of log4j.dtd and is therefore not compliant.