head 1.4; access; symbols; locks; strict; comment @# @; 1.4 date 2009.11.25.03.14.41; author GarryJolleyRogers; state Exp; branches; next 1.3; 1.3 date 2009.11.20.02.45.34; author LeeBelbin; state Exp; branches; next 1.2; 1.2 date 2006.09.26.21.42.16; author BobMorris; state Exp; branches; next 1.1; 1.1 date 2005.11.30.11.38.04; author RogerHyam; state Exp; branches; next ; desc @none @ 1.4 log @none @ text @%META:TOPICINFO{author="GarryJolleyRogers" date="1259118881" format="1.1" version="1.4"}% %META:TOPICPARENT{name="DocsDocumentation"}% For completeness this is a cut and paste from [[%ATTACHURL%/DocsDraftStrategy.doc][Draft Strategy Word Doc]] ---+ A Documentation and Supporting Software Strategy for TDWG ---++ 1 Requirements ---+++ 1.1 All TDWG standards should be represented as a folio of documents. At a minimum, each standard should contain: 1. The normative form of the standard itself. (e.g. XML Schema). 2. A 'Cover Page' document that summarises the content and history of the standard. 3. A 'Motivations' document that describes the reasons for the standards existence. 4. A 'Rationale' document that describes why the standard takes the form it does. 5. A 'Change History' document that describes how this version has changed since the last version. ---+++ 1.2 TDWG must attach clear Copyright and Intellectual Property Rights statements to all standards documents. ---+++ 1.3 Each TDWG standard should be accompanied by a ladder of documentation that gives easy access to those making implementation decisions at all levels. A task group charter is pivotal in defining the documents that will be produced as part of a standard. When the charter is created and accepted by the Executive, it must define a list of documents that will be created as part of the work of the group. The Executive should ensure that documentation is created for managers and supporting agencies as well as developers. ---+++ 1.4 TDWG standards that are not available on-line should be retired. The role of a standards body is to hold standards. If the standards are not readily available TDWG is not fulfilling its role. ---+++ 1.5 All TDWG standards documentation should be of a structured form. Archives of meetings, instant messengers conversations, email lists and unstructured wikis should not be considered documentation for the purposes of the TDWG standards process. ---+++ 1.6 All TDWG standards documentation should be in a limited range of open, archival formats. ---+++ 1.7 There should be 3 Levels of documentation Level 1 documents are the normative parts of a standard. Level 2 documents are part of the standard that are non-normative. Level 3 documents are not part of the standard and will not be controlled by the TDWG process but will provide help and support to people working with the standard. ---+++ 1.8 The process group should publish a standards track with a limited range of maturity levels. As standards pass along the track they will increase their maturity level. The process document should produce a clear list of maturity levels. These levels should be applied to standards (i.e. the collection of documents composing the standard) and not to individual documents. It should not be possible for documents within a standard to have different maturity levels. ---+++ 1.9 All Level 1 documents should be in English using US spellings and grammatical constructs. Translations of Level 1 documents may be supplied, but the translations will be treated as either Level 2 or Level 3 documents. The Executive may require the production of Level 2 translations as part of the standard. All Level 2 documents should be in US English possibly with translations. The Executive may require the production of these translations. Level 3 documents may be produced in any language with no requirement for translation. ---+++ 1.10 All TDWG standards should have the same documentation standards. Even if a standard is in the form of a web service, a folio of documents describing the interface to the services and data curation methods should follow the TDWG standards process. ---+++ 1.11 Six documents should be created to initialise the TDWG process. This document is a bootstrap document and so does not form part of the standards process. Six documents are required to initialise the documentation process. 1. File Formats: The file formats, naming and versioning conventions used in TDWG standards. 2. Cover Page Specification: The format of the cover page document that should accompany every standard. This will be an XML Schema document. 3. Layout Template: A specification of how human readable documents should be laid out including the minimum metadata fields such as IPR and Copyright statements. 4. Process Document: A document specifying how the TDWG process track is administered. This document will include a list of standards maturity levels. 5. Copyright Notice: The text of a copyright notice to be used on TDWG standards documents along with notes on how it may or may not be modified. 6. Intellectual Property Statement: Guidelines for inclusion of Intellectual Property Rights statements in TDWG standards documents. ---+++ 1.12 Task group charters should explain how systems can test whether they comply with the proposed standard. Testing will usually take the form of producing compliance test suite of some kind. In the case of XML Schemas the schema itself could be considered the test suite in the case of exchange protocols the test suites are likely to be more complex. ---+++ 1.13 The TDWG process should encourage the development of reference implementations for standards by requiring it as part of the task group charter, where it is appropriate. Compliance test suites should be required alongside reference implementations to show that they conform to the standard. In some situations (such as data exchange protocols and schemas) it may be possible to have two reference implementations that act as compliance tests for each other. ---+++ 1.14 Two independent teams must develop the systems that are subsequently used to test each other. Ideally these implementations should also be implemented using different technologies. ---+++ 1.15 If TDWG is to ensure quality control of data being served dynamically four things must happen: 1. Documents need to be created that describe the interface to the data and how the data will be curated. These documents must go through the TDWG standards track. 2. Software needs to be provided that will serve the data in the way specified in the documentation. This software may be hosted by TDWG or a third party. 3. Test software needs to be created that will regularly monitor that data is being served in a way that is compliant with the documentation. This has to be hosted by TDWG or on TDWG's behalf. 4. Demonstration client software needs to be created to demonstrate how to use the service. ---++ 2 Justification A review of Current TDWG standards documentation and examples of the approaches taken by other organisations are given in appendices A and B. ---+++ 2.1 Why should TDWG standards be composed of multiple documents? It is unlikely that a third party will see a developing TDWG standard and spontaneously write introductory literature, tutorials and criticisms as would happen with a W3C Draft Recommendation. It is therefore necessary for the TDWG standards to require a minimum level of documentation. Without this documentation, standards are less likely to become widely adopted or maintained into the future and TDWG will fail in its mission. [See 1.1] ---+++ 2.2 Why should documentation be in set structures and formats? Feedback from current developers suggests that getting people to review their work is a major issue. Potential users of standards have complained that they find it hard to get information on both managerial and technical aspects of the standards. Simplified, uniform documentation procedures will help in this. When TDWG ratifies a standard, it is making a commitment of resources to host that standard, and to migrate it into any future collaborative environment. In order to maintain the viability of this process, TDWG will need to sanction the different formats in which it will accept documentation for particular purposes. [See 1.5 an 1.6] ---+++ 2.3 Why is Copyright and IPR important? Copyright and Intellectual Property matters must be unambiguous. Public sector and not-for-profit organisations are becoming increasingly aware of the value of the intellectual property they possess and expect clear terms on which they will release it. Commercial organisations are unlikely to become involved in development and implementation of standards if ownership is ambiguous. Less scrupulous organisations may try to take ownership of standards through copyright and patenting if they are produced in a legal vacuum. [See 1.2 ] ---+++ 2.4 What are the three document levels? The three different levels are illustrated in Diagram 1 and Table 1. ---++++ Diagram 1: Documentation Levels doc_levels_diagram.jpg ---++++ Table 1: Enumeration of document levels. ||Level 1|Level 2|Level 3| |Normative|Yes|No|No| |Part of a standard|Yes|Yes|No| |Function|Defines|Explains and justifies|Helps and Supports| |Versioned with Standard|Yes|Yes|No| |Controlled by TDWG Process|Yes|Yes|No| |Document format|Tightly Controlled|Tightly Controlled|Not Controlled| |Document content|Tightly Controlled|Loosely Controlled|Not Controlled| |Example formats|XML/ RTF/PDF/XSD|XML/RTF/PDF|Wiki/HTML/Word/PDF| |Maintained in collaborative infrastructure|Yes|Yes|Sometimes or possibly only as a link.| ---+++ 2.5 Why is compliance testing needed? A compliance test suite is a series of procedures or software applications that can be applied to a system to test whether it meets a particular standard. Most TDWG standards have had intrinsic compliance testing. The paper-based data standards are lists of facts. Compliance is trivial. If the abbreviation is not in the list then it isn't compliant. The newer XML Schema based standards are test suites. By definition, if an XML document validates against a schema then it is compliant with the standard. As more complex standards, such as exchange protocols are ratified, the need for compliance testing will increase. Compliance is central to other standards organisations like OGC (http://www.opengeospatial.org/resources/?page=testing). [See 1.12] ---+++ 2.6 Why are reference implementations needed? A reference implementation is an example system where software is used to demonstrate a standard. There are three specific ways reference implementations can help the standards process. a. The most powerful barrier to reaching consensus on a standard is the criticism “it will never work”. A reference implementation – demonstrating that it does work - is the most effective way to counter this argument. b. A reference implementation is perhaps the most effective Level 3 documentation. “A standard is much easier to understand with a working example in hand.” (http://en.wikipedia.org/wiki/Reference_implementation) c. A reference implementation demonstrates the independence of the standard from an implementation. See [1.13] ---+++ 2.7 Why should TDWG ratify data being served dynamically? Botanical author abbreviations and herbarium codes are examples of TDWG data standards that are now databases. If these databases are to be ratified as TDWG standards, either static, versioned snapshots need to be released as regular standard documents, or the dynamic data has to be served in a controlled way. There appears to be little interest in formalising older TDWG standards, but this situation could soon change with the introduction of Globally Unique Identifiers. TDWG may need to ratify standard lists such as herbaria or entomological collections. The TDWG Process should handle these cases. [See 1.15] ---+ Appendix A: Review of Current Standards Documentation ---++ 1 Current Standards: * Authors of Plant Names: A book listing plant name authors and approved abbreviations published in 1992. Now integrated into International Plant Names Index. Help text associated with on-line searches. * Botanical Periodicals: A list of botanical periodicals. Published 1968 with supplement 1991. Currently out of print. * Economic Botany Data Collection Standard: A book published in 1995. Still in print. * XDF: A language for the definition and exchange of biological data sets. * Floristic Regions of the World: A section of a book published in 1986. Now out of print. * Geographical Codes: A book published in 1992 and updated in 2001(not clear if 2001 version is ratified by TDWG). Polygons of listed regions have been made available by Kew but not been put forward for ratification by TDWG. (http://www.tdwg.org/geo2.htm) * Herbarium Index Codes: A book published in 1990. Now maintained as an on-line database. Not clear if on-line version is TDWG standard. Help text associated with on-line searches. (http://sciweb.nybg.org/science2/IndexHerbariorum.asp) * Herbarium Information Standards and Protocols for Interchange of Data. Available as a document on-line or in print. Latest version available as a searchable database. (http://plantnet.rbgsyd.nsw.gov.au/HISCOM/HISPID/HISPID3/hispidright.html) * Plant Names in Botanical Databases: A 1995 publication available on-line (http://www.tdwg.org/plants.html) * Publication Abbreviations: A book series started in 1976. * Plant occurrence and status scheme: A document published in 1995 and available on-line (http://www.tdwg.org/poss_standard.html). * DELTA (Description Language for Taxonomy): User guide published in 1986 with subsequent updates. Much available on line though linked to a particular implementation (http://delta-intkey.com/) ---++ 2 Proposed Standards: * ABCD: XML Schema, schema annotations, misc pdf documentation, database of 'concepts' used within the schema. (http://www.bgbm.org/TDWG/CODATA/ABCD-Versions.htm & http://ww3.bgbm.org/abcddocs/AbcdIntroduction) * BDI.SDD_: Extensive Wiki documentation and discussion including an initial draft of a 'Primer' document to get people started. Authors have difficulty keeping documentation up to date. (BDI.SDD_) * Taxon Concept Schema: XML Schema, schema annotations exported as html, user guide as pdf, development wiki. (http://www.soc.napier.ac.uk/tdwg/index.php) ---+ Appendix B: Documentation in Other Standards Bodies – Best Practice In contrast to TDWG, most successful standards bodies are focussed on documents and managing them. * One of the primary purposes of the Global Grid Forum (GGF) is to publish documents. These documents provide information and specifications to developers and others involved with Grid computing.” The Global Grid Forum (http://www.ggf.org/) * Specifications are simply engineering documents that describe how the OGC membership has agreed to solve an interoperability problem.” Open Geospatial Consortium (http://www.opengeospatial.org) * The W3C working groups are principally involved in moving a 'technical report' document along the Recommendation Track to becoming a W3C Recommendation. (http://www.w3.org/2004/02/Process-20040205/) Organization for the Advancement of Structured Information Standards (OASIS) use documents to drive their Technical Committee Process. (http://www.oasis-open.org/committees/process.php) -- Main.RogerHyam - 30 Nov 2005 %META:FILEATTACHMENT{name="doc_levels_diagram.jpg" attr="" autoattached="1" comment="document_levels_diagram" date="1143055794" path="doc_levels_diagram.jpg" size="49049" user="Main.RogerHyam" version=""}% %META:FILEATTACHMENT{name="DocsDraftStrategy.doc" attr="" autoattached="1" comment="Draft Strategy - Word Doc" date="1143055794" path="DocsDraftStrategy.doc" size="133120" user="Main.RogerHyam" version=""}% @ 1.3 log @none @ text @d1 1 a1 1 %META:TOPICINFO{author="LeeBelbin" date="1258685134" format="1.1" reprev="1.3" version="1.3"}% d104 1 a104 1 * BDI.SDD: Extensive Wiki documentation and discussion including an initial draft of a 'Primer' document to get people started. Authors have difficulty keeping documentation up to date. (BDI.SDD) @ 1.2 log @none @ text @d1 1 a1 1 %META:TOPICINFO{author="BobMorris" date="1159306936" format="1.1" version="1.2"}% d104 1 a104 1 * SDD: Extensive Wiki documentation and discussion including an initial draft of a 'Primer' document to get people started. Authors have difficulty keeping documentation up to date. (SDD.WebHome) @ 1.1 log @none @ text @d1 1 a1 1 %META:TOPICINFO{author="RogerHyam" date="1133350684" format="1.1" version="1.1"}% d104 1 a104 1 * SDD: Extensive Wiki documentation and discussion including an initial draft of a 'Primer' document to get people started. Authors have difficulty keeping documentation up to date. (http://bdei.cs.umb.edu/twiki/bin/view/SDD/WebHome) d118 2 a119 2 %META:FILEATTACHMENT{name="DocsDraftStrategy.doc" attr="" autoattached="1" comment="Draft Strategy - Word Doc" date="1133350373" path="DocsDraftStrategy.doc" size="133120" user="Main.RogerHyam" version=""}% %META:FILEATTACHMENT{name="doc_levels_diagram.jpg" attr="" autoattached="1" comment="document_levels_diagram" date="1133350512" path="doc_levels_diagram.jpg" size="49049" user="Main.RogerHyam" version=""}% @