GMLWriter & Schema Generation

Description

Can you tell me how FeatureTypeTransformer must be changed/improved so that it is "in good enough shape" for cite. What is missing? I might look into this and improve FeatureTypeTransformer. Or is it just the problem that FeatureType and AttributeType do not provide enough information?

Simon

Am 16.01.2004 um 21:24 schrieb Chris Holmes:

> Will you put this as a bug in JIRA? Also for the cite tests you should
> use schema.xml files, there's no way the FeatureTypeTransformer is in good
> enough shape to handle cite. The featureTypes that you need are defined
> in the misc/cite/citeTypes.tar.gz, just unzip it and it will build the
> featureType structure that you need.
>
> Chris
>
> On Fri, 16 Jan 2004, David Zwiers wrote:
>
>> My response.
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs" xmlns:gml="http://www.opengis.net/gml" xmlns:cdf="http://www.opengis.net/cite/data" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs http://office.refractions.net:8081/geoserver/wfs/1.0.0/WFS-basic.xsd - http://office.refractions.net:8081/geoserver/wfs/DescribeFeatureType? typeName=cdf:Other">
>> <gml:boundedBy>
>> <gml:Box srsName="http://www.opengis.net/gml/srs/epsg.xml#32118">
>> <gml:coordinates decimal="." cs="," ts=" ">500000,500000 500000,500100 500100,500100 500100,500000</gml:coordinates>
>>
>> </gml:Box>
>>
>> </gml:boundedBy>
>> <gml:featureMember>
>> <cdf:Other fid="Other.676316386">
>> <cdf:gmldescription>A Single Feature used to test returning of properties</cdf:gmldescription>
>> <cdf:gmlname>singleFeature</cdf:gmlname>
>> <gml:boundedBy>
>> <gml:Box srsName="http://www.opengis.net/gml/srs/epsg.xml#32118">
>> <gml:coordinates decimal="." cs="," ts=" ">500000,500000 500000,500100 500100,500100 500100,500000</gml:coordinates>
>>
>> </gml:Box>
>>
>> </gml:boundedBy>
>> <gmlointProperty>
>> <gmloint srsName="http://www.opengis.net/gml/srs/epsg.xml#32118">
>> <gml:coordinates decimal="." cs="," ts=" ">500050,500050</gml:coordinates>
>>
>> </gmloint>
>>
>> </gmlointProperty>
>> <cdf:string1>always</cdf:string1>
>> <cdf:string2>sometimes</cdf:string2>
>> <cdf:integers>7</cdf:integers>
>> <cdf:dates>2002-12-02</cdf:dates>
>>
>> </cdf:Other>
>>
>> </gml:featureMember>
>>
>> </wfs:FeatureCollection>
>>
>> The buged lines of this:
>>
>> <cdf:gmldescription>A Single Feature used to test returning of properties</cdf:gmldescription>
>> <cdf:gmlname>singleFeature</cdf:gmlname>
>>
>>
>>
>> This should read:
>>
>> <gml:description>A Single Feature used to test returning of properties</cdf:gmldescription>
>> <gml:name>singleFeature</cdf:gmlname>
>>
>> The problem has to do with extracting these from the database ...
>>
>> The Other table is defined as:
>>
>> Table "public.Other"
>> Column | Type | Modifiers
&gt;&gt; ---------------<del><ins></del>----------------<del></ins></del>--------- &gt;&gt; gmldescription | character varying |
&gt;&gt; gmlname | character varying |
&gt;&gt; boundedBy | geometry |
&gt;&gt; pointProperty | geometry |
&gt;&gt; string1 | character varying | not null
&gt;&gt; string2 | character varying |
&gt;&gt; integers | integer |
&gt;&gt; dates | date |
&gt;&gt; Check constraints: "$1" (srid("pointProperty") = 32615)
&gt;&gt; "$2" ((geometrytype("pointProperty") = 'POINT'::text)
&gt;&gt; OR ("pointProperty" IS NULL))
&gt;&gt; "$3" (srid("boundedBy") = 32615)
&gt;&gt; "$4" ((geometrytype("boundedBy") = 'POLYGON'::text)
&gt;&gt; OR ("boundedBy" IS NULL))
&gt;&gt;
&gt;&gt;
&gt;&gt; David
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt; ------------------------------------------------------- &gt;&gt; The SF.Net email is sponsored by EclipseCon 2004
&gt;&gt; Premiere Conference on Open Tools Development and Integration
&gt;&gt; See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
&gt;&gt; http://www.eclipsecon.org/osdn - http://www.eclipsecon.org/osdn
&gt;&gt; _______________________________________________
&gt;&gt; Geoserver-devel mailing list
&gt;&gt; Geoserver-devel@lists.sourceforge.net
&gt;&gt; https://lists.sourceforge.net/lists/listinfo/geoserver-devel - https://lists.sourceforge.net/lists/listinfo/geoserver-devel
&gt;&gt;
&gt;
&gt; &#8211;
&gt;
&gt;
&gt;
&gt; ------------------------------------------------------- &gt; The SF.Net email is sponsored by EclipseCon 2004
&gt; Premiere Conference on Open Tools Development and Integration
&gt; See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
&gt; http://www.eclipsecon.org/osdn - http://www.eclipsecon.org/osdn
&gt; _______________________________________________
&gt; Geoserver-devel mailing list
&gt; Geoserver-devel@lists.sourceforge.net
&gt; https://lists.sourceforge.net/lists/listinfo/geoserver-devel - https://lists.sourceforge.net/lists/listinfo/geoserver-devel
&gt;

------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn - http://www.eclipsecon.org/osdn
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel - https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Environment

None

Activity

Show:
codehaus
April 10, 2015, 4:18 PM

CodeHaus Comment From: jgarnett - Time: Wed, 4 Feb 2004 17:31:53 -0600
---------------------
<p>These issues all involve intergrating the knowledge of GML and XS namespace contained in the GMLUtils class into schema genration and gml output routines.</p>

<p>Secondly this knowledge seems to want to migrate to geotools2.</p>

codehaus
April 10, 2015, 4:18 PM

CodeHaus Comment From: jgarnett - Time: Wed, 4 Feb 2004 17:33:12 -0600
---------------------
<p>The description for Geos-49 is more complete then Geos-50</p>

codehaus
April 10, 2015, 4:18 PM

CodeHaus Comment From: cholmes - Time: Sun, 15 Feb 2004 16:06:22 -0600
---------------------
<p>Related IRC conversation 2/12/04 ~11:00am pst on geotools channel.

(sorry for the bad formatting, chatzilla isn't so great on the saves)</p>

<p>dzwiers We have two cases expert user/non expert user:

dzwiers Non expert -&gt; no problems there other than bug#1.

dzwiers Expert user -&gt; Currently they can only create fragments, so cannot link outside the schema file (they need the header to do this ... and this is missing!). They can link inside the file, and this not being loaded is a bug (can fix that). Assuming we can parse a schema fragment (ie. what was there before we ever touched this), we can write this in two ways. Your example with internal linking is the better way, and I can certainly do th

dzwiers Proposed plan of action:

dzwiers fix the first bug (needs to be done!)

dzwiers fix reader to support infile linking (about 1hr)

dzwiers fix writer to write all complexTypes as links (about 2hr)

cholmes wait, which is bug one?

dzwiers was the writer bug

cholmes huh?

dzwiers where you had garbage returned to you

cholmes And how are you going to handle restriction bases? You're just going to read them and then spit them out at a later date, doing nothing with them?

cholmes And holding them in AttributeTypeMetaData?

dzwiers this doesn't have anything to do with atmd

dzwiers that's jody's stuff

dzwiers he was planning on killing it i though though

cholmes Ok, where are you storing the information about restriction bases?

dzwiers I'm just looking at parsing the config files for the ui and server

jgarnett Hi guys: sorry had a phone call there

dzwiers as an xml fragment in a dto object

cholmes Which dto object?

dzwiers if it's fancy i don't plan on touching it (AttributeTypeInfoDTO)

cholmes Which is the delegate for AttributeTypeInfo which is an implementation of AttributeTypeMetaData, right?

dzwiers was the delegate

dzwiers removed the delegates ... but yes

cholmes So now it's just an implementation of AttributeTypeMetaData, so it does have something to do with atmd.

cholmes or am I wrong?

dzwiers what we are parsing and how we do it does not have anything to do with how the data is used. It is parsed into a DTO object

dzwiers i'm only concerned with the xml &lt;-&gt; DTO part

cholmes I guess that sort of my problem, is that we should parse it to be able to <b>do</b> something with it.

cholmes Because that is valuable information.

dzwiers if it is all represented then it can be used as people see fit

cholmes That said, I may fine with hacking it in, and just holding the information somewhere in geoserver, but I don't want to see that code in geotools.

dzwiers i'm recomending that we start by representing some as Java structs and some as xml

dzwiers yah, only providing the fancy namespace stuff, not the parser to geotools

jgarnett Cool that is a nice clear direction: after we a figured out what we should see if we want <b>any</b> of this in Geotools

jgarnett (I am tempted to bring across a little bit for the XMLSchema generation that geotools already has)

cholmes Well, we want it in geotools, but we want it in geotools data structures where it's actually useful.

cholmes So Jody, you'll get rid of AttributeTypeMetaData in geotools?

jgarnett Catching up on email it appears that david has jumped several steps ahead, to the conflict over how to write out XMLSchema from GeoServer.

jgarnett Yes chris ATMD will die

jgarnett I said that much in email

cholmes Ok, just making sure.

dzwiers basicaly I can provide the same level of service as before i started parsing the stuff is we accept 2 things: The complex data definitions will not be parsed entirely and the expert user's document may be rearanged if they save from the UI

jgarnett David/Chris can you both walk through the following life cycle agreeing on every step of the way: load xml into dto, dto into geoserver, dto back to xml

jgarnett (See above comment about david jumping to the end)

cholmes Ok, I'm fine with experts stuff being rearranged if they use the ui, that's totally reasonable.

dzwiers no reason to go over the dto -&gt; geoserver, that's determined by the make up of the dto object

jgarnett chris keeps worring about it with resepect to atmd

dzwiers if the expert doesn't save ...

dzwiers I agree that the internal references should be supported, was an oversight on my part

cholmes As for complex data definitions not being parsed entirely - I'm hesitant, since I <em>like</em> not fucking with experts data, since I don't know xml schemas well enough to try to think of everything people will want to do.

cholmes But that said I've never had anyone really complain.

dzwiers they can't link other files in, it's a fragment

jgarnett I think davids point was that we have <b>already</b> limited the expert user so much, that we now stand a chance of parsing what we do allow them to write

cholmes So let's say this, if someone complains and actually wants that functionality you will either program stuff in to handle it or do the work to revert to schema.xml style, where we just store the whole thing.

dzwiers the most they can do is define new types in terms of XML schema and gml

jgarnett If they actually complain we would actually need to redesign the code to allow a full xml document and parse enough of it to mix the result with the describe type document

cholmes Ok, if you're confident in it (and I don't see any reason why you shouldn't be, if you've read xml schema spec and gml spec completely), then go ahead and take my offer.

jgarnett In short allowing the expert user to do more, even for the geoserver1.1.1 release would involve a lot of work

dzwiers they are limited to 2 namespaces unless they copy past into the doc ... but yes this solution could still screw the expert

jgarnett And boys we should try and keep our language in check since I like sucking these conversations up into jira

dzwiers &lt;xs:simpleType name="descType1"&gt;

dzwiers &lt;xs:restriction base="descType2"/&gt;

dzwiers &lt;/xs:simpleType&gt;

dzwiers &lt;xs:simpleType name="descType2"&gt;

dzwiers &lt;xs:restriction base="xs:string"/&gt;

dzwiers &lt;/xs:simpleType&gt;

dzwiers &lt;xs:element type="descType1" minOccurs="0" name="btrn_bc_id"

dzwiers nillable="false" maxOccurs="1"/&gt;

dzwiers this would not be correctly parsed ... and would be very hard to do

jgarnett So david does this mean we have to give up and store the whole text? Or does this mean we draw the line at one level of indirection in the file? Or should we draw the line at no levels of indirection.

jgarnett Remember what started us down this road was the extra minOccurs/maxOccurs properties file

jgarnett (and the ui request for featureType, not to mention the attributes list for reording of a genreated schema)

jgarnett I still would hate to keep three+one descriptions of attributes in sync

jgarnett That is both as a ui guy, a geoserver guy and a user

dzwiers my though is to store the whole xml, and parse for the ui. If they save it, then it gets replaced. If it's complex then the WHOLE doc is displayed ...

cholmes That's about my thought as well.

jgarnett David it still does not allow us access to the minOccurs/maxOccurs

dzwiers so it's either basic and the ui helps, or it's a box

jgarnett I thought we needed that at runtime

jgarnett Chris?

dzwiers min/max is at the element level, before the levels of abstraction and is relatively easy to parse out

cholmes You parse the min/max occurs, right? But store the file as it is.

jgarnett I had an email where I described 4 levels of madness: generatered, ordered, ordered and subtyped, or expert

cholmes huh?

jgarnett So david, we need to parse for the min/max occurs are you telling me we cannot do this for the two levels of indirection example above?

jgarnett Sorry I see you answered me already

cholmes In my opinion that's fine if we don't parse min/max occurs for two levels of indirection.

jgarnett Chris was this all we needed the parsing for?&gt;

jgarnett chris: several lines ago he said he could still handle min/max

jgarnett (I missed it too)

cholmes I think so...you guys were the ones who started doing the parsing...

jgarnett huh? I will repost: min/max is at the element level, before the levels of abstraction and is relatively easy to parse out

dzwiers might stuff the schema parsing in the UI side of things, make the dto pass the schema as one

dzwiers and then limited stuff if required in global

cholmes It was the putting it into datastructures that were messing up when I tried a few slightly more difficult things that I had issue with. I'd like to keep min/maxOccurs parsing if possible, and then just return the file of what the 'expert' wrote directly, without trying to put it into datastructures.

jgarnett Store the xml fragment as one big string at the FeatureTypeDTO level?

cholmes yes.

cholmes That's what you said too, right David? Or was a I misunderstanding?

dzwiers chris: can parse the attribute names with min/max out and store the schema as text.

cholmes Perfect.

cholmes I'm set.

cholmes (of course I'll have to test it out, but I'm pretty sure this is all I wanted).

jgarnett Remember that we need the GMLUtils stuff to figure out attribtue name for pointPropertyType and friends

dzwiers writing the schema will only work for some schemas created by the ui

jgarnett Can't the expert still mess us up, in the same way as pointPropertyType does?

dzwiers the others wont be writen

jgarnett huh?

dzwiers we won't write expert schemas, only read then

jgarnett Hmm or allow them to edit in a big text area?

cholmes No, they're experts, they don't need a big text area.

cholmes And we don't want to confuse normal users with a big text area.

jgarnett I still need a way to say "use file" in the big text area chris

jgarnett That is if they are using a file, I need to ui to show that

jgarnett bleck

cholmes How about a button that says 'use file'

dzwiers just do display only

dzwiers ie text area that is grey

jgarnett display only is good

jgarnett Maybe should do that for generated XMLSchema as well

jgarnett (the display only part)

cholmes That would actually be nice - when users put in ui buttons it shows what their generated schema is going to look like.

dzwiers sure, and a grey/nongrey part to user edits

cholmes Oh yeah, one small little thing on that (I didn't get to my small issues on email yet).

cholmes For the ui I don't think you need to have minOccurs/maxOccurs boxes.

cholmes Because we don't actually support maxOccurs other than 1

cholmes And we only support minOccurs of 0 and 1.

jgarnett So to back up for chris buy in: need to leave the expert user's text alone and have it "pass through the system". For jody buy in we need to have a ui thought out (and I will drop back to a big text area if we run out of time), to have geoserer work we need min/max occurs parsed

cholmes So I think you just need a 'mandatory' button, that changes minOccurs to 1.

cholmes Yes.

jgarnett Okay noted about the "manditory" button.

jgarnett David what do we need for buy in from you?

dzwiers so far i can do all this

dzwiers so i'm happy

jgarnett Yeah happy

cholmes +1 happy!

cholmes Ok, can I bring up another issue? It's another bug report, so it'll go in a different jira task.

jgarnett So we still need GMLUtils to associate attribute name at genearte and parse for ui time

jgarnett do we still need it for GeoServer min/max parsing

jgarnett Not yet chris, we are happy but not finished

dzwiers don't think so, but will see when i get to writing it

dzwiers think i just map mandatory to attr name

dzwiers i'll be looking at the old code from 1.1.1 for some of this probably

jgarnett huh? I though pointPropertyType did not provide a name, we had to follow references in GML land (which we were faking with GMLUtils)

jgarnett I still did not get an answer about attribute name being defined a couple of references back

jgarnett (within the same document)

dzwiers it took a schema in as text and created a feature type which is essentially what we need

dzwiers every element has either a 'name' or 'ref' attrbribute which mean the same thing to us

dzwiers that was what we were using gmlutils for during the parse mostly

cholmes Yeah, I liked that code - though it took me awhile to figure out what it was doing.

cholmes You should add a few more comments and examples before rolling it into geotools (and test cases)

cholmes hello? Am I still connected?

dzwiers yup

dzwiers thinking

jgarnett What I am trying to get at is a list of restrictions on the expert user,

jgarnett falling back to the old code won't help

jgarnett I am pretty sure it had these restrictions as well

jgarnett (just that nobody reported them)

cholmes You mean the pointProperty stuff?

jgarnett So I want use to think through what we are making, so we can be document what the expert user can and can't do

dzwiers did the second example i gave work before?

jgarnett I meen multiple levels of indirection in the expert user's schema.xml file

dzwiers with the internal references, ie does it work for 1.1.1?

jgarnett like in david's earlier example

cholmes Yeah, because we did no parsing of it, we just past it to the user.

cholmes We got minOccurs from duplicated information in the info.xml file.

jgarnett David thinks you did do parsing of it - in order to generate a feature type

dzwiers umm, how does that restrict the ft then?

cholmes (which I agree is bad, but it made it so we did no parsing).

dzwiers i noticed code for generating FTs from parts of the schema

dzwiers is it just an on/off of attributes?

cholmes Um...I don't think so.

cholmes Oh, yeah, it's just the on/off of attributes.

jgarnett david can we grab a link to that code in SF cvs, we can look at what we are talking about

dzwiers ah, though that actually did more

jgarnett ANd it is this on/off of attributes that would fail given david's earlier example

cholmes From another datastructure - the attributes. They could be present or not, and mandatory or not.

cholmes Ah.

jgarnett Personally I would be happeir limiting the expert user's schema.xml file, and actually doing something with it

dzwiers though it was all parsed and allplied as filters

dzwiers *applied

jgarnett Let me rephrase that - we are limiting the expert user, I want to figure out what the limitations are, and what we can do within those limitations.

dzwiers but then we aren't doing that now so no idea why i though that

dzwiers so if i understand: the schema is for the client machine, and provides an active attribute list for fts

cholmes The onus was on the geoserver admin to match his schema to his limited attributes.

jgarnett And I have been trying to reduce information duplication and make use of the schema.xml file to drive the attribute on/off and attribute ordering

dzwiers got it

jgarnett And I am afraid I need to either: a) restrict the expert user's creativity a bit more or b) back off pass the schema.xml through unfiltered and break out the duplication of information

jgarnett I am afraid we are breaking up for lunch soon, david has to head out

cholmes Thinking (though I'm starting to think we're spending way too much time on an issue that no one is really going to care about).

jgarnett And I should be working with richard on the ui

jgarnett Yep

dzwiers will read the rest in a while

cholmes Ok, I agree that duplication of information is bad, and I don't want to see it.

cholmes So let's go ahead and restrict the user like you want to Jody.

jgarnett I am leaning towards storing attribute list/order in info.xml file, and generating schema if needed, or trusting the expert user if they provide one

jgarnett blind trust <img class="emoticon" src="https://jira.codehaus.org/images/icons/emoticons/smile.gif" height="16" width="16" align="absmiddle" alt="" border="0"/>

cholmes And duplicating the information?

cholmes I think we've both just convinced the other of our points - I was going to go with limiting the experts creativity.

jgarnett Is the expert user's problem

jgarnett that is the other option

cholmes Just agreeing to go back and support it if and when someone actually complained (ie betting that no one actually cares about this issue).

jgarnett We need a clear write up, posted to the list and then a decision

cholmes Though I must say when I thought of that for maxFeatures I was definitely wrong.

cholmes Yeah, that'd be good, actually get feedback from users.

jgarnett This is a hard problem, we are guarnteed at least one split - schema vs DataStore

cholmes huh? what does that mean?

jgarnett We will always have duplicated information

jgarnett Even if it is only between the schema.xml and the actual DataStore FeatureType

jgarnett I agree that the is a lot of time for an expert user we have never met

cholmes seriously.

cholmes Ok, since you're the one who actually has to implement this, you can pull the trigger on the decision.

cholmes I just wanted you to understand where I was coming from.

jgarnett Okay - stick with what we have, on the grounds we can build a nicer ui out of it

cholmes So if you want to put it to the list go for it, if you want to just decide that's fine.

jgarnett And more of our user's will care about a ui <img class="emoticon" src="https://jira.codehaus.org/images/icons/emoticons/smile.gif" height="16" width="16" align="absmiddle" alt="" border="0"/>

cholmes Agreed.

jgarnett I will do the pros/cons and conclusion email

cholmes And we can also hold out for someone to really write a schema parser.

jgarnett So onto item # 2

cholmes And then just use that for all our expert users.

jgarnett Is this code have any place in gt2

jgarnett <img class="emoticon" src="https://jira.codehaus.org/images/icons/emoticons/smile.gif" height="16" width="16" align="absmiddle" alt="" border="0"/>

cholmes Which code?

jgarnett smily was for you holding out for a schema parser

jgarnett GMLUtils++ and the GeoServer schema generation

jgarnett It sounds like those two are good, and the limitied schema parsing is bad

cholmes Gml utils yes. GeoServer schema generation - are you using attributeTypeMetaData for it?

cholmes If so then no.

cholmes If you're just using FeatureTypes then yes.

jgarnett No attribute type meta data required

cholmes Where's the code? I don't think I've reviewed it.

jgarnett Okay, we are both good at decisions then

cholmes GmlUtils I liked - just needs better explanations and examples, and unit test coverage (preferably with xerces validation, which is a non-trival task).

jgarnett The code is making the FeatureTypeDTO from a FeatureType and then writing a XMLSChema from that

cholmes Where's the code?</p>

codehaus
April 10, 2015, 4:18 PM

CodeHaus Comment From: dzwiers - Time: Tue, 17 Feb 2004 14:54:10 -0600
---------------------
<p>Has been checked, fixed ... several times and also partially revamped. The schema generation has been opened up for all of geoserver.</p>

codehaus
April 10, 2015, 4:18 PM

CodeHaus Comment From: cholmes - Time: Tue, 17 Feb 2004 15:25:47 -0600
---------------------
<p>Not really depends upon, but 49 contains the irc where we resolved what to do about this.</p>

Assignee

Unassigned

Reporter

codehaus

Triage

None

Fix versions

Affects versions

Components

Priority

Medium
Configure