SDMX Users Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - JamesGardner

Pages: [1] 2
1
I would just like to mention that the sdmx-js javascript sdmx client, now supports a lot more providers, and shows data in a pivot table (pivotUI.js)

2
Here is how some providers do it;
2.0 ABS : statistical usage warning on DataSet but no copyright warning on DataSet, No copyright notice on any structures
2.0 IStat: no copyright notice on Data, No copyright notice on structure
2.0 OECD: copyright in xml comment in message ID in data and structure, copyright in annotations on KeyFamily in list of keyfamilies, copyright in annotation on DataSet
2.0 Knoema: copyright in api documentation, no copyright notice on data or structure
2.1 ESTAT: no copyright notice in structure or data message
2.1 INSEE: no copyright notice in structure or data message

I think there are a couple of main ways of doing it;
as an annotation tag associated with the DataStructure and DataSet (you can also place annotations on codelists, concepts & just about everything else!)
Personally I think that for sdmx 2.1 files, copyright messages could also be placed in the footer as a TextType entry
sdmx 2.0 doesn't have a footer at all.. so you could only use Annotations

probably safest to put copyright notices as an annotation on whichever thing you want to claim copyright over (DataStructure,Codelist,Code,ConceptScheme,Concept,DataSet)
please, whatever you do.. dont just add a custom attribute to the DataSet tag... this for example; <DataSet copyright="me"> or <DataSet note="copyright(c) 2016 me">
this mixes your copyright notice up with columns in the dataset...

this is taken from near the end of an ABS SDMX 2.0 CompactData file;
                  </ABS:Series>
                  <ABS:Annotations>
                     <common:Annotation>
                        <common:AnnotationTitle>Statistical usage warning</common:AnnotationTitle>
                        <common:AnnotationText>ABS.Stat beta is continuing to be developed.  Data will be updated as soon as possible following its 11:30 am release on the ABS website.</common:AnnotationText>
                     </common:Annotation>
                  </ABS:Annotations>
               </ABS:DataSet>

you can add your own custom Annotations with different Titles and different AnnotationTexts
such as;

<common:Annotation>
<common:AnnotationTitle>Copyright</common:AnnotationTitle>
<common:AnnotationText>Copyright (c) 2016 me</common:AnnotationText>
</common:Annotation>
<common:Annotation>
<common:AnnotationTitle>License</common:AnnotationTitle>
<common:AnnotationText>Creative Commons By Attribution License 2.0</common:AnnotationText>
</common:Annotation>

Kind Regards
   James Gardner

3
Hi!
I am in the process of porting my java SDMX parser over to JavaScript.. here is a little demo using the OECD service and pure javascript;
http://rawgit.com/kalessine/sdmx-js/master/public_html/test.html
Thanks
 James Gardner

4
SDMX 2.1 Public review / SDMX-Json Question
« on: October 26, 2015, 08:46:41 AM »
Hi!
I have a question about the sdmx-json spec...
in the spec, there is an annotations array, this array can contain multiple annotations,

in sdmx-ml, annotations can be attached to either dataset, series or observation,

in sdmx-json is the fact that an annotation attached to an observation lost in the translation from sdmx-ml to sdmx-json? I can't see any way to put in the sdmx-json that the annotation is attached to an observation?? all the annotations are grouped together in the array with no field on the observation to denote that a particular annotation is associated with that observation?

Thanks in advance for clearing things up for me..
  -JG

5
Hi!
Here are some statistics from the International Labour Organisation on unemployment around the world...
in English, French and Spanish
http://www.notionworks.com.au/sdmxsax-faces/vis/22
I think this is unique because the browser's Accept-Language header is used to choose the language of the visual...

I have added a database to my SDMX visualisation.. and now you can save and load visualisations, search by categories...
http://www.notionworks.com.au/sdmxsax-faces/ at this page..

I am interested in feedback..
Regards
   James Gardner

6
I have now got an example of plotting sdmx data on a map....
the demo is here;
http://www.notionworks.com.au/sdmxsax-faces/
Instructions;
  1. Choose ABS as SDMX service. (next)
  2. Choose any Census Cube (any labeled 'T' with a number or 'B' with a number) (next)
  3. Change the 'Region' or 'ASGS2011' dimension to 'Geospatial'
      Change TimeX to 'TimeDropDown'
      Change Observation from 'ContinuousY' to 'ContinuousColour' (next)
  4. Choose 'Map 1' as adapter(next)
  6. Change the 'State' field to a state, and the "Region Type' to something like 'Statistical Area Level 2'
      and change the measure to something you are interested in (this varies for each cube) (click query)
  5. Wait some time for the vectors to load, this can be up to 2 minutes! (30mb geojson file)
     On some browsers (chrome, latest firefox) subsequent queries hit the local cache rather than download again
     so it gets faster after the first load.
   6. Send me feedback! I am interested in what other people would find useful / difficult
     (yes.. i know the bindings screen is a nightmare to use... I will make it simpler)
Thanks
    James Gardner

7
Hi!
Does anyone know of any other public sdmx data services besides this list?
ABS,OECD,IMF,ISTAT,ESTAT,ECB,UIS(UN),INEGI,ILO,Knoema,opendataforafrica.org

Thanks
   James Gardner

8
Hi All!
I just thought i would show off my latest demo project.. a web based sdmx visualiser...
so far it only shows timeseries data really.. i am working on a mapping component to visualise census data as well...

http://www.notionworks.com.au/sdmxsax-faces/

This is a very early web based demonstration that allows you to choose how you would like to filter the cube.. (ie - drop down list)
so far the only things you can choose is a drop down, or put a dimension at the series level (and you can only have 1 series!)
then it shows you a graph with the data on it.. and you can select different values from drop down boxes to show different slices of the cube..
supported data services:
ABS, ISTAT, OECD, IMF, ESTAT

Feedback is appreciated at jsg@internode.on.net
Thanks
   -James

9
Hi All!
I just thought i would post to show you a prototype i just put on SourceForge..
it allows you to browse the public SDMX services (ABS, OECD, IMF)..
it can discover a list of the datasets on the server..
download the structure of a dataset,
and then you can filter the dataset by the dimensions of the dataset...(single value, multiple values, or all)
then you can query, and it will show you a tabular form of the data you just selected...
(substituting names of the codes for the 'id's of the data)...

it is located here;
  http://sourceforge.net/projects/sdmx-sax/files/SdmxSax%20-%20SwingDemo.zip/download

it is my test prototype for opening sdmx documents..
Thanks
   James Gardner

10
Technical Specifications / Re: ConceptRefType
« on: May 03, 2014, 06:15:26 AM »
Hi!
I just thought i would mention.. i think
'ItemSchemeRefType' is missing agency, id and version...

   <xs:complexType name="ItemSchemeRefType">
      <xs:annotation>
         <xs:documentation>ItemSchemeRefType contains a complete set of reference fields for referencing any item scheme. The class and package a required so that the reference is explicit as to the exact object being referenced.</xs:documentation>
      </xs:annotation>
      <xs:complexContent>
         <xs:restriction base="ItemSchemeRefBaseType">
            <xs:attribute name="class" type="ItemSchemeTypeCodelistType" use="required"/>
            <xs:attribute name="package" type="ItemSchemePackageTypeCodelistType" use="required"/>
         </xs:restriction>
      </xs:complexContent>
   </xs:complexType>

once again its extended by restriction, and doesn't have the fields copied from the base type...
Regards
   James Gardner

11
SDMX Tools, technical implementations and case studies / Sdmx-Sax Tool
« on: August 13, 2013, 06:00:13 AM »
Hi!
I'm still working sporadically on sdmx-sax, a Java parser to get data out of an SDMX service...
It can query a REST registry for data structures, and query SOAP 2.0 services for Data Structures,
(list data structures, and dump some basic information from them)
once it has the DSD, it can go through all a 'DataMessage's values, and swap Strings for CodeType objects..
I'm just working on doing some basic queries now..
the basic premise of sdmx-sax is to try and keep things modern, ie - sdmx 2.1, so sdmx 2.0 structures are shoe-horned into an sdmx2.1 object model...
it's new home is at www.github.com/kalessine/sdmx-sax

this is some example code querying a SOAP service;
    @Test
    public void testABSList() throws IOException {
        Sdmx20SOAPQueryable queryable = new Sdmx20SOAPQueryable("ABS", "http://stat.abs.gov.au/sdmxws/sdmx.asmx");
        QueryableServiceRegistry registry = new QueryableServiceRegistry(queryable);
        List<DataStructureReferenceType> list = registry.listDataSets();
        Iterator<DataStructureReferenceType> it = list.iterator();
        while(it.hasNext()){
            DataStructureReferenceType ref = it.next();
            DataStructureType ds = registry.findDataStructure(ref.getRef().getAgencyId(), new IDType(ref.getRef().getId().toString()),ref.getRef().getVersion());
            ds.dump();
            System.out.println(ref.getRef().getAgencyId()+":"+ref.getRef().getId()+":"+ref.getRef().getVersion());
        }
    }
This code queries the ABS sdmx service for data structures (returned as a list of DataStructureReferences)
then loads the DataStructure, and dumps a little bit of information about the DataStructure to System.out

It can also piece together a complete set of dependant Codelists and ConceptSchemes from http://www.fao.org/figis/sdmx/
when a datastructure is retrieved from the RESTful service...

Regards
   James Gardner


12
Technical Specifications / ConceptRefType
« on: June 24, 2013, 04:09:36 AM »
Hi...
ConceptRefType seems to be designed for ConceptTypes to be packaged into ConceptSchemes
(there is a field 'maintainableParentID' that is required, and the codelisttypes point to 'Concept/conceptscheme'), however it is possible for concepts to live as identifiable types inside the <Concepts> tag..  (quite where they get their AgencyId and version i haven't figured out, since they don't extend MaintainableType at all)
I am trying to convert the sdmx 2.0 sample 'StructureSample.xml' into an sdmx 2.1 file.. and i am having problems thinking about what to do with the Concepts... I need the ConceptReference for the dimensions' ConceptReference... and ConceptReferences cannot point to maintainable artefacts.. only ConceptSchemes...
also.. ConceptRefType seems to be missing an AgencyID...
Thanks..
   James Gardner

13
Technical Specifications / Re: MetadataStructureRefType
« on: February 20, 2013, 05:38:27 PM »
it does, but it's just strange that in every other RefType the 'important' fields are listed in the subclass... for example in MetadataTargetRefType, the field 'ID" is listed in the subclass... just so i know that MetadataTargetRefType the important field is the 'ID' to use to lookup the appropriate object... i haven't found very helpful help on the xs:restriction tag at all.. and it doesn't fit in with all the other RefTypes listing the field used to lookup the object...
i think i have it now.. PayloadStructureType has a 'structureID' field.. so i guess 'ID" is the appropriate field to use to lookup the MetadataStructure...
Thanks for your help
   James Gardner

14
Technical Specifications / MetadataStructureRefType
« on: February 19, 2013, 10:46:25 PM »
Hi!
for MetadataStructureRefType, the type is constrained to only have a class and package target..
From SDMXCommonReferences.xsd
++SNIP++
      <xs:annotation>
         <xs:documentation>MetadataStructureRefType contains a set of reference fields for a metadata structure definition.</xs:documentation>
      </xs:annotation>
      <xs:complexContent>
         <xs:restriction base="StructureRefBaseType">
            <xs:attribute name="class" type="StructureTypeCodelistType" use="optional" fixed="MetadataStructure"/>
            <xs:attribute name="package" type="StructurePackageTypeCodelistType" use="optional" fixed="metadatastructure"/>
         </xs:restriction>
      </xs:complexContent>
++SNIP++
what field am i supposed to use to resolve this reference into a MetadataStructure when there's no Agency, ID, Version, or URN field on this type?
or is MetadataStructureRef just supposed to indicate that the target object is a Metadatastructure??
Thanks
   James Gardner


15
Technical Specifications / Re: Dataflow Constrained?
« on: November 10, 2011, 04:19:26 PM »
Hi Stratos!
Thanks for your prompt reply!
I have given up trying to use XMLBeans, and Jaxb(OpenSDMX) to parse DSDs since i found the source code to your data structure wizard... it is a lot better than the similar sax parser i was working on... i think i can modify your code and take what i like about my sax parser and add the extra behaviour to your model...

Thanks!
   James Gardner

Pages: [1] 2