Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

Real World Experience with Oracle XML Database 11g An Oracle ACE’s Perspective Marco Gralike – AMIS – OOW 2008

Who Am I Started as DBA with Oracle 7 on Windows NT 3.1 (1994) Experienced with Oracle 7.x / 8.x / 9.x / 10.x and 11.1 Started with Oracle XMLDB in 2003 (Oracle 10.1.3) Oracle 11g Beta tester for Oracle XMLDB Dedicated XMLDB Blog (blog.gralike.com) Active Oracle OTN XMLDB Forum Member Oracle ACE Award for XMLDB Community Contributions

Disclaimer Pay attention to: “ Choice” “ Design” The following are “ Rules of Numb ” Every environment has its own unique criteria and needs… “ Testing” “ Performance” “ Maintainability” “ Extendibility”

Overview XML IS NOT RELATIONAL What is Oracle XML Database Set-up and Configuration XMLType Storage The Protocol Server The Repository Data Handling Security Troubleshooting

Overview Small description / Introduction of the functionality Encountered issues OTN XMLDB Forum FAQ Hits on keyword, tags on my personal blog Personal view on XMLDB Topics / Tips & Tricks

Let’s Presume it is Relational

Query Optimizer - Access Paths

Structured Data Unstructured (XML) Data

How to Containerize a Tree? Keeping it intact? Too much empty space…! Access Optimization..? Shredding it? Rebuilding and shredding off the tree structure…? There is no solution for this…yet…

Impedance Mismatch Different data models. E.g. XPath models an XML document as a tree, while most general purpose programming languages have no native data types for a tree. Different programming paradigms. XSLT is a functional language, while Java is object-oriented , and Perl is a procedural one. Source: Wikipedia SXML

Issues in General NO PROBLEM Data Shipping / Transport XML via XML UI’s that “Understand”, are based on XML PROBLEM Accessing Data in XML structures CPU Intensive Memory Intensive  ORA-04030, XML Fragments, DOM Transformation: XML  Relational Risk : Data / Information Loss

So the issues you will encounter with XML The nature of XML: It is (most of the time) not Structured It can not be stored in a Universal XML “Container” It can not be accessed via a standard Access Path Method that “understands” the free format XML structure all the time… It is all about free format OK and Not OK Standards: W3C, Oasis, etc Performance problem induced by its Nature… Nature is not structured…(as is Human Nature)

Why Oracle XML Database? File Storage XML Database Native XML Enabled Oracle XMLDB A World within a World More than one Option Complies to Standards Still License Free Core part of (OXJR)DBMS The (relational) foundation is there and can also be used dealing with XML

Oracle XML Database SQL*Net Protocol Server Thin, Thick Clients XMLDB Funtionality

XML Handling XML data exchange SQL, C, C++, Java, PL/SQL, .Net, etc Via Protocol Listener supported methods Via Oracle Advanced Queuing methods (Oracle 11gR1) XML data storage CLOB, Object Relational, Binary XML (11gR1) XML data validation XMLSchema, Programmatically, via Storage definitions XML data creation XML Operators, Packages

Based on (XML) Standards World Wide Web Consortium (W3) XML, XML Schema, XSL, XSLT, XPath, XQuery, (11gR1) XLink, XInclude International Standard Organisation (ISO) SQL/XML Internet Engineering Task Force (IETF) FTP, HTTP(s), WebDAV They don’t always agree on solutions… How are we going to solve those problems… 

Making Choices - Versions 8i 9i 10g 9iR2 10gR2 XDK XMLType XSQL XDB XQuery Robustness 11g Binary XML, Partitioning XMLIndex, NDWS Performance, Strictness to Standards

Memory Structures and Background Processes JAVA_POOL (Java Classes, Wrappers) LARGE_POOL (Shared Server, aka MTS) PGA_AGGREGATE_TARGET (DOM) LOCAL_LISTENER = {NOT_DEFAULT_LISTENER_NAME} DISPATCHERS = "(PROTOCOL=TCP) (SERVICE={SID}XDB)“ SHARED_SERVERS = 5  Security / Performance Issues: 10.1, 10.2, 11.1

National Langue Settings CHARACTER SET: AL32UTF8 NATIONAL CHARACTER SET: AL16UTF16 NLS_LENGTH_SEMANTICS ='BYTE'; Before $ORACLE_HOME\RDBMS\ADMIN\catqm.sql SQL>@?/rdbms/admin/catnoqm Avoid “Strange” NLS related errors: Registering an XML Schema via WebDAV, Drag & Drop Registering an XML Schema via BFILENAME

OTN XMLDB Forum - FAQ Database character set is AL32UTF Client character set is WIN1252 . SQL> drop table test; SQL> create table test of xmltype; SQL> insert into test values (xmltype('<?xml version="1.0" encoding=" UTF-8 "?><FOO/>')); SQL> select object_value from test; OBJECT_VALUE --------------------------------------------------- <?xml version="1.0" encoding=" WINDOWS-1252 "?><FOO/>

NLS FAQ Continued SQL> select x.object_value.getClobVal() from test x; X.OBJECT_VALUE.GETCLOBVAL() -------------------------------------------- <?xml version="1.0" encoding=" UTF-8 "?><FOO/> SQL> exit C:\TEMP>set NLS_LANG=AMERICAN_AMERICA. AL32UTF8 C:\TEMP>sqlplus scott/tiger SQL> select * from test; SYS_NC_ROWINFO$ -------------------------------------------- <?xml version="1.0" encoding=" UTF-8 "?><FOO/>

Database COMPATIBLE (e.g. XQuery 10.2.0.1 <> 10.2.0.3) RECYCLE_BIN copy_evolve (xml evolution) dropping schemata Control via XMLType Storage (  Containers, Models)

Relational Contra XML Data 1 Record = ???

XMLTYPE Storage Logical Design Level Physical Design Level

Choose but choose wisely Think in Terms of… Design Usage of XML Schemata? Future Extendibility XML Evolution Maintainability Backup & Recovery Access Paths, The Cost Base Optimizer = Performance Selecting Data Indexes

CLOB, Object Relational, Binary XML XMLType (Tables) XMLType Views XMLDB Data Storage (Object) Relational Objects Relational Tables Relational World XMLDB World Binary XML Structured Mixed complex[y] BINARY XSD [y/n] XMLIndex Content complex[n] OR XSD [y] BTree, IOT Hybrid Content complex[n] Mixed XSD [y] BTree, IOT Unstructured Document na CLOB XSD [n] XMLIndex

Generate or Manual Creation Generate via DBMS_XMLSCHEMA XML Schema Based XDB Annotations JDeveloper or XMLSpy Best Practise Manual Creation DBMS_METADATA.GET_DDL EVENT setting alter session set events='31098 trace name context forever'; /oracle/diag/rdbms/xmldb/XMLDB/trace Use Oracle 11g generation to apply in 10gR2 (IOT <> BTREE)

Storage Factors Design Unstructured contra Structured (XML Schema) Validation, CPU Data Optimization Binary XML (De)Composition of Structures Flexibility Binary XML contra Object Relational

Continued… Whitespace More storage needed Do you remember Nullology? “ Air” has to be processed as well Amount of Storage Binary XML contra Object Relational Nested Storage contra Out of Line XML Partitioning

Big XML Schemata (Object Relational) Out of Line Storage Twice the amount of inserts Avoiding the 1000 columns per table limit Nested Table Storage More memory consumption can lead to  ORA-04030 Memory needed for all the number of attributes in all descendants of the object http://forums.oracle.com/forums/message.jspa?messageID=2736485

Equi Partitioning EMP_PROJ_P11 reference_id EMPLOYEES_PROJ_TAB “ employees”.”employee” EMP_PROJ_P12 PROJ_DETAILS_TAB

Interfacing Build on Shared Server (aka MTS) Config file: xdbconfig.xml “ Extension” on the Oracle Listener Enables access for / via: WebDAV, HTTP(s), FTP ASM DBUri’s Foundation for Your “local” APEX implementation Native Database Web Services (NDWS) Your own Servlet’s, PL/SQL

Possibilities via the Protocol Server WebDAV FTP HTTP(s)

Native Database Web Services WSDL Structure Architecture

NDWS is Simple Enable the Protocol Server for HTTP access Enable the orawsv entry point in xdbconfig.xml Create a user and grant access DBMS_NETWORK_ACL_ADMIN XDB_WEBSERVICES XDB_WEBSERVICES_OVER_HTTP XDB_WEBSERVICES_WITH_PUBLIC Create your procedure - for instance a proc. SQUARE TEST: http://localhost:8080/orawsv/SCOTT/SQUARE?wsdl

The Repository A foundation between the inside and outside world /sys should not be used. XDB$RESOURCE Security via ACL, XML, References to objects, etc Contains info about registered XML schema, its objects, etc Out of the Box, Simple CMS Security Oracle Search Accessible via WebDAV and SQL Store for, e.g. scripting, etc

Storing your Scripts in the Repository SELECT xmlquery( xdburitype('/public/q1.xqy').getClob() passing OBJECT_VALUE returning content) FROM MY_XML_TABLE / declare res BOOLEAN; begin res := dbms_xdb.createResource ('/public/q1.xqy',<My Xquery>); commit; end; /

XML Operators & Functions in 11gR1 XMLAGG XMLCAST XMLCDATA XMLCOLATTVAL XMLCOMMENT XMLCONCAT XMLDIFF XMLELEMENT XMLEXISTS XMLFOREST XMLPARSE XMLPATCH XMLPI XMLQUERY XMLROOT XMLSEQUENCE XMLSERIALIZE XMLTABLE XMLTRANSFORM … DELETEXML EXTRACTVALUE EXISTNODE EXTRACT INSERTCHILDXML INSERTXMLBEFORE SYS_XMLAGG SYS_XMLGEN UPDATEXML …

Changes More strict in using Standards from: 9.2  10.x  11.x Namespaces required while using functions like EXTRACT “ Pretty Print” EXTRACT(xmltype,’/*’) XMLSERIALIZE Moving from packages to operators Use SQL/XML Operators instead of DBMS_XMLGEN

Extends the Relational World XML Operators, etc, can be used for dealing with SQL Extra Opportunities for data handling / interfacing WebDAV, FTP Storage and Retrieval of data outside the database XQuery, XMLTable

Selecting XML data from your Local Disk SQL> SELECT 2 xdata 3 FROM ( XMLTABLE 4 ('*' 5 passing 6 ( XMLTYPE ( bfilename ('XMLSTORE','data.xml') 7 ,NLS_CHARSET_ID('AL32UTF8'))) 8 columns xdata xmltype PATH '/*')); XDATA ------------------------------------ <root><content>Text</content></root> 1 row selected.

Using XML operators with SQL SQL> SELECT 2 table_name, 3 to_number( 4 extractvalue( 5 xmltype(dbms_xmlgen.getxml 6 ('select count(*) C from '||table_name)) 7 ,' /ROWSET/ROW/C ')) count 8 FROM user_tables; TABLE_NAME COUNT ------------------------------ ------ DEPT 4 EMP 14 BONUS 0 SALGRADE 5 Source Laurent Schneider : How do i store the counts of all tables …

RSS Example SQL> SELECT * 2 FROM XMLTABLE (XMLNAMESPACES('http://purl.org/dc/elements/1.1/' AS "MGR") 3 ,'//item' 4 passing HTTPURITYPE ('http://my.blog.com/blog/?feed=rss2').getXML() 5 columns title varchar2(50) path '/item/title/text()', 6 link varchar2(50) path '/item/link/text()', 7 publication_date varchar2(50) path '/item/pubDate/text()', 8 creator varchar2(50) path '/item/MGR:creator/text()', 9 description varchar2(250) path '/item/description/text()', 10 category XMLTYPE path '/item/category/text()');

Accessing the outside World TITLE LINK PUBLICATION_DATE CREATOR ----------------------------------------------------------------- DESCRIPTION CATEGORY ----------------------------------------------------------------- Quering RSS Feeds The XMLDB Way http://feeds.feedburner.com/~r/Bloggralikecom/~3/3 Wed, 25 Jun 2008 16:47:19 +0000 Marco Gralike Actually this IS old stuff (2006), but it got lost IN a… < ![CDATA[11g]]>< ![CDATA[Howto]]>

XMLDB Security Foundation It’s build in so why not (also) use it for your environment XMLDB: Core part of Oracle Database Across (SOA) Tiers: Oracle Web Service Manager OLAP Methods Roles Access Control Lists (ACL)

Statement of Direction? Access Control for Network Utility Packages ( Oracle 11gR1 Upgrade Guide ) The default behavior for access control to network utility packages has been changed to disallow network operations to all nonprivileged users. This default behavior is different from, and is incompatible with, previous versions of Oracle Database.

ORA-24247 ORA-24247 network access denied by access control list (ACL) Fine-Grained Access to External Network Services Packages involved (11gR1) UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP, UTL_INADDR Solution: DBMS_NETWORK_ACL_ADMIN Check, Add, Remove Privileges

Troubleshooting XMLDB Use What You Know: SET AUTOTRACE ON DBMS_XPLAN.DISPLAY TKPROF, SQL_TRACE EVENTS like 10046 etc. Schema Registration: ALTER session SET events = '31098 trace name context forever' Protocol Server: ALTER system SET event = '31098 trace name context forever, level 2' scope=spfile Trace files  UDUMP Directory

Recap… XML IS NOT RELATIONAL What is Oracle XML Database Set-up and Configuration XMLType Storage The Protocol Server The Repository Data Handling Security Troubleshooting

Oracle Open World 2008 - XMLDB Sessions

References Oracle 11g XMLDB Developers Guide The XMLDB Forum http:// forums.oracle.com/forums/forum.jspa?forumID =34 OTN XML DB FAQ Thread http://forums.oracle.com/forums/thread.jspa?threadID=410714 Technology Blogs http:// technology.amis.nl/blog http:// blog.gralike.com http:// laurentschneider.com

Questions? [email_address] http://www.amis.nl

Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

More Related Content

What's hot

Viewers also liked

Similar to Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

More from Marco Gralike

Recently uploaded

Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike