opendap.crawler
Class DDXRetriever

java.lang.Object
  extended by opendap.crawler.DDXRetriever

public class DDXRetriever
extends java.lang.Object

Use this with crawlers to manage a collection of DDX responses and the URLs that reference them. This class can cache just the URLs, or both the URLs and the matching responses. It can perform conditional gets as well, using postgres (via ResponseCachePostgres) as a simple HTTP/1.1 cache.

Author:
jimg

Constructor Summary
DDXRetriever(boolean readOnly, java.lang.String namePrefix)
           
DDXRetriever(java.lang.String cacheName)
           
 
Method Summary
 void cacheDDXURL(java.lang.String ddxUrl)
           
 java.lang.String getCachedDDXDoc(java.lang.String DDXURL)
           
 long getCachedDDXLMT(java.lang.String DDXURL)
           
 java.util.Enumeration<java.lang.String> getCachedDDXURLs()
           
 java.lang.String getDDXDoc(java.lang.String DDXURL)
          Given a URL to a DDX, get the DDX document.
 boolean isWellFormedDDX(java.lang.String ddxString)
          Simple method to test if the DDX will parse.
static void main(java.lang.String[] args)
          This 'main' has many conflicting features/options.
 void saveDDXCache()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DDXRetriever

public DDXRetriever(java.lang.String cacheName)
             throws java.lang.Exception
Throws:
java.lang.Exception

DDXRetriever

public DDXRetriever(boolean readOnly,
                    java.lang.String namePrefix)
             throws java.lang.Exception
Throws:
java.lang.Exception
Method Detail

main

public static void main(java.lang.String[] args)
This 'main' has many conflicting features/options. I've changed the function of this class considerably since it was written.

Parameters:
args -

isWellFormedDDX

public boolean isWellFormedDDX(java.lang.String ddxString)
Simple method to test if the DDX will parse. Generally there's no need to call this but it'll be useful when developing the crawler.

Parameters:
ddxString - The DDX to test
Returns:
true if the DDX parses, false if the SAX parser throws an exception

getDDXDoc

public java.lang.String getDDXDoc(java.lang.String DDXURL)
                           throws java.lang.Exception
Given a URL to a DDX, get the DDX document. If the DDXRetriever was built with caching turned on, this uses a poor man's HTTP/1.1 cache based on Last Modified Times. If caching is on, then calling this on a series of DDX URLs will fill the cache. If the cache is saved and later used again it is possible to re-read the URLs straight from the cache.

Parameters:
DDXURL - Get the DDX referenced by this URL
Returns:
The DDX document, in a String
Throws:
java.lang.Exception
See Also:
getCache()

cacheDDXURL

public void cacheDDXURL(java.lang.String ddxUrl)
                 throws java.lang.Exception
Throws:
java.lang.Exception

getCachedDDXURLs

public java.util.Enumeration<java.lang.String> getCachedDDXURLs()

getCachedDDXDoc

public java.lang.String getCachedDDXDoc(java.lang.String DDXURL)
                                 throws java.lang.Exception
Throws:
java.lang.Exception

getCachedDDXLMT

public long getCachedDDXLMT(java.lang.String DDXURL)
                     throws java.lang.Exception
Throws:
java.lang.Exception

saveDDXCache

public void saveDDXCache()
                  throws java.lang.Exception
Throws:
java.lang.Exception