|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectopendap.crawler.ResponseCachePostgres
public class ResponseCachePostgres
Provide a cache for XML/HTTP response objects. This can hold both the DDX XML/Text version of the response/object and the Last Modified Time (LMT). The class thus can provide the basis for a simple HTTP 1.1 cache where a conditional GET can be used to eliminate repeat transfers of a Response. The cache can also be used directly to process a collection of Responses retrieved earlier. The cache uses Postgres to store the information responses (i.e., documents) and a ConcurrentHashMap (that it serializes those to disk files for persistence) to store the LMT times of the URLs visited. In order for Postgres to be used as the cache, the database must be setup. There must be a database called 'crawl_cache' and it should have a table whose name will be passed to the constructor of this class using the 'tableName' parameter. That table should have columns named: key, url and doc. Make these using "CREATE TABLE ddx_responses (key SERIAL PRIMARY KEY, url varchar(256), doc text);" where 'ddx_responses' is the value of 'tableName'. This class was modified from a version that could optionally use a hash map to store the responses.
Nested Class Summary | |
---|---|
class |
ResponseCachePostgres.ResponseCacheKeysEnumeration
|
class |
ResponseCachePostgres.ResponseVisitedKeysEnumeration
|
Constructor Summary | |
---|---|
ResponseCachePostgres(boolean readOnly,
java.lang.String cacheName,
java.lang.String tableName)
Build an instance of ResponseCachePostgres. |
|
ResponseCachePostgres(java.lang.String cacheName,
java.lang.String tableName)
Build an instance of ResponseCachePostgres. |
Method Summary | |
---|---|
protected void |
finalize()
This won't be called when an out of memory exception is thrown. |
java.lang.String |
getCachedResponse(java.lang.String URL)
Retrieve a Response document from the cache. |
long |
getLastVisited(java.lang.String URL)
When was this Response URL last visited? |
java.util.Enumeration<java.lang.String> |
getLastVisitedKeys()
Get all of the keys in the URL cache named when this class was instantiated. |
java.util.Enumeration<java.lang.String> |
getResponseKeys()
Get all of the keys in the Response document (postgres) cache. |
boolean |
isVisited(java.lang.String URL)
Has this URL been visited? |
void |
saveState()
Force the cache to save its state now. |
void |
setCachedResponse(java.lang.String URL,
java.lang.String doc)
Add a Response document to the cache using its URL as a key. |
void |
setLastVisited(java.lang.String URL,
long d)
Add or update the entry in Response URL cache. |
Methods inherited from class java.lang.Object |
---|
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public ResponseCachePostgres(java.lang.String cacheName, java.lang.String tableName) throws java.lang.Exception
cacheName
- The basename to use for the 'visited' cache.tableName
- The name for the Postgres table in the 'crawl_cache' database
where responses should be stored.
java.lang.Exception
ResponseCachePostgres(boolean readOnly, String cacheName, String tableName)
public ResponseCachePostgres(boolean readOnly, java.lang.String cacheName, java.lang.String tableName) throws java.lang.Exception
readOnly
- Is this cache object being created so that a client
program can read previously cached data? If this is true, the client
should never try to write to the cache.cacheName
- The basename to use for the 'visited' cache.tableName
- The name for the Postgres table in the 'crawl_cache'
database where responses should be stored.
java.lang.Exception
Method Detail |
---|
protected void finalize() throws java.lang.Exception
finalize
in class java.lang.Object
java.lang.Exception
public void saveState() throws java.lang.Exception
java.lang.Exception
public void setCachedResponse(java.lang.String URL, java.lang.String doc) throws java.lang.Exception
URL
- The URLdoc
- The Docuemnt
java.lang.Exception
public java.lang.String getCachedResponse(java.lang.String URL) throws java.lang.Exception
URL
- Get the document paired with this URL
java.lang.Exception
public java.util.Enumeration<java.lang.String> getResponseKeys() throws java.lang.Exception
java.lang.Exception
getLastVisitedKeys()
public long getLastVisited(java.lang.String URL)
URL
- The URL
public boolean isVisited(java.lang.String URL)
URL
- The URL
public void setLastVisited(java.lang.String URL, long d) throws java.lang.Exception
URL
- The URLd
- The Last modified time to be paired with the URL
java.lang.Exception
public java.util.Enumeration<java.lang.String> getLastVisitedKeys()
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |