海量简单数据不想用SQL?试试高效的嵌入式数据库BerkeleyDBJE吧!(二)

2014-11-24 11:13:17 · 作者: · 浏览: 5
.getBytes("utf-8")); System.out.println(" ---- > put data with key :" + key); OperationStatus status = myDatabase.put(null, keyEntry, valEntry); System.out.println(" ---- > put status: " + status); //查询 System.out.println(" ---- > get key :" + key); DatabaseEntry valEntry4Get = new DatabaseEntry(); status = myDatabase.get(null, keyEntry, valEntry4Get, LockMode.DEFAULT); if (status == OperationStatus.SUCCESS) { value = new String(valEntry4Get.getData(), "utf-8"); System.out.println(" ---- > get key success value:" + value); } else { System.out.println(" ---- > get key failed."); } //更新 System.out.println(" ---- > update data with key :" + key); value = "Hello Google"; status = myDatabase.put(null, keyEntry, new DatabaseEntry(value.getBytes("utf-8"))); System.out.println(" ---- > update status: " + status); System.out.println(" ---- > after update get key :" + key); status = myDatabase.get(null, keyEntry, valEntry4Get, LockMode.DEFAULT); if (status == OperationStatus.SUCCESS) { value = new String(valEntry4Get.getData(), "utf-8"); System.out.println(" ---- > get success key value:" + value); } else { System.out.println(" ---- > get key value failed."); } //删除 System.out.println(" ---- > delete key :" + key); status = myDatabase.delete(null, keyEntry); System.out.println(" ---- > delete status: " + status); System.out.println(" ---- > after delete get key :" + key); status = myDatabase.get(null, keyEntry, valEntry4Get, LockMode.DEFAULT); if (status == OperationStatus.SUCCESS) { value = new String(valEntry4Get.getData(), "utf-8"); System.out.println(" ---- > get success key value:" + value); } else { System.out.println(" ---- > get key value failed."); } } catch (Exception e) { e.printStackTrace(); } finally { if (null != myDatabase) { myDatabase.close(); } if (null != mydbEnv) { // 在关闭环境前清理下日志 mydbEnv.cleanLog(); mydbEnv.close(); mydbEnv = null; } } } /** * @param args */ public static void main(String[] args) { System.out.println(" ==== >
Demo Test Start < ===="); String envHomePath = "D:/test"; String databaseName = "first-db"; SimpleDemo.testSimpleData(envHomePath, databaseName); System.out.println(" ==== > Demo Test End < ===="); } }

自定义TupleBinding来对复杂对象的读写

首先自定义一个类继承TupleBinding,并把模型WebURL要存入DB的属性加上:
public class WebURLTupleBinding extends TupleBinding {

	@Override
	public WebURL entryToObject(TupleInput input) {
		WebURL webURL = new WebURL();
		webURL.setURL(input.readString());
		webURL.setDocid(input.readInt());
		webURL.setParentDocid(input.readInt());
		webURL.setParentUrl(input.readString());
		webURL.setDepth(input.readShort());
		webURL.setPriority(input.readByte());
		webURL.setAnchor(input.readString());
		webURL.setElement(input.readString());
		return webURL;
	}

	@Override
	public void objectToEntry(WebURL url, TupleOutput output) {		
		output.writeString(url.getURL());
		output.writeInt(url.getDocid());
		output.writeInt(url.getParentDocid());
		output.writeString(url.getParentUrl());
		output.writeShort(url.getDepth());
		output.writeByte(url.getPriority());
		output.writeString(url.getAnchor());
		output.writeString(url.getElement());
	}
}
用自定义类TupleBinding转化后即可存入DB:

		DatabaseConfig dbConfig = new DatabaseConfig();
		dbConfig.setAllowCreate(true);
		Database urlsDB = env.openDatabase(null, dbName, dbConfig);
		WebURLTupleBinding webURLBinding = new WebURLTupleBinding();
		DatabaseEntry value = new DatabaseEntry();
		WebURL url = new WebURL();
		webURLBinding.objectToEntry(url, value); 
		urlsDB.put(txn, getDatabaseEntryKey(url), value);

JAVA开源爬虫crawler4j就是使用JE进行URL存储的,测过数据量在百万以下都是非常快的,百万以上也还可以接受,千万就不知道了,个人觉得到百万以上还是用非嵌入式数据库好一些!