嘗試性的把 Lucene 放到 Android 上面來跑,結果不是太理想,但仍是一些心得分享出來,省去後人嘗試的時間。

Lucene 要跑在 Android 上,第一個碰上的問題是,如何把 index files 傳到手機上去,在 Lucene 中,對 index 的讀取,是以目錄為單位的,所以說,無法把 index files 放在 apk 中直接讀取,一定要存放在 device or external storage 上,才能夠使用;或者是自己弄個虛擬目錄出來,不過,這會耗用過多的計憶體空間。

我是選用把 index 放在 'src/res/raw' 底下,讓他變成 apk 的一部份,省去在網路上找個空間來放置 index 的問題,當要更新 index 時,就重編個 apk 叫使用者更新就好。

在放 lucene index 時,如果你想用 compound format 的話,可以用底下的指令,把多個 index files 包裹成單一檔案 .cfs

// run REPL with 'scala -cp luke-3.4.0_1-all.jar'

import org.apache.lucene.store.FSDirectory
import org.apache.lucene.index._
import org.apache.lucene.analysis.standard._

val source = FSDirectory.open(new java.io.File("source"))
val dest = FSDirectory.open(new java.io.File("dest"))

// open source index
val reader = IndexReader.open(source)

// create writer for compound index.
val analyzer = new StandardAnalyzer(org.apache.lucene.util.Version.LUCENE_34)
val writer = new org.apache.lucene.index.IndexWriter(dir, analyzer, IndexWriter.MaxFieldLength.UNLIMITED)

// force writer always use compound index format.
writer.getMergePolicy.asInstanceOf[LogByteSizeMergePolicy].setNoCFSRatio(1.0)


// add source index to dest index.
writer.addIndexes(reader)
writer.optimized
writer.close

reader.close

接著,是把產生的 segment, segments_1, _0.cfs 拷到 src/res/raw 底下,讓這些檔案變成 .apk 的一部份。


接著,是要在第一次執行時,把這些 index 從 apk 中覆製到 SD 卡上或是機子上,這邊,我寫了個小工具來做這件事

import android.content.Context
import android.os.Environment
import android.util.Log

import com.bluetangstudio.android.disastermap.TaipeiDisasterApp.LogTag

import org.apache.commons.io.FileUtils
import org.apache.lucene.store.{FSDirectory, Directory}

import scala.collection.JavaConversions._
import java.io.File

/**
* Helper class that search for lucene index directories on the device. The search order is
* external storage first then local storage. If lucene index does not exist on device, this
* class will copy the index from the apk to the device storage.
*
* @param context the application context
* @param path the root folder name of the index directory to use and to look for.
* @param source the source of index resource to copy if index does not exist on the device.
* format: Seq[(filename, resourceId)]
*/
case class LuceneOpenHelper(context: Context, path: String, source: Seq[Tuple2[String, Int]]) {

/**
* create or open an Directory.
*/
def open(): Option[Directory] = {
val candidates = Seq(externalFolder, internalFolder).flatten

// find the folder with index in it.
val folder = candidates.filter(f => f.exists() && f.list().length > 0).headOption
val withIndex = folder.orElse(
candidates.find(
f => {
// ensure folder is available.
f.exists() || f.mkdirs() match {
// folder is not accessible
case false => false

case _ => {
Log.d(LogTag, "Duplicating index from apk to %s...".format(f))
source.foreach(
s => {
val is = context.getResources.openRawResource(s._2)
try {
FileUtils.copyInputStreamToFile(is, new File(f, s._1))
} finally {
is.close()
}
}
)
true
}
}
}
)
)

return withIndex.map(FSDirectory.open(_))
}

private def externalFolder: Option[File] = {
Environment.getExternalStorageState match {
case Environment.MEDIA_MOUNTED => Option(context.getExternalFilesDir(path))
case _ => None
}
}

private def internalFolder: Option[File] = {
return Option(new File(context.getFilesDir, path))
}

}

最後,是在 *App 上加上這段

object MyApp {
private val INDEX_DIRECTORY = "idx"

private val INDEX_FILES = Seq(
("_0.cfs", R.raw.idx_0),
("segments", R.raw.segments),
("segments_1", R.raw.segments_1)
)
}
class MyApp extends android.app.Application {

import MyApp._

private var _luceneSearcher: Option[IndexSearcher] = None

def luceneSearcher: Option[IndexSearcher] = {
if (_luceneSearcher.isEmpty) {
Log.d(LogTag, "Initializing new IndexSearcher...")
_luceneSearcher = LuceneOpenHelper(this, INDEX_DIRECTORY, INDEX_FILES).open().map(new IndexSearcher(_))
}
_luceneSearcher
}

override def onLowMemory() {
_luceneSearcher.foreach(s => s.close())
_luceneSearcher = None
}
}

這樣一來,就能在 Android 上面跑 lucene-core 了.