Compare commits

...

10 Commits

Author SHA1 Message Date
Ian Molee 070c9616a6 Add build and tests to Github Actions 2024-04-08 18:24:21 -07:00
Ian Molee 3e3ae58aa2 Add additional readme documentation and diagram 2024-04-08 18:13:02 -07:00
Ian Molee 6d683b7406 Add golangci-lint Github Action 2024-04-08 18:12:33 -07:00
Ian Molee 4f4c3f152a Add additional documents for e2e test 2024-04-08 18:12:07 -07:00
Ian Molee 0cced3a14e Add additional precision to output histogram
Show a single decimal place for the percentages in the overview
histogram shown on completion.
2024-04-08 18:08:28 -07:00
Ian Molee 3eec47c0b0 Add .dockerignore and adjust Dockerfile 2024-04-08 18:08:15 -07:00
Ian Molee 52373cff45 Update .gitignore; improve documentation
Add the `files` directories to the .gitignore to prevent them from being
committed again. Update the readme with the latest command line options,
and revise method documentation to match implementation.
2024-04-05 05:31:32 -07:00
Ian Molee e11464082b Allow file-based output and add some cosmetics
Allow the important data to be explicitly written to a file via a
command line switch. The default is still stdout, and redirecting
output will still only redirect the important data to the file, ignoring
summary data on stderr.

Add status during runtime and summary upon completion, for a better user
experience.
2024-04-05 05:01:31 -07:00
Ian Molee c8c2d9a9e0 Alter similarity calculation
Use a slightly more sophisticated method to determine similarity than
just trying to find duplicated lines, which falls apart fairly quickly.
Instead add value to the histogram while scanning the first file, and
subtract while scanning the second. After this, any entries with a
vvalue of 0 indicate matching lines. The magnitudes of anything elsefrom
zero are summed and used to calculate a similarity fraction.
2024-04-05 04:54:56 -07:00
Ian Molee 03c0840041 Split up worker and worker logic
Break the worker function into one that ranges over the channel and one
that actually does the work of associating the file with a document if
it is determined to match.
2024-04-05 02:51:11 -07:00
17 changed files with 1362 additions and 92 deletions

5
.dockerignore Normal file
View File

@ -0,0 +1,5 @@
*.log
output.*.txt
.vscode
files
files.*/

28
.github/workflows/golangci-lint.yml vendored Normal file
View File

@ -0,0 +1,28 @@
name: lint-build-test
on:
push:
branches:
- master
pull_request:
permissions:
contents: read
jobs:
lint-build-test:
name: lint-build-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
cache: false
- name: golangci-lint
uses: golangci/golangci-lint-action@v4
with:
version: latest
- name: build
run: go build -v ./...
- name: Run tests
run: go test

2
.gitignore vendored
View File

@ -1,3 +1,5 @@
*.log
output.*.txt
.vscode
files
files.*/

View File

@ -1,11 +1,19 @@
ARG bin_name=docgrouper
FROM golang:1.22 as builder
WORKDIR /go/src/docgrouper
COPY testdata testdata/
COPY *.go go.mod go.sum ./
ARG bin_name
WORKDIR /go/src/$bin_name
COPY . .
RUN go mod download
RUN go test -v ./... && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o docgroup
RUN go test -v ./... && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o $bin_name
FROM gcr.io/distroless/base-nossl-debian12
COPY --from=builder /go/src/docgrouper/docgrouper /bin/docgrouper
ARG bin_name
COPY --from=builder /go/src/${bin_name}/${bin_name} /bin/${bin_name}
VOLUME [ "/files" ]
ENTRYPOINT [ "docgroup" ]
# ARGs are stupid and cannot be used in ENTRYPOINT, so we still have to
# hard-code the binary name here, but this is still better than having to
# hard-code it in the stages above, since it doesn't really matter much what the
# entrypoint is called.
ENTRYPOINT [ "docgrouper" ]

View File

@ -26,6 +26,8 @@ arguments cannot be passed to a Makefile target.
## Options
```
-output string
output file (default is stdout)
-path string
path to the file pool (default "files")
-prefix
@ -37,3 +39,51 @@ arguments cannot be passed to a Makefile target.
-workers int
number of workers to use (default 2*<number-of-cores>)
```
# Approach
To satisfy the project requirements, the following approach was used:
1. Order files by timestamp, to be able to step through the "evolution" of the
associated documents.
2. Iterate through all the timestamps, and for all documents identified,
determine whether any of the files associated with the current timestamp can be
related to an existing document. For each file that remains unassociated with an
existing document at this point, create a new document.
3. Sort the associated files for each document in order by their filename (which
is a number).
## Practical considerations
The corpus provided for this assignment is comprised of 20,000 files; a
substantial enough amount to take time and memory complexity into account. To
manage time complexity, a pool of workers compares documents against candidate
files in a parallel manner, rather than serially stepping through all the files.
On my MacBook Pro M1 Max, this reduces runtime from 27 minutes to roughly five
minutes. Additionally, file contents are cached to prevent repeatedly reading
the same files off disk.
Cached files are purged as soon as their contents won't ever be needed again, to
conserve a large amount of memory that would be used by caching the contents of
20,000 files. We can be sure a file's contents aren't needed anymore when its
associated timestamp has already been processed, and the file isn't the most
recent file associated with a given document: we only need the latest version of
the document to associate files from future timestamps.
There is some status text written to the console by default, but it is written
to stderr, so if only the pure list of files associated with one another is
desired, redirecting stdout will yield a clean list without any status text. The
same can be accomplished by using the `-output` flag, as well.
# High level design
The main representational types in the program are the `DocumentManager` and the
`Document` types. `DocumentManager` handles starting workers and comparing
documents against candidate files. The `Document` type keeps track of which
files are associated with it, and the timestamp of the latest file. Work is
performed by sending "work items" to the workers on a channel, who pull a work
item off and perform it, altering documents if they need to be updated with a
new associated file. When all files have been checked, the final set of
documents is presented to the user.
![High level design](design.png)

BIN
design.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

180
main.go
View File

@ -13,6 +13,7 @@ import (
"flag"
"fmt"
"log/slog"
"math"
"os"
"path"
"runtime"
@ -21,6 +22,7 @@ import (
"strings"
"sync"
"sync/atomic"
"time"
)
const (
@ -38,37 +40,72 @@ const (
var (
dataFilePath string
similarityThreshold float64
outputFile string
useDocPrefix bool
verbose bool
numWorkers int
)
func main() {
documents, err := run(os.Args)
var (
start = time.Now()
padding = "\n"
)
documents, output, err := run(os.Args)
if err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(-1)
}
for _, doc := range documents {
fmt.Println(doc)
if output != os.Stdout {
defer output.Close()
padding = ""
}
duration := time.Since(start)
v := make(Visualizer)
for _, doc := range documents {
fmt.Fprintln(output, doc)
v.Add(*doc)
}
fmt.Fprintf(
os.Stderr,
"%s%d documents identified in %s\n\n%s\n",
padding,
len(documents),
duration.Truncate(time.Millisecond),
v.Render("Distribution for Number of Associated Files per Document", 60),
)
}
// run is the main entry point for the program.
func run(args []string) ([]*Document, error) {
func run(args []string) ([]*Document, *os.File, error) {
flags := flag.NewFlagSet(args[0], flag.ExitOnError)
flags.StringVar(&dataFilePath, "path", defaultDataFilePath, "path to the file pool")
flags.Float64Var(&similarityThreshold, "threshold", defaultSimilarityThreshold, "similarity threshold")
flags.StringVar(&outputFile, "output", "", "output file (default is stdout)")
flags.IntVar(&numWorkers, "workers", runtime.NumCPU()*2, "number of workers to use")
flags.BoolVar(&useDocPrefix, "prefix", false, "use '[doc ###]' prefix for output")
flags.BoolVar(&verbose, "verbose", false, "enable verbose logging")
_ = flags.Parse(args[1:])
output := os.Stdout
if outputFile != "" {
var err error
output, err = os.Create(outputFile)
if err != nil {
return nil, nil, fmt.Errorf("creating output file: %w", err)
}
}
shouldReportStatus := !verbose && output != os.Stdout
if shouldReportStatus {
defer fmt.Fprintln(os.Stderr)
}
// The files need to be processed in order of time, so determine the
// timestamp of each file and sort them by time.
fileTimes, times, err := orderFiles(dataFilePath)
if err != nil {
return nil, err
return nil, nil, err
}
dm := NewDocumentManager(dataFilePath, similarityThreshold, numWorkers)
@ -79,7 +116,7 @@ func run(args []string) ([]*Document, error) {
// documents, so we can identify unassociated files later and then
// create new documents for them. This needs to be distinct for each
// timestamp, so it's created inside the timestamp loop's scope.
associatedFiles sync.Map
claimedFiles sync.Map
// We might need to create new documents for files that weren't
// associated with any document at this timestamp, so we need to make
@ -90,13 +127,16 @@ func run(args []string) ([]*Document, error) {
)
log("processing timestamp", "timestamp", timestamp, "timestampIndex", i, "totalTimestamps", len(times))
if shouldReportStatus {
fmt.Fprintf(os.Stderr, "\rProcessing timestamp %d of %d...", i+1, len(times))
}
for i, doc := range dm.Documents {
wg.Add(1)
dm.WorkCh <- WorkItem{
doc: doc,
fileNumbers: fileTimes[timestamp],
timestamp: timestamp,
associatedFiles: &associatedFiles,
claimedFiles: &claimedFiles,
wg: &wg,
}
log(
@ -115,7 +155,7 @@ func run(args []string) ([]*Document, error) {
// documents for them.
var docsAdded int
for _, fileNumber := range fileTimes[timestamp] {
if _, ok := associatedFiles.Load(fileNumber); !ok {
if _, ok := claimedFiles.Load(fileNumber); !ok {
dm.AddNewDocument(fileNumber, timestamp)
docsAdded++
}
@ -129,7 +169,7 @@ func run(args []string) ([]*Document, error) {
}
dm.Shutdown()
return dm.SortedDocuments(), nil
return dm.SortedDocuments(), output, nil
}
// WorkItem is what will be sent to the the workers in the worker pool.
@ -137,7 +177,7 @@ type WorkItem struct {
doc *Document
fileNumbers []int
timestamp int
associatedFiles *sync.Map
claimedFiles *sync.Map
wg *sync.WaitGroup
}
@ -229,55 +269,52 @@ func (dm *DocumentManager) SortedDocuments() []*Document {
// document against each file and if a match is found, associate the file with the
// document sent in the work item, and record the file as having been matched.
func (dm *DocumentManager) ComparisonWorker(workerID int) {
defer dm.wg.Done()
for workItem := range dm.WorkCh {
for _, fileNumber := range workItem.fileNumbers {
if _, ok := workItem.associatedFiles.Load(fileNumber); ok {
// This file has already been matched; skip it.
dm.maybeAssociateFileWithDocument(workItem, workerID)
}
}
func (dm *DocumentManager) maybeAssociateFileWithDocument(workItem WorkItem, workerID int) {
defer workItem.wg.Done()
for _, candidateFileNumber := range workItem.fileNumbers {
if _, ok := workItem.claimedFiles.Load(candidateFileNumber); ok {
// This file has already been matched with another document, so skip it.
continue
}
latestFileNumber := workItem.doc.LatestAssociatedFile()
similarity, err := dm.compareFiles(latestFileNumber, fileNumber)
similarity, err := dm.compareFiles(latestFileNumber, candidateFileNumber)
if err != nil {
// Simplistic error handling: log the error and continue.
slog.Error(
"error comparing files",
"file1", latestFileNumber,
"file2", fileNumber,
"latestAssociatedFile", latestFileNumber,
"candidateFile", candidateFileNumber,
"document", workItem.doc.ID,
"worker", workerID,
)
}
// If current file doesn't match current document, skip to the next file.
if similarity < dm.similarityThreshold {
continue
}
// Current file matches current document, so record this.
workItem.doc.AssociateFile(fileNumber, workItem.timestamp)
workItem.associatedFiles.Store(fileNumber, struct{}{})
// If current file matches current document, record it and exit.
if similarity >= dm.similarityThreshold {
workItem.doc.AssociateFile(candidateFileNumber, workItem.timestamp)
workItem.claimedFiles.Store(candidateFileNumber, struct{}{})
log(
"match found",
"document", workItem.doc.ID,
"file", fileNumber,
"file", candidateFileNumber,
"time", workItem.timestamp,
"worker", workerID,
)
// We don't need to consider this document anymore since we've found
// a match. End processing and wait for more work.
break
return
}
workItem.wg.Done()
}
// Report that this worker is shutting down.
dm.wg.Done()
}
// compareFiles computes how much two files overlap, on a scale
// of 0 to 1 by iterating through the files and identifying lines
// that are duplicated.
// compareFiles computes how much two files overlap on a scale of 0 to 1 by
// iterating through the files and calculating a similarity score that's based
// on the number of line-centric differences between the contents of the two
// files.
func (dm *DocumentManager) compareFiles(f1Number, f2Number int) (float64, error) {
f1, err := dm.fcc.GetFileContents(f1Number)
if err != nil {
@ -289,23 +326,20 @@ func (dm *DocumentManager) compareFiles(f1Number, f2Number int) (float64, error)
}
histogram := make(map[string]int)
for _, lines := range [][]string{f1, f2} {
for _, line := range lines {
// Skip blank lines, which can throw off the count.
if line == "" {
continue
}
for _, line := range f1 {
histogram[line]++
}
for _, line := range f2 {
histogram[line]--
}
var overlap int
var differences float64
for _, v := range histogram {
if v == 2 {
overlap++
differences += math.Abs(float64(v))
}
}
return float64(overlap) / float64(len(histogram)), nil
similarity := 1 - (differences / float64(len(f1)+len(f2)))
return similarity, nil
}
// Document stores a document ID and a list of associated files.
@ -485,6 +519,58 @@ func orderFiles(dir string) (map[int][]int, []int, error) {
return timeMap, timeSlice, nil
}
// Visualizer is a utility to provide insight into the shape of the data
// processed.
type Visualizer map[int]int
func (v Visualizer) Add(d Document) {
numAssocFiles := len(d.AssociatedFiles)
v[numAssocFiles]++
}
func (v Visualizer) Render(title string, width int) string {
if len(v) == 0 {
return ""
}
type pair struct {
numAssocFiles int
numDocsWithThisNumFiles int
}
var (
slicedMap []pair
totalNumDocs int
)
for naf, nd := range v {
slicedMap = append(slicedMap, pair{numAssocFiles: naf, numDocsWithThisNumFiles: nd})
totalNumDocs += nd
}
slices.SortFunc(slicedMap, func(a, b pair) int {
return a.numDocsWithThisNumFiles - b.numDocsWithThisNumFiles
})
slices.Reverse(slicedMap)
var sb strings.Builder
sb.WriteString(title)
sb.WriteRune('\n')
sb.WriteString(strings.Repeat("=", width))
sb.WriteRune('\n')
scaleFactor := float64(totalNumDocs) / float64(slicedMap[0].numDocsWithThisNumFiles)
for _, p := range slicedMap {
ratio := float64(p.numDocsWithThisNumFiles) / float64(totalNumDocs)
numChars := int(math.Ceil(ratio * float64(width-14) * scaleFactor))
sb.WriteString(fmt.Sprintf(
"%3d | %s (%.1f%%)\n",
p.numAssocFiles,
strings.Repeat("*", numChars),
float64(p.numDocsWithThisNumFiles)*100/float64(totalNumDocs),
))
}
return sb.String()[0 : sb.Len()-1]
}
func makeFileName(number int) string {
return fmt.Sprintf("%d.txt", number)
}

View File

@ -102,22 +102,41 @@ func TestFileContentsCache(t *testing.T) {
}
func TestEndToEnd(t *testing.T) {
docs, err := run([]string{"argv0", "-path", "testdata/e2e"})
want := []int{1, 6, 9, 12, 14, 18}
docs, _, err := run([]string{"argv0", "-path", "testdata/e2e"})
want := []Document{
{
ID: 1,
AssociatedFiles: []int{1, 6, 9, 12, 14, 18},
LatestTimestamp: 5,
},
{
ID: 2,
AssociatedFiles: []int{2, 7, 13, 15, 22, 25},
LatestTimestamp: 5,
},
{
ID: 3,
AssociatedFiles: []int{11, 19},
LatestTimestamp: 7,
},
{
ID: 4,
AssociatedFiles: []int{21},
LatestTimestamp: 3,
},
}
if err != nil {
t.Fatal("error running program: ", err)
}
if len(docs) != 1 {
t.Fatalf("expected %d documents, got %d", 1, len(docs))
for i, doc := range docs {
if got, want := doc.ID, want[i].ID; got != want {
t.Errorf("expected ID %d, got %d", want, got)
}
doc := docs[0]
if doc.ID != 1 {
t.Errorf("expected ID %d, got %d", 0, doc.ID)
if want, got := doc.LatestTimestamp, want[i].LatestTimestamp; got != want {
t.Errorf("expected latest timestamp %d, got %d", want, got)
}
if doc.LatestTimestamp != 5 {
t.Errorf("expected latest timestamp %d, got %d", 3, doc.LatestTimestamp)
if want, got := doc.AssociatedFiles, want[i].AssociatedFiles; !reflect.DeepEqual(got, want) {
t.Errorf("expected associated files %v, got %v", want, got)
}
if !reflect.DeepEqual(doc.AssociatedFiles, want) {
t.Errorf("expected associated files %v, got %v", want, doc.AssociatedFiles)
}
}

133
testdata/e2e/11.txt vendored Normal file
View File

@ -0,0 +1,133 @@
7
CHAPTER 2. The Carpet-Bag.
I stuffed a shirt or two into my old carpet-bag, tucked it under my
arm, and started for Cape Horn and the Pacific. Quitting the good city
of old Manhatto, I duly arrived in New Bedford. It was a Saturday night
in December. Much was I disappointed upon learning that the little
packet for Nantucket had already sailed, and that no way of reaching
that place would offer, till the following Monday.
As most young candidates for the pains and penalties of whaling stop at
this same New Bedford, thence to embark on their voyage, it may as well
be related that I, for one, had no idea of so doing. For my mind was
made up to sail in no other than a Nantucket craft, because there was a
fine, boisterous something about everything connected with that famous
old island, which amazingly pleased me. Besides though New Bedford has
of late been gradually monopolising the business of whaling, and though
in this matter poor old Nantucket is now much behind her, yet Nantucket
was her great original—the Tyre of this Carthage;—the place where the
first dead American whale was stranded. Where else but from Nantucket
did those aboriginal whalemen, the Red-Men, first sally out in canoes
to give chase to the Leviathan? And where but from Nantucket, too, did
that first adventurous little sloop put forth, partly laden with
imported cobblestones—so goes the story—to throw at the whales, in
order to discover when they were nigh enough to risk a harpoon from the
bowsprit?
Now having a night, a day, and still another night following before me
in New Bedford, ere I could embark for my destined port, it became a
matter of concernment where I was to eat and sleep meanwhile. It was a
very dubious-looking, nay, a very dark and dismal night, bitingly cold
and cheerless. I knew no one in the place. With anxious grapnels I had
sounded my pocket, and only brought up a few pieces of silver,—So,
wherever you go, Ishmael, said I to myself, as I stood in the middle of
a dreary street shouldering my bag, and comparing the gloom towards the
north with the darkness towards the south—wherever in your wisdom you
may conclude to lodge for the night, my dear Ishmael, be sure to
inquire the price, and dont be too particular.
With halting steps I paced the streets, and passed the sign of “The
Crossed Harpoons”—but it looked too expensive and jolly there. Further
on, from the bright red windows of the “Sword-Fish Inn,” there came
such fervent rays, that it seemed to have melted the packed snow and
ice from before the house, for everywhere else the congealed frost lay
ten inches thick in a hard, asphaltic pavement,—rather weary for me,
when I struck my foot against the flinty projections, because from
hard, remorseless service the soles of my boots were in a most
miserable plight. Too expensive and jolly, again thought I, pausing one
moment to watch the broad glare in the street, and hear the sounds of
the tinkling glasses within. But go on, Ishmael, said I at last; dont
you hear? get away from before the door; your patched boots are
stopping the way. So on I went. I now by instinct followed the streets
that took me waterward, for there, doubtless, were the cheapest, if not
the cheeriest inns.
Such dreary streets! blocks of blackness, not houses, on either hand,
and here and there a candle, like a candle moving about in a tomb. At
this hour of the night, of the last day of the week, that quarter of
the town proved all but deserted. But presently I came to a smoky light
proceeding from a low, wide building, the door of which stood
invitingly open. It had a careless look, as if it were meant for the
uses of the public; so, entering, the first thing I did was to stumble
over an ash-box in the porch. Ha! thought I, ha, as the flying
particles almost choked me, are these ashes from that destroyed city,
Gomorrah? But “The Crossed Harpoons,” and “The Sword-Fish?”—this, then
must needs be the sign of “The Trap.” However, I picked myself up and
hearing a loud voice within, pushed on and opened a second, interior
door.
It seemed the great Black Parliament sitting in Tophet. A hundred black
faces turned round in their rows to peer; and beyond, a black Angel of
Doom was beating a book in a pulpit. It was a negro church; and the
preachers text was about the blackness of darkness, and the weeping
and wailing and teeth-gnashing there. Ha, Ishmael, muttered I, backing
out, Wretched entertainment at the sign of The Trap!
Moving on, I at last came to a dim sort of light not far from the
docks, and heard a forlorn creaking in the air; and looking up, saw a
swinging sign over the door with a white painting upon it, faintly
representing a tall straight jet of misty spray, and these words
underneath—“The Spouter Inn:—Peter Coffin.”
Coffin?—Spouter?—Rather ominous in that particular connexion, thought
I. But it is a common name in Nantucket, they say, and I suppose this
Peter here is an emigrant from there. As the light looked so dim, and
the place, for the time, looked quiet enough, and the dilapidated
little wooden house itself looked as if it might have been carted here
from the ruins of some burnt district, and as the swinging sign had a
poverty-stricken sort of creak to it, I thought that here was the very
spot for cheap lodgings, and the best of pea coffee.
It was a queer sort of place—a gable-ended old house, one side palsied
as it were, and leaning over sadly. It stood on a sharp bleak corner,
where that tempestuous wind Euroclydon kept up a worse howling than
ever it did about poor Pauls tossed craft. Euroclydon, nevertheless,
is a mighty pleasant zephyr to any one in-doors, with his feet on the
hob quietly toasting for bed. “In judging of that tempestuous wind
called Euroclydon,” says an old writer—of whose works I possess the
only copy extant—“it maketh a marvellous difference, whether thou
lookest out at it from a glass window where the frost is all on the
outside, or whether thou observest it from that sashless window, where
the frost is on both sides, and of which the wight Death is the only
glazier.” True enough, thought I, as this passage occurred to my
mind—old black-letter, thou reasonest well. Yes, these eyes are
windows, and this body of mine is the house. What a pity they didnt
stop up the chinks and the crannies though, and thrust in a little lint
here and there. But its too late to make any improvements now. The
universe is finished; the copestone is on, and the chips were carted
off a million years ago. Poor Lazarus there, chattering his teeth
against the curbstone for his pillow, and shaking off his tatters with
his shiverings, he might plug up both ears with rags, and put a
corn-cob into his mouth, and yet that would not keep out the
tempestuous Euroclydon. Euroclydon! says old Dives, in his red silken
wrapper—(he had a redder one afterwards) pooh, pooh! What a fine frosty
night; how Orion glitters; what northern lights! Let them talk of their
oriental summer climes of everlasting conservatories; give me the
privilege of making my own summer with my own coals.
But what thinks Lazarus? Can he warm his blue hands by holding them up
to the grand northern lights? Would not Lazarus rather be in Sumatra
than here? Would he not far rather lay him down lengthwise along the
line of the equator; yea, ye gods! go down to the fiery pit itself, in
order to keep out this frost?
Now, that Lazarus should lie stranded there on the curbstone before the
door of Dives, this is more wonderful than that an iceberg should be
moored to one of the Moluccas. Yet Dives himself, he too lives like a
Czar in an ice palace made of frozen sighs, and being a president of a
temperance society, he only drinks the tepid tears of orphans.
But no more of this blubbering now, we are going a-whaling, and there
is plenty of that yet to come. Let us scrape the ice from our frosted
feet, and see what sort of a place this “Spouter” may be.

75
testdata/e2e/13.txt vendored Normal file
View File

@ -0,0 +1,75 @@
1
CHAPTER 1. Loomings.
Call me Ishmael. Some years ago—never mind how long precisely—having
little or no money in my purse, and nothing particular to interest me
on shore, I thought I would sail about a little and see the watery part
of the world. It is a way I have of driving off the spleen and
regulating the circulation. Whenever I find myself growing grim about
the mouth; whenever it is a damp, drizzly November in my soul; whenever
I find myself involuntarily pausing before coffin warehouses, and
bringing up the rear of every funeral I meet; and especially whenever
my hypos get such an upper hand of me, that it requires a strong moral
principle to prevent me from deliberately stepping into the street, and
methodically knocking peoples hats off—then, I account it high time to
get to sea as soon as I can. This is my substitute for pistol and ball.
With a philosophical flourish Cato throws himself upon his sword; I
quietly take to the ship. There is nothing surprising in this. If they
but knew it, almost all men in their degree, some time or other,
cherish very nearly the same feelings towards the ocean with me.
There now is your insular city of the Manhattoes, belted round by
wharves as Indian isles by coral reefs—commerce surrounds it with her
surf. Right and left, the streets take you waterward. Its extreme
downtown is the battery, where that noble mole is washed by waves, and
cooled by breezes, which a few hours previous were out of sight of
land. Look at the crowds of water-gazers there.
But look! here come more crowds, pacing straight for the water, and
seemingly bound for a dive. Strange! Nothing will content them but the
extremest limit of the land; loitering under the shady lee of yonder
warehouses will not suffice. No. They must get just as nigh the water
as they possibly can without falling in. And there they stand—miles of
them—leagues. Inlanders all, they come from lanes and alleys, streets
and avenues—north, east, south, and west. Yet here they all unite. Tell
me, does the magnetic virtue of the needles of the compasses of all
those ships attract them thither?
No, when I go to sea, I go as a simple sailor, right before the mast,
plumb down into the forecastle, aloft there to the royal mast-head.
True, they rather order me about some, and make me jump from spar to
spar, like a grasshopper in a May meadow. And at first, this sort of
thing is unpleasant enough. It touches ones sense of honor,
particularly if you come of an old established family in the land, the
Van Rensselaers, or Randolphs, or Hardicanutes. And more than all, if
just previous to putting your hand into the tar-pot, you have been
lording it as a country schoolmaster, making the tallest boys stand in
awe of you. The transition is a keen one, I assure you, from a
schoolmaster to a sailor, and requires a strong decoction of Seneca and
the Stoics to enable you to grin and bear it. But even this wears off
in time.
What of it, if some old hunks of a sea-captain orders me to get a broom
and sweep down the decks? What does that indignity amount to, weighed,
I mean, in the scales of the New Testament? Do you think the archangel
Gabriel thinks anything the less of me, because I promptly and
respectfully obey that old hunks in that particular instance? Who aint
a slave? Tell me that. Well, then, however the old sea-captains may
order me about—however they may thump and punch me about, I have the
satisfaction of knowing that it is all right; that everybody else is
one way or other served in much the same way—either in a physical or
metaphysical point of view, that is; and so the universal thump is
passed round, and all hands should rub each others shoulder-blades,
and be content.
Again, I always go to sea as a sailor, because they make a point of
paying me for my trouble, whereas they never pay passengers a single
penny that I ever heard of. On the contrary, passengers themselves must
pay. And there is all the difference in the world between paying and
being paid. The act of paying is perhaps the most uncomfortable
infliction that the two orchard thieves entailed upon us. But _being
paid_,—what will compare with it? The urbane activity with which a man
receives money is really marvellous, considering that we so earnestly
believe money to be the root of all earthly ills, and that on no
account can a monied man enter heaven. Ah! how cheerfully we consign
ourselves to perdition!

158
testdata/e2e/15.txt vendored Normal file
View File

@ -0,0 +1,158 @@
3
CHAPTER 1. Loomings.
Call me Ishmael. Some years ago—never mind how long precisely—having
little or no money in my purse, and nothing particular to interest me
on shore, I thought I would sail about a little and see the watery part
of the world. It is a way I have of driving off the spleen and
regulating the circulation. Whenever I find myself growing grim about
the mouth; whenever it is a damp, drizzly November in my soul; whenever
I find myself involuntarily pausing before coffin warehouses, and
bringing up the rear of every funeral I meet; and especially whenever
my hypos get such an upper hand of me, that it requires a strong moral
principle to prevent me from deliberately stepping into the street, and
methodically knocking peoples hats off—then, I account it high time to
get to sea as soon as I can. This is my substitute for pistol and ball.
With a philosophical flourish Cato throws himself upon his sword; I
quietly take to the ship. There is nothing surprising in this. If they
but knew it, almost all men in their degree, some time or other,
cherish very nearly the same feelings towards the ocean with me.
There now is your insular city of the Manhattoes, belted round by
wharves as Indian isles by coral reefs—commerce surrounds it with her
surf. Right and left, the streets take you waterward. Its extreme
downtown is the battery, where that noble mole is washed by waves, and
cooled by breezes, which a few hours previous were out of sight of
land. Look at the crowds of water-gazers there.
Circumambulate the city of a dreamy Sabbath afternoon. Go from Corlears
Hook to Coenties Slip, and from thence, by Whitehall, northward. What
do you see?—Posted like silent sentinels all around the town, stand
thousands upon thousands of mortal men fixed in ocean reveries. Some
leaning against the spiles; some seated upon the pier-heads; some
looking over the bulwarks of ships from China; some high aloft in the
rigging, as if striving to get a still better seaward peep. But these
are all landsmen; of week days pent up in lath and plaster—tied to
counters, nailed to benches, clinched to desks. How then is this? Are
the green fields gone? What do they here?
But here is an artist. He desires to paint you the dreamiest, shadiest,
quietest, most enchanting bit of romantic landscape in all the valley
of the Saco. What is the chief element he employs? There stand his
trees, each with a hollow trunk, as if a hermit and a crucifix were
within; and here sleeps his meadow, and there sleep his cattle; and up
from yonder cottage goes a sleepy smoke. Deep into distant woodlands
winds a mazy way, reaching to overlapping spurs of mountains bathed in
their hill-side blue. But though the picture lies thus tranced, and
though this pine-tree shakes down its sighs like leaves upon this
shepherds head, yet all were vain, unless the shepherds eye were
fixed upon the magic stream before him. Go visit the Prairies in June,
when for scores on scores of miles you wade knee-deep among
Tiger-lilies—what is the one charm wanting?—Water—there is not a drop
of water there! Were Niagara but a cataract of sand, would you travel
your thousand miles to see it? Why did the poor poet of Tennessee, upon
suddenly receiving two handfuls of silver, deliberate whether to buy
him a coat, which he sadly needed, or invest his money in a pedestrian
trip to Rockaway Beach? Why is almost every robust healthy boy with a
robust healthy soul in him, at some time or other crazy to go to sea?
Why upon your first voyage as a passenger, did you yourself feel such a
mystical vibration, when first told that you and your ship were now out
of sight of land? Why did the old Persians hold the sea holy? Why did
the Greeks give it a separate deity, and own brother of Jove? Surely
all this is not without meaning. And still deeper the meaning of that
story of Narcissus, who because he could not grasp the tormenting, mild
image he saw in the fountain, plunged into it and was drowned. But that
same image, we ourselves see in all rivers and oceans. It is the image
of the ungraspable phantom of life; and this is the key to it all.
No, when I go to sea, I go as a simple sailor, right before the mast,
plumb down into the forecastle, aloft there to the royal mast-head.
True, they rather order me about some, and make me jump from spar to
spar, like a grasshopper in a May meadow. And at first, this sort of
thing is unpleasant enough. It touches ones sense of honor,
particularly if you come of an old established family in the land, the
Van Rensselaers, or Randolphs, or Hardicanutes. And more than all, if
just previous to putting your hand into the tar-pot, you have been
lording it as a country schoolmaster, making the tallest boys stand in
awe of you. The transition is a keen one, I assure you, from a
schoolmaster to a sailor, and requires a strong decoction of Seneca and
the Stoics to enable you to grin and bear it. But even this wears off
in time.
What of it, if some old hunks of a sea-captain orders me to get a broom
and sweep down the decks? What does that indignity amount to, weighed,
I mean, in the scales of the New Testament? Do you think the archangel
Gabriel thinks anything the less of me, because I promptly and
respectfully obey that old hunks in that particular instance? Who aint
a slave? Tell me that. Well, then, however the old sea-captains may
order me about—however they may thump and punch me about, I have the
satisfaction of knowing that it is all right; that everybody else is
one way or other served in much the same way—either in a physical or
metaphysical point of view, that is; and so the universal thump is
passed round, and all hands should rub each others shoulder-blades,
and be content.
Again, I always go to sea as a sailor, because they make a point of
paying me for my trouble, whereas they never pay passengers a single
penny that I ever heard of. On the contrary, passengers themselves must
pay. And there is all the difference in the world between paying and
being paid. The act of paying is perhaps the most uncomfortable
infliction that the two orchard thieves entailed upon us. But _being
paid_,—what will compare with it? The urbane activity with which a man
receives money is really marvellous, considering that we so earnestly
believe money to be the root of all earthly ills, and that on no
account can a monied man enter heaven. Ah! how cheerfully we consign
ourselves to perdition!
Finally, I always go to sea as a sailor, because of the wholesome
exercise and pure air of the fore-castle deck. For as in this world,
head winds are far more prevalent than winds from astern (that is, if
you never violate the Pythagorean maxim), so for the most part the
Commodore on the quarter-deck gets his atmosphere at second hand from
the sailors on the forecastle. He thinks he breathes it first; but not
so. In much the same way do the commonalty lead their leaders in many
other things, at the same time that the leaders little suspect it. But
wherefore it was that after having repeatedly smelt the sea as a
merchant sailor, I should now take it into my head to go on a whaling
voyage; this the invisible police officer of the Fates, who has the
constant surveillance of me, and secretly dogs me, and influences me in
some unaccountable way—he can better answer than any one else. And,
doubtless, my going on this whaling voyage, formed part of the grand
programme of Providence that was drawn up a long time ago. It came in
as a sort of brief interlude and solo between more extensive
performances. I take it that this part of the bill must have run
something like this:
“_Grand Contested Election for the Presidency of the United States._
“WHALING VOYAGE BY ONE ISHMAEL. “BLOODY BATTLE IN AFFGHANISTAN.”
Though I cannot tell why it was exactly that those stage managers, the
Fates, put me down for this shabby part of a whaling voyage, when
others were set down for magnificent parts in high tragedies, and short
and easy parts in genteel comedies, and jolly parts in farces—though I
cannot tell why this was exactly; yet, now that I recall all the
circumstances, I think I can see a little into the springs and motives
which being cunningly presented to me under various disguises, induced
me to set about performing the part I did, besides cajoling me into the
delusion that it was a choice resulting from my own unbiased freewill
and discriminating judgment.
Chief among these motives was the overwhelming idea of the great whale
himself. Such a portentous and mysterious monster roused all my
curiosity. Then the wild and distant seas where he rolled his island
bulk; the undeliverable, nameless perils of the whale; these, with all
the attending marvels of a thousand Patagonian sights and sounds,
helped to sway me to my wish. With other men, perhaps, such things
would not have been inducements; but as for me, I am tormented with an
everlasting itch for things remote. I love to sail forbidden seas, and
land on barbarous coasts. Not ignoring what is good, I am quick to
perceive a horror, and could still be social with it—would they let
me—since it is but well to be on friendly terms with all the inmates of
the place one lodges in.
By reason of these things, then, the whaling voyage was welcome; the
great flood-gates of the wonder-world swung open, and in the wild
conceits that swayed me to my purpose, two and two there floated into
my inmost soul, endless processions of the whale, and, mid most of them
all, one grand hooded phantom, like a snow hill in the air.

90
testdata/e2e/19.txt vendored Normal file
View File

@ -0,0 +1,90 @@
3
CHAPTER 2. The Carpet-Bag.
I stuffed a shirt or two into my old carpet-bag, tucked it under my
arm, and started for Cape Horn and the Pacific. Quitting the good city
of old Manhatto, I duly arrived in New Bedford. It was a Saturday night
in December. Much was I disappointed upon learning that the little
packet for Nantucket had already sailed, and that no way of reaching
that place would offer, till the following Monday.
As most young candidates for the pains and penalties of whaling stop at
this same New Bedford, thence to embark on their voyage, it may as well
be related that I, for one, had no idea of so doing. For my mind was
made up to sail in no other than a Nantucket craft, because there was a
fine, boisterous something about everything connected with that famous
old island, which amazingly pleased me. Besides though New Bedford has
of late been gradually monopolising the business of whaling, and though
in this matter poor old Nantucket is now much behind her, yet Nantucket
was her great original—the Tyre of this Carthage;—the place where the
first dead American whale was stranded. Where else but from Nantucket
did those aboriginal whalemen, the Red-Men, first sally out in canoes
to give chase to the Leviathan? And where but from Nantucket, too, did
that first adventurous little sloop put forth, partly laden with
imported cobblestones—so goes the story—to throw at the whales, in
order to discover when they were nigh enough to risk a harpoon from the
bowsprit?
Now having a night, a day, and still another night following before me
in New Bedford, ere I could embark for my destined port, it became a
matter of concernment where I was to eat and sleep meanwhile. It was a
very dubious-looking, nay, a very dark and dismal night, bitingly cold
and cheerless. I knew no one in the place. With anxious grapnels I had
sounded my pocket, and only brought up a few pieces of silver,—So,
wherever you go, Ishmael, said I to myself, as I stood in the middle of
a dreary street shouldering my bag, and comparing the gloom towards the
north with the darkness towards the south—wherever in your wisdom you
may conclude to lodge for the night, my dear Ishmael, be sure to
inquire the price, and dont be too particular.
With halting steps I paced the streets, and passed the sign of “The
Crossed Harpoons”—but it looked too expensive and jolly there. Further
on, from the bright red windows of the “Sword-Fish Inn,” there came
such fervent rays, that it seemed to have melted the packed snow and
ice from before the house, for everywhere else the congealed frost lay
ten inches thick in a hard, asphaltic pavement,—rather weary for me,
when I struck my foot against the flinty projections, because from
hard, remorseless service the soles of my boots were in a most
miserable plight. Too expensive and jolly, again thought I, pausing one
moment to watch the broad glare in the street, and hear the sounds of
the tinkling glasses within. But go on, Ishmael, said I at last; dont
you hear? get away from before the door; your patched boots are
stopping the way. So on I went. I now by instinct followed the streets
that took me waterward, for there, doubtless, were the cheapest, if not
the cheeriest inns.
Such dreary streets! blocks of blackness, not houses, on either hand,
and here and there a candle, like a candle moving about in a tomb. At
this hour of the night, of the last day of the week, that quarter of
the town proved all but deserted. But presently I came to a smoky light
proceeding from a low, wide building, the door of which stood
invitingly open. It had a careless look, as if it were meant for the
uses of the public; so, entering, the first thing I did was to stumble
over an ash-box in the porch. Ha! thought I, ha, as the flying
particles almost choked me, are these ashes from that destroyed city,
Gomorrah? But “The Crossed Harpoons,” and “The Sword-Fish?”—this, then
must needs be the sign of “The Trap.” However, I picked myself up and
hearing a loud voice within, pushed on and opened a second, interior
door.
It seemed the great Black Parliament sitting in Tophet. A hundred black
faces turned round in their rows to peer; and beyond, a black Angel of
Doom was beating a book in a pulpit. It was a negro church; and the
preachers text was about the blackness of darkness, and the weeping
and wailing and teeth-gnashing there. Ha, Ishmael, muttered I, backing
out, Wretched entertainment at the sign of The Trap!
Moving on, I at last came to a dim sort of light not far from the
docks, and heard a forlorn creaking in the air; and looking up, saw a
swinging sign over the door with a white painting upon it, faintly
representing a tall straight jet of misty spray, and these words
underneath—“The Spouter Inn:—Peter Coffin.”
Coffin?—Spouter?—Rather ominous in that particular connexion, thought
I. But it is a common name in Nantucket, they say, and I suppose this
Peter here is an emigrant from there. As the light looked so dim, and
the place, for the time, looked quiet enough, and the dilapidated
little wooden house itself looked as if it might have been carted here
from the ruins of some burnt district, and as the swinging sign had a
poverty-stricken sort of creak to it, I thought that here was the very
spot for cheap lodgings, and the best of pea coffee.

200
testdata/e2e/2.txt vendored Normal file
View File

@ -0,0 +1,200 @@
5
CHAPTER 1. Loomings.
Call me Ishmael. Some years ago—never mind how long precisely—having
little or no money in my purse, and nothing particular to interest me
on shore, I thought I would sail about a little and see the watery part
of the world. It is a way I have of driving off the spleen and
regulating the circulation. Whenever I find myself growing grim about
the mouth; whenever it is a damp, drizzly November in my soul; whenever
I find myself involuntarily pausing before coffin warehouses, and
bringing up the rear of every funeral I meet; and especially whenever
my hypos get such an upper hand of me, that it requires a strong moral
principle to prevent me from deliberately stepping into the street, and
methodically knocking peoples hats off—then, I account it high time to
get to sea as soon as I can. This is my substitute for pistol and ball.
With a philosophical flourish Cato throws himself upon his sword; I
quietly take to the ship. There is nothing surprising in this. If they
but knew it, almost all men in their degree, some time or other,
cherish very nearly the same feelings towards the ocean with me.
There now is your insular city of the Manhattoes, belted round by
wharves as Indian isles by coral reefs—commerce surrounds it with her
surf. Right and left, the streets take you waterward. Its extreme
downtown is the battery, where that noble mole is washed by waves, and
cooled by breezes, which a few hours previous were out of sight of
land. Look at the crowds of water-gazers there.
Circumambulate the city of a dreamy Sabbath afternoon. Go from Corlears
Hook to Coenties Slip, and from thence, by Whitehall, northward. What
do you see?—Posted like silent sentinels all around the town, stand
thousands upon thousands of mortal men fixed in ocean reveries. Some
leaning against the spiles; some seated upon the pier-heads; some
looking over the bulwarks of ships from China; some high aloft in the
rigging, as if striving to get a still better seaward peep. But these
are all landsmen; of week days pent up in lath and plaster—tied to
counters, nailed to benches, clinched to desks. How then is this? Are
the green fields gone? What do they here?
But look! here come more crowds, pacing straight for the water, and
seemingly bound for a dive. Strange! Nothing will content them but the
extremest limit of the land; loitering under the shady lee of yonder
warehouses will not suffice. No. They must get just as nigh the water
as they possibly can without falling in. And there they stand—miles of
them—leagues. Inlanders all, they come from lanes and alleys, streets
and avenues—north, east, south, and west. Yet here they all unite. Tell
me, does the magnetic virtue of the needles of the compasses of all
those ships attract them thither?
Once more. Say you are in the country; in some high land of lakes. Take
almost any path you please, and ten to one it carries you down in a
dale, and leaves you there by a pool in the stream. There is magic in
it. Let the most absent-minded of men be plunged in his deepest
reveries—stand that man on his legs, set his feet a-going, and he will
infallibly lead you to water, if water there be in all that region.
Should you ever be athirst in the great American desert, try this
experiment, if your caravan happen to be supplied with a metaphysical
professor. Yes, as every one knows, meditation and water are wedded for
ever.
But here is an artist. He desires to paint you the dreamiest, shadiest,
quietest, most enchanting bit of romantic landscape in all the valley
of the Saco. What is the chief element he employs? There stand his
trees, each with a hollow trunk, as if a hermit and a crucifix were
within; and here sleeps his meadow, and there sleep his cattle; and up
from yonder cottage goes a sleepy smoke. Deep into distant woodlands
winds a mazy way, reaching to overlapping spurs of mountains bathed in
their hill-side blue. But though the picture lies thus tranced, and
though this pine-tree shakes down its sighs like leaves upon this
shepherds head, yet all were vain, unless the shepherds eye were
fixed upon the magic stream before him. Go visit the Prairies in June,
when for scores on scores of miles you wade knee-deep among
Tiger-lilies—what is the one charm wanting?—Water—there is not a drop
of water there! Were Niagara but a cataract of sand, would you travel
your thousand miles to see it? Why did the poor poet of Tennessee, upon
suddenly receiving two handfuls of silver, deliberate whether to buy
him a coat, which he sadly needed, or invest his money in a pedestrian
trip to Rockaway Beach? Why is almost every robust healthy boy with a
robust healthy soul in him, at some time or other crazy to go to sea?
Why upon your first voyage as a passenger, did you yourself feel such a
mystical vibration, when first told that you and your ship were now out
of sight of land? Why did the old Persians hold the sea holy? Why did
the Greeks give it a separate deity, and own brother of Jove? Surely
all this is not without meaning. And still deeper the meaning of that
story of Narcissus, who because he could not grasp the tormenting, mild
image he saw in the fountain, plunged into it and was drowned. But that
same image, we ourselves see in all rivers and oceans. It is the image
of the ungraspable phantom of life; and this is the key to it all.
Now, when I say that I am in the habit of going to sea whenever I begin
to grow hazy about the eyes, and begin to be over conscious of my
lungs, I do not mean to have it inferred that I ever go to sea as a
passenger. For to go as a passenger you must needs have a purse, and a
purse is but a rag unless you have something in it. Besides, passengers
get sea-sick—grow quarrelsome—dont sleep of nights—do not enjoy
themselves much, as a general thing;—no, I never go as a passenger;
nor, though I am something of a salt, do I ever go to sea as a
Commodore, or a Captain, or a Cook. I abandon the glory and distinction
of such offices to those who like them. For my part, I abominate all
honorable respectable toils, trials, and tribulations of every kind
whatsoever. It is quite as much as I can do to take care of myself,
without taking care of ships, barques, brigs, schooners, and what not.
And as for going as cook,—though I confess there is considerable glory
in that, a cook being a sort of officer on ship-board—yet, somehow, I
never fancied broiling fowls;—though once broiled, judiciously
buttered, and judgmatically salted and peppered, there is no one who
will speak more respectfully, not to say reverentially, of a broiled
fowl than I will. It is out of the idolatrous dotings of the old
Egyptians upon broiled ibis and roasted river horse, that you see the
mummies of those creatures in their huge bake-houses the pyramids.
No, when I go to sea, I go as a simple sailor, right before the mast,
plumb down into the forecastle, aloft there to the royal mast-head.
True, they rather order me about some, and make me jump from spar to
spar, like a grasshopper in a May meadow. And at first, this sort of
thing is unpleasant enough. It touches ones sense of honor,
particularly if you come of an old established family in the land, the
Van Rensselaers, or Randolphs, or Hardicanutes. And more than all, if
just previous to putting your hand into the tar-pot, you have been
lording it as a country schoolmaster, making the tallest boys stand in
awe of you. The transition is a keen one, I assure you, from a
schoolmaster to a sailor, and requires a strong decoction of Seneca and
the Stoics to enable you to grin and bear it. But even this wears off
in time.
What of it, if some old hunks of a sea-captain orders me to get a broom
and sweep down the decks? What does that indignity amount to, weighed,
I mean, in the scales of the New Testament? Do you think the archangel
Gabriel thinks anything the less of me, because I promptly and
respectfully obey that old hunks in that particular instance? Who aint
a slave? Tell me that. Well, then, however the old sea-captains may
order me about—however they may thump and punch me about, I have the
satisfaction of knowing that it is all right; that everybody else is
one way or other served in much the same way—either in a physical or
metaphysical point of view, that is; and so the universal thump is
passed round, and all hands should rub each others shoulder-blades,
and be content.
Again, I always go to sea as a sailor, because they make a point of
paying me for my trouble, whereas they never pay passengers a single
penny that I ever heard of. On the contrary, passengers themselves must
pay. And there is all the difference in the world between paying and
being paid. The act of paying is perhaps the most uncomfortable
infliction that the two orchard thieves entailed upon us. But _being
paid_,—what will compare with it? The urbane activity with which a man
receives money is really marvellous, considering that we so earnestly
believe money to be the root of all earthly ills, and that on no
account can a monied man enter heaven. Ah! how cheerfully we consign
ourselves to perdition!
Finally, I always go to sea as a sailor, because of the wholesome
exercise and pure air of the fore-castle deck. For as in this world,
head winds are far more prevalent than winds from astern (that is, if
you never violate the Pythagorean maxim), so for the most part the
Commodore on the quarter-deck gets his atmosphere at second hand from
the sailors on the forecastle. He thinks he breathes it first; but not
so. In much the same way do the commonalty lead their leaders in many
other things, at the same time that the leaders little suspect it. But
wherefore it was that after having repeatedly smelt the sea as a
merchant sailor, I should now take it into my head to go on a whaling
voyage; this the invisible police officer of the Fates, who has the
constant surveillance of me, and secretly dogs me, and influences me in
some unaccountable way—he can better answer than any one else. And,
doubtless, my going on this whaling voyage, formed part of the grand
programme of Providence that was drawn up a long time ago. It came in
as a sort of brief interlude and solo between more extensive
performances. I take it that this part of the bill must have run
something like this:
“_Grand Contested Election for the Presidency of the United States._
“WHALING VOYAGE BY ONE ISHMAEL. “BLOODY BATTLE IN AFFGHANISTAN.”
Though I cannot tell why it was exactly that those stage managers, the
Fates, put me down for this shabby part of a whaling voyage, when
others were set down for magnificent parts in high tragedies, and short
and easy parts in genteel comedies, and jolly parts in farces—though I
cannot tell why this was exactly; yet, now that I recall all the
circumstances, I think I can see a little into the springs and motives
which being cunningly presented to me under various disguises, induced
me to set about performing the part I did, besides cajoling me into the
delusion that it was a choice resulting from my own unbiased freewill
and discriminating judgment.
Chief among these motives was the overwhelming idea of the great whale
himself. Such a portentous and mysterious monster roused all my
curiosity. Then the wild and distant seas where he rolled his island
bulk; the undeliverable, nameless perils of the whale; these, with all
the attending marvels of a thousand Patagonian sights and sounds,
helped to sway me to my wish. With other men, perhaps, such things
would not have been inducements; but as for me, I am tormented with an
everlasting itch for things remote. I love to sail forbidden seas, and
land on barbarous coasts. Not ignoring what is good, I am quick to
perceive a horror, and could still be social with it—would they let
me—since it is but well to be on friendly terms with all the inmates of
the place one lodges in.
By reason of these things, then, the whaling voyage was welcome; the
great flood-gates of the wonder-world swung open, and in the wild
conceits that swayed me to my purpose, two and two there floated into
my inmost soul, endless processions of the whale, and, mid most of them
all, one grand hooded phantom, like a snow hill in the air.

58
testdata/e2e/21.txt vendored Normal file
View File

@ -0,0 +1,58 @@
3
CHAPTER 3. The Spouter-Inn.
Entering that gable-ended Spouter-Inn, you found yourself in a wide,
low, straggling entry with old-fashioned wainscots, reminding one of
the bulwarks of some condemned old craft. On one side hung a very large
oilpainting so thoroughly besmoked, and every way defaced, that in the
unequal crosslights by which you viewed it, it was only by diligent
study and a series of systematic visits to it, and careful inquiry of
the neighbors, that you could any way arrive at an understanding of its
purpose. Such unaccountable masses of shades and shadows, that at first
you almost thought some ambitious young artist, in the time of the New
England hags, had endeavored to delineate chaos bewitched. But by dint
of much and earnest contemplation, and oft repeated ponderings, and
especially by throwing open the little window towards the back of the
entry, you at last come to the conclusion that such an idea, however
wild, might not be altogether unwarranted.
But what most puzzled and confounded you was a long, limber,
portentous, black mass of something hovering in the centre of the
picture over three blue, dim, perpendicular lines floating in a
nameless yeast. A boggy, soggy, squitchy picture truly, enough to drive
a nervous man distracted. Yet was there a sort of indefinite,
half-attained, unimaginable sublimity about it that fairly froze you to
it, till you involuntarily took an oath with yourself to find out what
that marvellous painting meant. Ever and anon a bright, but, alas,
deceptive idea would dart you through.—Its the Black Sea in a midnight
gale.—Its the unnatural combat of the four primal elements.—Its a
blasted heath.—Its a Hyperborean winter scene.—Its the breaking-up of
the icebound stream of Time. But at last all these fancies yielded to
that one portentous something in the pictures midst. _That_ once found
out, and all the rest were plain. But stop; does it not bear a faint
resemblance to a gigantic fish? even the great leviathan himself?
In fact, the artists design seemed this: a final theory of my own,
partly based upon the aggregated opinions of many aged persons with
whom I conversed upon the subject. The picture represents a Cape-Horner
in a great hurricane; the half-foundered ship weltering there with its
three dismantled masts alone visible; and an exasperated whale,
purposing to spring clean over the craft, is in the enormous act of
impaling himself upon the three mast-heads.
The opposite wall of this entry was hung all over with a heathenish
array of monstrous clubs and spears. Some were thickly set with
glittering teeth resembling ivory saws; others were tufted with knots
of human hair; and one was sickle-shaped, with a vast handle sweeping
round like the segment made in the new-mown grass by a long-armed
mower. You shuddered as you gazed, and wondered what monstrous cannibal
and savage could ever have gone a death-harvesting with such a hacking,
horrifying implement. Mixed with these were rusty old whaling lances
and harpoons all broken and deformed. Some were storied weapons. With
this once long lance, now wildly elbowed, fifty years ago did Nathan
Swain kill fifteen whales between a sunrise and a sunset. And that
harpoon—so like a corkscrew now—was flung in Javan seas, and run away
with by a whale, years afterwards slain off the Cape of Blanco. The
original iron entered nigh the tail, and, like a restless needle
sojourning in the body of a man, travelled full forty feet, and at last
was found imbedded in the hump.

179
testdata/e2e/22.txt vendored Normal file
View File

@ -0,0 +1,179 @@
4
CHAPTER 1. Loomings.
Call me Ishmael. Some years ago—never mind how long precisely—having
little or no money in my purse, and nothing particular to interest me
on shore, I thought I would sail about a little and see the watery part
of the world. It is a way I have of driving off the spleen and
regulating the circulation. Whenever I find myself growing grim about
the mouth; whenever it is a damp, drizzly November in my soul; whenever
I find myself involuntarily pausing before coffin warehouses, and
bringing up the rear of every funeral I meet; and especially whenever
my hypos get such an upper hand of me, that it requires a strong moral
principle to prevent me from deliberately stepping into the street, and
methodically knocking peoples hats off—then, I account it high time to
get to sea as soon as I can. This is my substitute for pistol and ball.
With a philosophical flourish Cato throws himself upon his sword; I
quietly take to the ship. There is nothing surprising in this. If they
but knew it, almost all men in their degree, some time or other,
cherish very nearly the same feelings towards the ocean with me.
There now is your insular city of the Manhattoes, belted round by
wharves as Indian isles by coral reefs—commerce surrounds it with her
surf. Right and left, the streets take you waterward. Its extreme
downtown is the battery, where that noble mole is washed by waves, and
cooled by breezes, which a few hours previous were out of sight of
land. Look at the crowds of water-gazers there.
Circumambulate the city of a dreamy Sabbath afternoon. Go from Corlears
Hook to Coenties Slip, and from thence, by Whitehall, northward. What
do you see?—Posted like silent sentinels all around the town, stand
thousands upon thousands of mortal men fixed in ocean reveries. Some
leaning against the spiles; some seated upon the pier-heads; some
looking over the bulwarks of ships from China; some high aloft in the
rigging, as if striving to get a still better seaward peep. But these
are all landsmen; of week days pent up in lath and plaster—tied to
counters, nailed to benches, clinched to desks. How then is this? Are
the green fields gone? What do they here?
But look! here come more crowds, pacing straight for the water, and
seemingly bound for a dive. Strange! Nothing will content them but the
extremest limit of the land; loitering under the shady lee of yonder
warehouses will not suffice. No. They must get just as nigh the water
as they possibly can without falling in. And there they stand—miles of
them—leagues. Inlanders all, they come from lanes and alleys, streets
and avenues—north, east, south, and west. Yet here they all unite. Tell
me, does the magnetic virtue of the needles of the compasses of all
those ships attract them thither?
Once more. Say you are in the country; in some high land of lakes. Take
almost any path you please, and ten to one it carries you down in a
dale, and leaves you there by a pool in the stream. There is magic in
it. Let the most absent-minded of men be plunged in his deepest
reveries—stand that man on his legs, set his feet a-going, and he will
infallibly lead you to water, if water there be in all that region.
Should you ever be athirst in the great American desert, try this
experiment, if your caravan happen to be supplied with a metaphysical
professor. Yes, as every one knows, meditation and water are wedded for
ever.
But here is an artist. He desires to paint you the dreamiest, shadiest,
quietest, most enchanting bit of romantic landscape in all the valley
of the Saco. What is the chief element he employs? There stand his
trees, each with a hollow trunk, as if a hermit and a crucifix were
within; and here sleeps his meadow, and there sleep his cattle; and up
from yonder cottage goes a sleepy smoke. Deep into distant woodlands
winds a mazy way, reaching to overlapping spurs of mountains bathed in
their hill-side blue. But though the picture lies thus tranced, and
though this pine-tree shakes down its sighs like leaves upon this
shepherds head, yet all were vain, unless the shepherds eye were
fixed upon the magic stream before him. Go visit the Prairies in June,
when for scores on scores of miles you wade knee-deep among
Tiger-lilies—what is the one charm wanting?—Water—there is not a drop
of water there! Were Niagara but a cataract of sand, would you travel
your thousand miles to see it? Why did the poor poet of Tennessee, upon
suddenly receiving two handfuls of silver, deliberate whether to buy
him a coat, which he sadly needed, or invest his money in a pedestrian
trip to Rockaway Beach? Why is almost every robust healthy boy with a
robust healthy soul in him, at some time or other crazy to go to sea?
Why upon your first voyage as a passenger, did you yourself feel such a
mystical vibration, when first told that you and your ship were now out
of sight of land? Why did the old Persians hold the sea holy? Why did
the Greeks give it a separate deity, and own brother of Jove? Surely
all this is not without meaning. And still deeper the meaning of that
story of Narcissus, who because he could not grasp the tormenting, mild
image he saw in the fountain, plunged into it and was drowned. But that
same image, we ourselves see in all rivers and oceans. It is the image
of the ungraspable phantom of life; and this is the key to it all.
No, when I go to sea, I go as a simple sailor, right before the mast,
plumb down into the forecastle, aloft there to the royal mast-head.
True, they rather order me about some, and make me jump from spar to
spar, like a grasshopper in a May meadow. And at first, this sort of
thing is unpleasant enough. It touches ones sense of honor,
particularly if you come of an old established family in the land, the
Van Rensselaers, or Randolphs, or Hardicanutes. And more than all, if
just previous to putting your hand into the tar-pot, you have been
lording it as a country schoolmaster, making the tallest boys stand in
awe of you. The transition is a keen one, I assure you, from a
schoolmaster to a sailor, and requires a strong decoction of Seneca and
the Stoics to enable you to grin and bear it. But even this wears off
in time.
What of it, if some old hunks of a sea-captain orders me to get a broom
and sweep down the decks? What does that indignity amount to, weighed,
I mean, in the scales of the New Testament? Do you think the archangel
Gabriel thinks anything the less of me, because I promptly and
respectfully obey that old hunks in that particular instance? Who aint
a slave? Tell me that. Well, then, however the old sea-captains may
order me about—however they may thump and punch me about, I have the
satisfaction of knowing that it is all right; that everybody else is
one way or other served in much the same way—either in a physical or
metaphysical point of view, that is; and so the universal thump is
passed round, and all hands should rub each others shoulder-blades,
and be content.
Again, I always go to sea as a sailor, because they make a point of
paying me for my trouble, whereas they never pay passengers a single
penny that I ever heard of. On the contrary, passengers themselves must
pay. And there is all the difference in the world between paying and
being paid. The act of paying is perhaps the most uncomfortable
infliction that the two orchard thieves entailed upon us. But _being
paid_,—what will compare with it? The urbane activity with which a man
receives money is really marvellous, considering that we so earnestly
believe money to be the root of all earthly ills, and that on no
account can a monied man enter heaven. Ah! how cheerfully we consign
ourselves to perdition!
Finally, I always go to sea as a sailor, because of the wholesome
exercise and pure air of the fore-castle deck. For as in this world,
head winds are far more prevalent than winds from astern (that is, if
you never violate the Pythagorean maxim), so for the most part the
Commodore on the quarter-deck gets his atmosphere at second hand from
the sailors on the forecastle. He thinks he breathes it first; but not
so. In much the same way do the commonalty lead their leaders in many
other things, at the same time that the leaders little suspect it. But
wherefore it was that after having repeatedly smelt the sea as a
merchant sailor, I should now take it into my head to go on a whaling
voyage; this the invisible police officer of the Fates, who has the
constant surveillance of me, and secretly dogs me, and influences me in
some unaccountable way—he can better answer than any one else. And,
doubtless, my going on this whaling voyage, formed part of the grand
programme of Providence that was drawn up a long time ago. It came in
as a sort of brief interlude and solo between more extensive
performances. I take it that this part of the bill must have run
something like this:
“_Grand Contested Election for the Presidency of the United States._
“WHALING VOYAGE BY ONE ISHMAEL. “BLOODY BATTLE IN AFFGHANISTAN.”
Though I cannot tell why it was exactly that those stage managers, the
Fates, put me down for this shabby part of a whaling voyage, when
others were set down for magnificent parts in high tragedies, and short
and easy parts in genteel comedies, and jolly parts in farces—though I
cannot tell why this was exactly; yet, now that I recall all the
circumstances, I think I can see a little into the springs and motives
which being cunningly presented to me under various disguises, induced
me to set about performing the part I did, besides cajoling me into the
delusion that it was a choice resulting from my own unbiased freewill
and discriminating judgment.
Chief among these motives was the overwhelming idea of the great whale
himself. Such a portentous and mysterious monster roused all my
curiosity. Then the wild and distant seas where he rolled his island
bulk; the undeliverable, nameless perils of the whale; these, with all
the attending marvels of a thousand Patagonian sights and sounds,
helped to sway me to my wish. With other men, perhaps, such things
would not have been inducements; but as for me, I am tormented with an
everlasting itch for things remote. I love to sail forbidden seas, and
land on barbarous coasts. Not ignoring what is good, I am quick to
perceive a horror, and could still be social with it—would they let
me—since it is but well to be on friendly terms with all the inmates of
the place one lodges in.
By reason of these things, then, the whaling voyage was welcome; the
great flood-gates of the wonder-world swung open, and in the wild
conceits that swayed me to my purpose, two and two there floated into
my inmost soul, endless processions of the whale, and, mid most of them
all, one grand hooded phantom, like a snow hill in the air.

128
testdata/e2e/25.txt vendored Normal file
View File

@ -0,0 +1,128 @@
2
CHAPTER 1. Loomings.
Call me Ishmael. Some years ago—never mind how long precisely—having
little or no money in my purse, and nothing particular to interest me
on shore, I thought I would sail about a little and see the watery part
of the world. It is a way I have of driving off the spleen and
regulating the circulation. Whenever I find myself growing grim about
the mouth; whenever it is a damp, drizzly November in my soul; whenever
I find myself involuntarily pausing before coffin warehouses, and
bringing up the rear of every funeral I meet; and especially whenever
my hypos get such an upper hand of me, that it requires a strong moral
principle to prevent me from deliberately stepping into the street, and
methodically knocking peoples hats off—then, I account it high time to
get to sea as soon as I can. This is my substitute for pistol and ball.
With a philosophical flourish Cato throws himself upon his sword; I
quietly take to the ship. There is nothing surprising in this. If they
but knew it, almost all men in their degree, some time or other,
cherish very nearly the same feelings towards the ocean with me.
There now is your insular city of the Manhattoes, belted round by
wharves as Indian isles by coral reefs—commerce surrounds it with her
surf. Right and left, the streets take you waterward. Its extreme
downtown is the battery, where that noble mole is washed by waves, and
cooled by breezes, which a few hours previous were out of sight of
land. Look at the crowds of water-gazers there.
But look! here come more crowds, pacing straight for the water, and
seemingly bound for a dive. Strange! Nothing will content them but the
extremest limit of the land; loitering under the shady lee of yonder
warehouses will not suffice. No. They must get just as nigh the water
as they possibly can without falling in. And there they stand—miles of
them—leagues. Inlanders all, they come from lanes and alleys, streets
and avenues—north, east, south, and west. Yet here they all unite. Tell
me, does the magnetic virtue of the needles of the compasses of all
those ships attract them thither?
No, when I go to sea, I go as a simple sailor, right before the mast,
plumb down into the forecastle, aloft there to the royal mast-head.
True, they rather order me about some, and make me jump from spar to
spar, like a grasshopper in a May meadow. And at first, this sort of
thing is unpleasant enough. It touches ones sense of honor,
particularly if you come of an old established family in the land, the
Van Rensselaers, or Randolphs, or Hardicanutes. And more than all, if
just previous to putting your hand into the tar-pot, you have been
lording it as a country schoolmaster, making the tallest boys stand in
awe of you. The transition is a keen one, I assure you, from a
schoolmaster to a sailor, and requires a strong decoction of Seneca and
the Stoics to enable you to grin and bear it. But even this wears off
in time.
What of it, if some old hunks of a sea-captain orders me to get a broom
and sweep down the decks? What does that indignity amount to, weighed,
I mean, in the scales of the New Testament? Do you think the archangel
Gabriel thinks anything the less of me, because I promptly and
respectfully obey that old hunks in that particular instance? Who aint
a slave? Tell me that. Well, then, however the old sea-captains may
order me about—however they may thump and punch me about, I have the
satisfaction of knowing that it is all right; that everybody else is
one way or other served in much the same way—either in a physical or
metaphysical point of view, that is; and so the universal thump is
passed round, and all hands should rub each others shoulder-blades,
and be content.
Again, I always go to sea as a sailor, because they make a point of
paying me for my trouble, whereas they never pay passengers a single
penny that I ever heard of. On the contrary, passengers themselves must
pay. And there is all the difference in the world between paying and
being paid. The act of paying is perhaps the most uncomfortable
infliction that the two orchard thieves entailed upon us. But _being
paid_,—what will compare with it? The urbane activity with which a man
receives money is really marvellous, considering that we so earnestly
believe money to be the root of all earthly ills, and that on no
account can a monied man enter heaven. Ah! how cheerfully we consign
ourselves to perdition!
Finally, I always go to sea as a sailor, because of the wholesome
exercise and pure air of the fore-castle deck. For as in this world,
head winds are far more prevalent than winds from astern (that is, if
you never violate the Pythagorean maxim), so for the most part the
Commodore on the quarter-deck gets his atmosphere at second hand from
the sailors on the forecastle. He thinks he breathes it first; but not
so. In much the same way do the commonalty lead their leaders in many
other things, at the same time that the leaders little suspect it. But
wherefore it was that after having repeatedly smelt the sea as a
merchant sailor, I should now take it into my head to go on a whaling
voyage; this the invisible police officer of the Fates, who has the
constant surveillance of me, and secretly dogs me, and influences me in
some unaccountable way—he can better answer than any one else. And,
doubtless, my going on this whaling voyage, formed part of the grand
programme of Providence that was drawn up a long time ago. It came in
as a sort of brief interlude and solo between more extensive
performances. I take it that this part of the bill must have run
something like this:
“_Grand Contested Election for the Presidency of the United States._
“WHALING VOYAGE BY ONE ISHMAEL. “BLOODY BATTLE IN AFFGHANISTAN.”
Though I cannot tell why it was exactly that those stage managers, the
Fates, put me down for this shabby part of a whaling voyage, when
others were set down for magnificent parts in high tragedies, and short
and easy parts in genteel comedies, and jolly parts in farces—though I
cannot tell why this was exactly; yet, now that I recall all the
circumstances, I think I can see a little into the springs and motives
which being cunningly presented to me under various disguises, induced
me to set about performing the part I did, besides cajoling me into the
delusion that it was a choice resulting from my own unbiased freewill
and discriminating judgment.
Chief among these motives was the overwhelming idea of the great whale
himself. Such a portentous and mysterious monster roused all my
curiosity. Then the wild and distant seas where he rolled his island
bulk; the undeliverable, nameless perils of the whale; these, with all
the attending marvels of a thousand Patagonian sights and sounds,
helped to sway me to my wish. With other men, perhaps, such things
would not have been inducements; but as for me, I am tormented with an
everlasting itch for things remote. I love to sail forbidden seas, and
land on barbarous coasts. Not ignoring what is good, I am quick to
perceive a horror, and could still be social with it—would they let
me—since it is but well to be on friendly terms with all the inmates of
the place one lodges in.
By reason of these things, then, the whaling voyage was welcome; the
great flood-gates of the wonder-world swung open, and in the wild
conceits that swayed me to my purpose, two and two there floated into
my inmost soul, endless processions of the whale, and, mid most of them
all, one grand hooded phantom, like a snow hill in the air.

51
testdata/e2e/7.txt vendored Normal file
View File

@ -0,0 +1,51 @@
0
CHAPTER 1. Loomings.
Call me Ishmael. Some years ago—never mind how long precisely—having
little or no money in my purse, and nothing particular to interest me
on shore, I thought I would sail about a little and see the watery part
of the world. It is a way I have of driving off the spleen and
regulating the circulation. Whenever I find myself growing grim about
the mouth; whenever it is a damp, drizzly November in my soul; whenever
I find myself involuntarily pausing before coffin warehouses, and
bringing up the rear of every funeral I meet; and especially whenever
my hypos get such an upper hand of me, that it requires a strong moral
principle to prevent me from deliberately stepping into the street, and
methodically knocking peoples hats off—then, I account it high time to
get to sea as soon as I can. This is my substitute for pistol and ball.
With a philosophical flourish Cato throws himself upon his sword; I
quietly take to the ship. There is nothing surprising in this. If they
but knew it, almost all men in their degree, some time or other,
cherish very nearly the same feelings towards the ocean with me.
There now is your insular city of the Manhattoes, belted round by
wharves as Indian isles by coral reefs—commerce surrounds it with her
surf. Right and left, the streets take you waterward. Its extreme
downtown is the battery, where that noble mole is washed by waves, and
cooled by breezes, which a few hours previous were out of sight of
land. Look at the crowds of water-gazers there.
What of it, if some old hunks of a sea-captain orders me to get a broom
and sweep down the decks? What does that indignity amount to, weighed,
I mean, in the scales of the New Testament? Do you think the archangel
Gabriel thinks anything the less of me, because I promptly and
respectfully obey that old hunks in that particular instance? Who aint
a slave? Tell me that. Well, then, however the old sea-captains may
order me about—however they may thump and punch me about, I have the
satisfaction of knowing that it is all right; that everybody else is
one way or other served in much the same way—either in a physical or
metaphysical point of view, that is; and so the universal thump is
passed round, and all hands should rub each others shoulder-blades,
and be content.
Again, I always go to sea as a sailor, because they make a point of
paying me for my trouble, whereas they never pay passengers a single
penny that I ever heard of. On the contrary, passengers themselves must
pay. And there is all the difference in the world between paying and
being paid. The act of paying is perhaps the most uncomfortable
infliction that the two orchard thieves entailed upon us. But _being
paid_,—what will compare with it? The urbane activity with which a man
receives money is really marvellous, considering that we so earnestly
believe money to be the root of all earthly ills, and that on no
account can a monied man enter heaven. Ah! how cheerfully we consign
ourselves to perdition!