Every so often there's talk about converting the various EPU works for eBook formats.

My fiancée got a Kindle for Christmas, and is interested in re-reading and catching up on UF works (she read a bunch of them a number of years ago).

The Kindle is terrible at displaying 80-column ASCII text, though, and the various automated conversion tools mangle the text to a greater or lesser extent (particularly song lyrics and credits).

So has anyone made any progress in converting the stories (particularly Symphony of the Sword and related works)?

LAST EDITED ON Jan-08-12 AT 06:00 PM (EST)
 
It's not exactly a conversion, but there is a way to display Eyrie works on a Kindle without too much hassle.

What you do is this: Go into the Options menu, and click Screen Rotation. Change it to one of the sideways options (I opt for the one with the buttons to the right, but I'm right handed). Open a text file (I used the first part of Symphony 1), open the Options menu again, and click Change Font Size (you can also change screen rotation in this menu, if you haven't already).

Now, you have a couple options. You can go to the third font size on the left and the Condensed typeface, the second size with the Regular typeface, or Sans-Serif with size 3. I have yet to read through a full Eyrie piece with any of these settings, but they seem to display correctly at the moment. You may encounter a broken line or two.

As for modifying the files so they can be read 'normally', the only thing you have to do is remove the line breaks; your Kindle will take care of the rest. Slightly tedious, but not impossible.

LAST EDITED ON Jan-09-12 AT 09:46 PM (EST)
 
>As for modifying the files so they can be read 'normally', the only
>thing you have to do is remove the line breaks; your Kindle will take
>care of the rest. Slightly tedious, but not impossible.

Here's the vim commands I use, which work pretty well for "de-breaking" text files in general (the struck-through "b"s represent spaces):

:1,$s/\t/bbbbbbbb/g
:g/^[^b]/-1j
:1,$s/^bbbbb*/bbbb/

Translated:

1. Convert all tabs into eight spaces.
   
2. Search for lines that don't start with a space, and join each one to the line above itself.
   
3. Reduce any group of five or more leading spaces to four spaces.
   

This does a good job on the majority of the non-Core stories; the parallel-cast-and-song credits on some of the stories don't fare as well.

(The Core stories have idiosyncratic sections that would require a good deal of hand-editing to make right, but most of the body text comes out okay.)

>This does a good job on the majority of the non-Core stories; the
>parallel-cast-and-song credits on some of the stories don't fare as
>well.

Doesn't it also tend to mangle (i.e., wrap) the song lyrics in the Symphony?

Wounded Rose has been my test input file so far.

>Doesn't it also tend to mangle (i.e., wrap) the song lyrics in the
>Symphony?
>
>Wounded Rose has been my test input file so far.

Well, yeah--that's why I said it does a "good job," not a "perfect job." :) I haven't yet figured out a practical heuristic to accurately identify song lyrics within the text.

The workaround is to edit the file before you process it, and insert one or more spaces at the beginning of every line that's part of a song lyric. Then, when you run the process, it will see the leading spaces and won't join the lines together.

Shorter lines. Song lyric lines typically don't go across the full 70 columns, and they don't start with an indent. If the line is under 55 characters, it's probably a lyric.

But my code just guesses by checking for zero indent and eliminating other possibilities, like dual column credits or inline emails.

>This does a good job on the majority of the non-Core stories; the
>parallel-cast-and-song credits on some of the stories don't fare as
>well.
>
>(The Core stories have idiosyncratic sections that would require a
>good deal of hand-editing to make right, but most of the body text
>comes out okay.)

I converted a bunch of SoS stories to latex a few years back; my code to do so kept getting more and more complex (I'm not sure I ever found a good solution for the cast and song credits).

The problem, of course, is that the stories were written for people to read, not for programs to process. Clearly the solution is strong AI (as long as we can keep them focused on the job, and not distracted by reading the stories instead of processing them).

dstar

Now, if I had any idea how to actually -use- that, I'd be good to go.

LAST EDITED ON Jan-09-12 AT 02:01 PM (EST)
 
>Every so often there's talk about converting the various EPU works for
>eBook formats.
>
>My fiancée got a Kindle for Christmas, and is interested in re-reading
>and catching up on UF works (she read a bunch of them a number of
>years ago).
>
>The Kindle is terrible at displaying 80-column ASCII text, though, and
>the various automated conversion tools mangle the text to a greater or
>lesser extent (particularly song lyrics and credits).
>
>So has anyone made any progress in converting the stories
>(particularly Symphony of the Sword and related works)?

AFAIK, there's a rule about formats, namely "They don't give a crap about keepng up with the newest ereader format, and 80 column text is good enough"

Lemme see what I can BASH (well, python, really) together.

ETA: Alright, so it looks like the big hurdle is going to be actually formatting the files into html formatting, which I don't see an easy way to automate (well, I could script it fairly easily, but then I'd have to go and clean up everywhere that it didn't match up perfectly).

Then again, I've got a bunch of free time, and wanted a ruby learning project, so...

--
-Pasha
What was that feeling again?
Oh yes.
-Rage-

>AFAIK, there's a rule about formats, namely "They don't give a crap
>about keepng up with the newest ereader format, and 80 column text is
>good enough"

Oh, yeah, I was querying the community, not the Management -- Gryphon's said in the past he's not interested in doing the conversion.

But several people here have expressed interest, previously (most recently in the "Strange German Words from DJ" thread in the NXE forum, a year ago).

I did the Newton conversions of the early NXE stuff, and let me tell you it's a pain in the behind. It took a great deal of semi-manual editing to convert the text files into something useful for the converter. Truss did conversions of the early Symphony stuff. He used some shell scripts to automate much of what I had done semi-automatically in Emacs, but the results still required a fair bit of tweaking by hand. So, really, it's not likely to happen on any kind of mass scale.

I have a Kindle DX and a DXG. What I do for preformatted text is a simple copy-paste into LibreOffice followed by an export to PDF. The key is setting the page size correctly and the margins to zero. From what I've found from a quick Google search, a good page size for Kindle 3 (6" display) is 5.49x7.33 inches. Set the page size and margins, then copy-paste in the text. Select all and set the font to a monospaced font. I use Liberation Sans Mono Bold; YMMV. Set the font size just small enough to eliminate line wrap. Save it off so you have a copy, then export to PDF. Copy that PDF over to your Kindle and enjoy.

A trick for the smaller Kindles is to set Landscape mode for the base document. The greater width means you can use a larger font size without wrapping text.

Has anyone tried converting to PDF? I've found that most of the PDF's I've loaded into iBooks on the iPad read fine, even those meant to be printed at 8.5x11.

... isn't that what I described doing? :)

I have all of SOS in a Kindle-friendly MOBI format. Also both parts of Aegis Florea, all of Rogue Squadron, all of Twilight, the 4 Mini-Story Omnibuses (Omnibii?), and the now-removed Day of Infamy.

In most cases I took the original text files to Word, formatted them so they lay out nicely, and then ran them through Calibre to convert to MOBI.

I do admit to making a few format changes in line with my own reading preferences, such as (among others) text bracketed by non-alpha characters used to provide some sort of emphasis to the text became italicized text without the non-alpha characters.

Jeanne

	Jeanne Hedge http://www.jhedge.com
"Believe me, if I have to go the rest of my life without companionship, knowing myself won't be a problem." -- Gabrielle of Potadeia

Might I prevail upon you for a zip file of these, particularly SOS? Pretty please with sugar and chocolate shavings on top?

>Might I prevail upon you for a zip file of these, particularly SOS?
>Pretty please with sugar and chocolate shavings on top?

Anyone else want supplied as I posted a couple messages up? I've just finished emailing Symphony 1 (in small batches) to a couple folks.

Jeanne

	Jeanne Hedge http://www.jhedge.com
"Believe me, if I have to go the rest of my life without companionship, knowing myself won't be a problem." -- Gabrielle of Potadeia

>Anyone else want supplied as I posted a couple messages up? I've just
>finished emailing Symphony 1 (in small batches) to a couple folks.

When you have the time, Jeanne, I wouldn't mind.

-- Bob
-------------------
My race is pacifist and does not believe in war. We kill only out of personal spite.

I would love a copy too, Jeanne, whenever you have the time. Don't rush on my account.

Thanks,
Michael

Michael C. Fortner
"Rule 37: There is no such thing as "overkill".
There is only "open fire" and "I need to reload".

>Anyone else want supplied as I posted a couple messages up? I've just
>finished emailing Symphony 1 (in small batches) to a couple folks.

Me five! Or maybe more than that :-)

When it's convenient of course, thanks :)

Senji.

Jeanne,
I'd be interested if/when convenient as well.

Brian

LAST EDITED ON Feb-17-12 AT 11:44 PM (EST)
 
http://xkcd.com/208/

Did someone mention regular expressions?

Okay, I saw this thread, and I thought that I'd take a crack at an ebook conversion. Since I got a Nook for Christmas, I'm a little familiar with the EPUB format. And since that is basically just XHTML+CSS (wait, when did EPUB 3 come out... okay, EPUB 2 then), an epub version seems like something that could be put together fairly easily.

So I wrote a script to convert the text files to HTML, using a mess of regular expressions. (Or a quagmire of regular expressions? What's the collective noun for regexes?) It's doing a pretty good job of guessing what things are, but it's not perfect yet. But most of the important stuff is working, like 2-column credits and centered headers. And most of the heavy lifting of the formatting is in the CSS, and that isn't perfect yet.

But Fulcrum of Fate is pretty readable on my Nook right now, despite not having a working table of contents. I'll let you know when I'm a little farther along.

>What's the collective noun for regexes?

A "my-God-that-is-uglier-than-TECO".

Hey, hey, what's wrong with TECO? I still remember my first program:
ER%#@!$(DO)EV'S~_+F:!&#MD|><:-)

Okay, I got sidetracked by starting to write my own fanfic, but I decided to finish up my program to a somewhat usable state. There still needs to be some work (especially in the older stories), and some things will require human intervention (the naval prize table in Gunboat Diplomacy, for example).

That said, I'm looking at an ePub copy of Project Phoenix on my Nook now, and it looks great with no manual code editing. Sure, I could have fixed up the titles a bit, but that's just nitpicking.

Anyway, if you want to play around with it and make your own ebooks, I uploaded it here:
https://github.com/labster/epu2html

I look forward to the inevitable bug reports.

(Or a quagmire of regular expressions?
>What's the collective noun for regexes?)

"Regular expressions." ;-p

Actually, yes. I've worked on moving Eyrie.net stories into MOBI format.

In order to do that, I wrote a tool (Text-Transformation Language) in QB64, so that I could write a re-usable "script" to move Gryphon's ASCII format into .HTML, and then I could use something like "Calibre" to move that HTML into MOBI and EPUB.

You can find Text-Transformation Language here:
http://dustinian.com/pg/texttransformationlanguage

You can find an explanation of Text-Transformation language, including a "sample" script that it written expressly for moving Eyrie stories into HTML here:
http://www.qb64.net/forum/index.php?topic=4502.0

I meant to get more posted by now, but my wife and I are expecting, and our due date was literally yesterday. I'm still on pins-and-needles waiting for my daughter (our first-born).

I have a 1.6 of TTL in-progress that fixes a few bugs, but if you compile TTL in QB64... the current eyrie sample script has run on all the stories that I've tried.

As I have a Kindle v1 in my hands (and have used it for a long time--we're talking first production batch Kindle,) I've had a long time interest in converting selected stories into a readable format.

I've made an attempt in the past using php code (with partial success--BOfH stories came out fine, but not SotS.) Now, since I want to re-familarize myself with C#, I'm taking a working mini-Notepad program I made for a course project and kludging in an internal find&replace system for HTML editing. I intend to improve on the interface for that in the future, but this'll give me something to chew on during Canadian apres ski through next week. It will, of course, be a self-contained EXE easily run anywhere, so I can one-off multiple pages and store them on my website for you guys to stick elsewhere as you like.

LAST EDITED ON Apr-06-12 AT 02:17 PM (EDT)
 
http://ebook.online-convert.com/convert-to-mobi

I simply saved the 7 txt files of WL into a single batch and shoved it through the above today, just out of curiosity. It isn't... horrible. about a 1/3 way through Battle 1, it italicised EVERYTHING. But readable.

4/6 Now that I've actually read through the output from above, some notes. The odd italics issue appears to be an artifact related to Ben's tags around music cues. Causes the italics to turn on/off at odd intervals. But otherwise, it handled the actual text blocking fairly well.

Brian

Yeah, see, this is the problem I ran into with the NXE Paperback conversions. Markup languages like HTML and TeX are made to be machine processed. EPU's works are not. Getting it right consistently and entirely automatically is... not impossible but it requires a great deal of hand-holding which contradicts the "entirely automatically" bit.

Try Calibre. It typically does a good job with format conversions and at the least it gives you something that you can tweak by hand if^h^hwhen necessary.

As an aside, I've approached this from the opposite direction, and I can report that while the stock browser does a workmanlike job on anything from www.eyrie.net/, the Polaris Office text reader is absolute _crap_ at handling, well, anything. Source is large chunks of eyrie.net across various of UF, NXE, WL, TIA, etc. All formats are either .txt as delivered by Rei or offline versions of same.

uBook reader, my go-to software of choice for, well, reading, isn't stable on Android yet.

This is the only mention I will make of iOS devices.

--
Kokuten Daysleeper
RCW #13013
(Insert Witticism Here)