[ EPU Foyer ] [ Lab and Grill ] [ Bonus Theater!! ] [ Rhetorical Questions ] [ CSRANTronix ] [ GNDN ] [ Subterranean Vault ] [ Discussion Forum ] [ Gun of the Week ]

Eyrie Productions, Unlimited

Subject: "All Eyrie .TXT converted to .HTML, .EPUB, and .MOBI" Archived thread - Read only
 
  Previous Topic | Next Topic
Printer-friendly copy    
Conferences General Topic #1361
Reading Topic #1361
dustinian
Member since Sep-18-11
9 posts
Jun-21-15, 10:06 PM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
"All Eyrie .TXT converted to .HTML, .EPUB, and .MOBI"
 
   LAST EDITED ON Jun-29-15 AT 03:46 PM (EDT)
 

TL;DR


I've converted all Eyrie Productions, Unlimited .TXT files into .HTML, then the .HTML to .EPUB and .MOBI. It is not perfect. I created this output with a script that needs some work (and I will keep working on it). But they're available here, as .ZIP files:

Background


Hoo, boy, I am jittery as I type this. I have been working on this off-and-on since this thread back in January 2012. Please don't judge too harshly... my daughter was born in February 2012.
I haven't kept up with this site like I used to. I hope nobody's already got any major work done with a .TXT to .HTML conversion. I did notice Gryphon was starting to post in .HTML, so I hope this means he'd be open to seeing some work on a conversion of the old material.
If there's already ongoing work, or if Gryphon wants to preserve the .TXT stories as-is, no biggie. My ultimate goal is to get the .HTML cleaned up so I can use Calibre to convert to .MOBI/.EPUB for my personal Kindle and Android tablets. But if Gryphon can use the .HTML work, I'm delighted to share it. I would love to "give back" to this site somehow.
I started "Warrior's Legacy" back in the mid-90s, and "Symphony of the Sword" almost tanked my first semester of college back in '99. I've sent Gryphon a few fan letters over the years (like 2 or 3 tops). These stories are some of my favorite to re-read, easily as re-readable as Sandy Mitchell's "Cain" books or Aaron Allson's "Wraith Squadron."

Methodology


You might ask: Why on Earth did this take him so long?
Well, I didn't just write a script to convert the .TXT to .HTML. First I created my own scripting language...

TTL: Text Transformation Language


TTL allows the user to write a script to transform text files, and then run that script from the command-line. The best place to read about it is GitHub. I keep the README file on GitHub quite up-to-date.

It's taken me most of the last 3-point-something years to get TTL stood up and error-free. I've made a big push on TTL in recent weeks (as you can see from the commit history on GitHub), and my wife very generously gave me this Father's Day (i.e. watched our daughter all day) so I could focus on this project.
BTW... In case you ask: "Good god, man, why didn't you just learn Regular Expressions?" Well... I tried that. I found spending 3 years writing my own scripting language easier... I think I have a mental block when it comes to Perl. Especially when dealing with the sleep deprivation that comes with an infant/toddler/preschooler.

Eyrie Script


So TTL can do any sort of script you write to any sort of ASCII file. But let me share the specific script I wrote for Eyrie files:

Help!


So if the .HTML isn't yet perfected, why am I posting this? I don't like posting incomplete stuff. But I've waited 3 years to post what I have. And what I have is pretty darn near complete: A totally working scripting language, a mostly-developed script, and legitimate output that at least gets all the paragraph tags mostly right to show for it.

Feedback


But now I need some feedback. What issues do you see with the HTML output? How can I get the script closer to "done" to minimize the hand-coding that'll need to take place?
Example: I can't figure out how to handle the "credits" at the beginning and end of each story in the script. Any thoughts?
Don't try to go edit a lot by hand. Just point out things you see consistently, and I'll try to script them across all the files.

Run it Yourself


If you read all this and you want to give TTL a shot, and you think after reading the GitHub documentation you can edit the script, then by all means! Here is a .ZIP file that contains:

  • All of all the .TXT files

  • The .EXE file

  • The .TTL script file

  • And a .BAT file that:

    • Runs the .TTL script in every folder

    • Copies the .TXT to .HTML

    • Deletes the .TXT


To edit the script, just edit the .TTL file. To run the script, just run the .BAT file.

Thank You


Long post, I know! Thank you for reading! And thanks in advance for any help you offer! Now that .TTL is up and running, I'm super excited to work on the script in my spare time.

Updates



  Printer-friendly page | Top

  Subject     Author     Message Date     ID  
  RE: All Eyrie .TXT converted to .HTML Gryphonadmin Jun-21-15 1
     RE: All Eyrie .TXT converted to .HTML zojojojo Jun-22-15 2
         RE: All Eyrie .TXT converted to .HTML Gryphonadmin Jun-22-15 3
             RE: All Eyrie .TXT converted to .HTML dustinian Jun-22-15 4
                 RE: All Eyrie .TXT converted to .HTML zwol Jun-22-15 5
                     RE: All Eyrie .TXT converted to .HTML TheOtherSean Jun-22-15 6
                         RE: All Eyrie .TXT converted to .HTML Gryphonadmin Jun-22-15 7
                             RE: All Eyrie .TXT converted to .HTML zwol Jun-22-15 8
     RE: All Eyrie .TXT converted to .HTML dustinian Jun-26-15 15
         RE: All Eyrie .TXT converted to .HTML dustinian Jun-29-15 19
  RE: All Eyrie .TXT converted to .HTML JeanneHedge Jun-23-15 9
     RE: All Eyrie .TXT converted to .HTML dustinian Jun-24-15 10
         RE: All Eyrie .TXT converted to .HTML JeanneHedge Jun-24-15 11
             RE: All Eyrie .TXT converted to .HTML dustinian Jun-25-15 14
     RE: All Eyrie .TXT converted to .HTML Bushido Jun-27-15 18
  RE: Text Transformation Language Proginoskes Jun-24-15 12
     RE: Text Transformation Language dustinian Jun-25-15 13
         RE: Text Transformation Language zwol Jun-26-15 16
             RE: Text Transformation Language Vorticity Aug-01-15 21
                 RE: Text Transformation Language zwol Aug-01-15 22
                     RE: Text Transformation Language Vorticity Aug-02-15 24
         RE: Text Transformation Language Proginoskes Jun-26-15 17
             RE: Text Transformation Language dustinian Jul-31-15 20
                 RE: Text Transformation Language Proginoskes Aug-02-15 23

Conferences | Topics | Previous Topic | Next Topic
Gryphonadmin
Charter Member
22375 posts
Jun-21-15, 10:31 PM (EDT)
Click to EMail Gryphon Click to send private message to Gryphon Click to view user profileClick to add this user to your buddy list  
1. "RE: All Eyrie .TXT converted to .HTML"
In response to message #0
 
   There are a few reasons why I can't really get into a detailed response to this right now, but I just wanted to note that I'm not against the concept in principle, and if you and others here want to keep working on it, you go right ahead. My take is "I ain't doin' that," not "don't be doin' that." :)

Also, a couple of quick notes:

- Feel free to cadge the stylesheet I've been using off the source of one of the more recent files if you want; there are a few variations of it as I've adapted it to the needs of different pieces, but the core of it is the same in all of them and you probably know a lot more about that kind of thing than I do, so you should be able to figure out what I was trying to do with it pretty easily. If you want, I can share some internal notes I've thrown together for the rest of the crew in re working with said stylesheet. (In practice, I do most of it by hand or with search-and-replaces within TextPad; the latter, for instance, is how I do the <p></p> tags on the bulk of the text, after initially writing it as plain text with tabbed indents.)

- Apart from a couple of early experiments and one case where I was recasting a bunch of the story anyway, I haven't attempted and don't plan to attempt my own back catalog conversion; too much work when I could (theoretically) be coming up with new material. In particular, manually sorting out all the hyphenated emphases in the old files (why, oh why, did I decide to do that that way?) is way too much of a faff for me to want to bother with it. So no worries about overlap or anything.

- I never figured out a way to do the two-column credits with lyrics thing in non-fixed text either, which is more or less why I stopped doing that.

Good luck & may the gods have mercy on your soul. :)

--G.
-><-
Benjamin D. Hutchins, Co-Founder, Editor-in-Chief, & Forum Mod
Eyrie Productions, Unlimited http://www.eyrie-productions.com/
zgryphon at that email service Google has
Ceterum censeo Carthaginem esse delendam.


  Printer-friendly page | Top
zojojojo
Charter Member
631 posts
Jun-22-15, 06:28 AM (EDT)
Click to EMail zojojojo Click to send private message to zojojojo Click to view user profileClick to add this user to your buddy list Click to send message via AOL IM  
2. "RE: All Eyrie .TXT converted to .HTML"
In response to message #1
 
  
> - I never figured out a way to do the two-column credits with lyrics
>thing in non-fixed text either, which is more or less why I stopped
>doing that.
>
>Good luck & may the gods have mercy on your soul. :)
>


this seems like something one can do with html tables fairly easily, if one were so inclined...

-Z


---
Remember kids: guns make you stupid, duct tape makes you smart.


  Printer-friendly page | Top
Gryphonadmin
Charter Member
22375 posts
Jun-22-15, 09:42 AM (EDT)
Click to EMail Gryphon Click to send private message to Gryphon Click to view user profileClick to add this user to your buddy list  
3. "RE: All Eyrie .TXT converted to .HTML"
In response to message #2
 
   >html tables fairly easily

What a fascinating phrase. :)

--G.
-><-
Benjamin D. Hutchins, Co-Founder, Editor-in-Chief, & Forum Mod
Eyrie Productions, Unlimited http://www.eyrie-productions.com/
zgryphon at that email service Google has
Ceterum censeo Carthaginem esse delendam.


  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jun-22-15, 10:49 AM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
4. "RE: All Eyrie .TXT converted to .HTML"
In response to message #3
 
   I can do tables, my limitation was the .MOBI doesn't support them. I'd like to find a solution that works seamlessly in .MOBI and .EPUB. But I may do tables and see how Calibre handles it...


  Printer-friendly page | Top
zwol
Member since Feb-24-12
299 posts
Jun-22-15, 03:29 PM (EDT)
Click to EMail zwol Click to send private message to zwol Click to view user profileClick to add this user to your buddy list  
5. "RE: All Eyrie .TXT converted to .HTML"
In response to message #4
 
   >I can do tables, my limitation was the .MOBI doesn't support them. I'd
>like to find a solution that works seamlessly in .MOBI and .EPUB. But
>I may do tables and see how Calibre handles it...

I don't know beans about the limitations of either .mobi or .epub, but here is a demo HTML conversion of a two-column credit roll from Symphony 1. No tables, just divs and CSS, but it's possible that e-readers do not implement this part of CSS correctly.


  Printer-friendly page | Top
TheOtherSean
Member since Jul-7-08
246 posts
Jun-22-15, 07:24 PM (EDT)
Click to EMail TheOtherSean Click to send private message to TheOtherSean Click to view user profileClick to add this user to your buddy list  
6. "RE: All Eyrie .TXT converted to .HTML"
In response to message #5
 
   How about just wrapping the two-column section in the code tag, or a span or div styled with a fixed-width font?

--
The Other Sean - Don't accept substitutes!
Quis Custodiet Ipsos Custodes?


  Printer-friendly page | Top
Gryphonadmin
Charter Member
22375 posts
Jun-22-15, 07:27 PM (EDT)
Click to EMail Gryphon Click to send private message to Gryphon Click to view user profileClick to add this user to your buddy list  
7. "RE: All Eyrie .TXT converted to .HTML"
In response to message #6
 
   >How about just wrapping the two-column section in the code tag, or a
>span or div styled with a fixed-width font?

Not my call, but if I were doing it, I would say the former sort of defeats the purpose of converting everything else into a pitch-independent format, and in either case, with the rest of the document converted into properly flowed variable-pitch text, it would look pretty bad.

--G.
-><-
Benjamin D. Hutchins, Co-Founder, Editor-in-Chief, & Forum Mod
Eyrie Productions, Unlimited http://www.eyrie-productions.com/
zgryphon at that email service Google has
Ceterum censeo Carthaginem esse delendam.


  Printer-friendly page | Top
zwol
Member since Feb-24-12
299 posts
Jun-22-15, 10:22 PM (EDT)
Click to EMail zwol Click to send private message to zwol Click to view user profileClick to add this user to your buddy list  
8. "RE: All Eyrie .TXT converted to .HTML"
In response to message #7
 
   Yeah, if I understand correctly (from the mention of .epub/.mobi/Calibre) the goal here is to get something that looks good on an e-reader, and those tend to have very narrow screens, like, 40 columns or less (not in a fixed font, but still). Fixed-pitch credits would look awful, probably every single line would get either cut off or wrapped.

TBH I'm not sure the two-column credit rolls can be made to look good on something that narrow -- you'll notice if you make your browser window narrow enough that my conversion gives up and shoves the song lyrics down below the credits.


  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jun-26-15, 06:31 AM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
15. "RE: All Eyrie .TXT converted to .HTML"
In response to message #1
 
  
>There are a few reasons why I can't really get into a detailed response to this right now, but I just wanted to note that I'm not against the concept in principle, and if you and others here want to keep working on it, you go right ahead. My take is "I ain't doin' that," not "don't be doin' that." :)

Awesome, it's nice to have permission!


>Feel free to cadge the stylesheet I've been using...

Not quite ready for a stylesheet yet, I want to get some good structure in place first. The next thing I need to research is whether or not Calibre handles the new "HTML5" tags well (article, aside, section), or if I should just use divs.

But I'll totally use your stylesheet once I get the structure down!

>If you want, I can share some internal notes I've thrown together for the rest of the crew in re working with said stylesheet.

If those are handy, that'd be great!

>In particular, manually sorting out all the hyphenated emphases in the old files (why, oh why, did I decide to do that that way?) is way too much of a faff for me to want to bother with it.

I actually added a few lines of TTL for this. Aside from some <i> tags that remain open in a few of the stories, it's worked pretty well. I'll have to manually fix those <i> tags after I'm good and done with the script.

I never figured out a way to do the two-column credits with lyrics thing in non-fixed text either, which is more or less why I stopped doing that.

I'm gonna tackle that LAST. ;)

Anyway, I've also updated the files behind the hyperlinks in the first post! The "input" file has a new .TTL script, and the "output" file has the HTML results! They're better than what I posted a few days ago, but it's a work-in-progress!


  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jun-29-15, 03:51 PM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
19. "RE: All Eyrie .TXT converted to .HTML"
In response to message #15
 
   Just a quick note to say that I updated the original post with links to .EPUB and .MOBI output!

Upcoming goals:


  • Take another cruise through the .HTML files looking for issues I can fix in the script.

  • Finalize the .HTML.

  • Start fixing the credit rolls by hand.

  • Add any illustrations to the .HTML by hand.

  • Output nicer .EPUB and .MOBI files with meta data (title, author, etc.) included.

Probably still months to go, but I'll keep posting progress so folks can use all this ASAP. I know I'll be adding the .MOBIs to my Kindle tonight, and the .EPUBs to my phone.


  Printer-friendly page | Top
JeanneHedge
Charter Member
933 posts
Jun-23-15, 07:15 PM (EDT)
Click to EMail JeanneHedge Click to send private message to JeanneHedge Click to view user profileClick to add this user to your buddy list  
9. "RE: All Eyrie .TXT converted to .HTML"
In response to message #0
 
   Anyone's interested, I've got pretty much everything as a MOBI file, complete with non-generic cover.


Jeanne


Jeanne Hedge
http://www.jhedge.com
1st Courier of the Heavenly Prophets for Tianxia
"Never give up, never surrender!"



  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jun-24-15, 11:19 AM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
10. "RE: All Eyrie .TXT converted to .HTML"
In response to message #9
 
   Please do post. How did you convert? Did you go straight from .TXT to .MOBI?


  Printer-friendly page | Top
JeanneHedge
Charter Member
933 posts
Jun-24-15, 06:17 PM (EDT)
Click to EMail JeanneHedge Click to send private message to JeanneHedge Click to view user profileClick to add this user to your buddy list  
11. "RE: All Eyrie .TXT converted to .HTML"
In response to message #10
 
   >Please do post. How did you convert? Did you go straight from .TXT to
>.MOBI?

Some stories required little re-formatting (I don't like hard breaks in the middle of a sentence). For those, I loaded the TXT into Word, got it looking how I wanted it to (italics, etc), then Save As "Web Page, Filtered". That creates a htm file, which I loaded into Calibre to generate the MOBI. For the most recent HTML stories, those go pretty much straight into Calibre.

As for covers, I don't like the generic covers Calibre does either, so pretty much everything got a custom cover via Calibre's "Generate Cover" option before converting to MOBI. The covers have images on them. I try to find something that has to do with the story, or the mood of the story. For series I've gone to EPU's series pages and used the main images as part of the covers of those stories to tie them together. If there was a Bacon Comics cover for a story, that was usually used as the MOBI's cover.


Jeanne


Jeanne Hedge
http://www.jhedge.com
1st Courier of the Heavenly Prophets for Tianxia
"Never give up, never surrender!"



  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jun-25-15, 10:07 PM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
14. "RE: All Eyrie .TXT converted to .HTML"
In response to message #11
 
  
>Some stories required little re-formatting (I don't like hard breaks in the middle of a sentence). For those, I loaded the TXT into Word, got it looking how I wanted it to (italics, etc), then Save As "Web Page, Filtered". That creates a htm file, which I loaded into Calibre to generate the MOBI.

I'd love to look at the MOBI output.


  Printer-friendly page | Top
Bushido
Member since Apr-8-10
376 posts
Jun-27-15, 11:51 AM (EDT)
Click to EMail Bushido Click to send private message to Bushido Click to view user profileClick to add this user to your buddy list  
18. "RE: All Eyrie .TXT converted to .HTML"
In response to message #9
 
   I'd like a copy of those please. I don't have an e-reader, but I can use .mobi files on my Nexus 7 tablet via Google Play Books. The hard line breaks in UF don't translate well on it, so the mobi versions would be nice to have. The nice thing about Google Play Books is once I've uploaded a mobi file to my account, I can access it on any device and it will even bookmark my place. I'll probably leave a copy of UF on my tablet though so I can access it while offline. I got through all of the Dresden Files series that way on a recent vacation where I didn't have wi-fi or cell service.

--------
Wedge Defense Force General
Order 12: "Try to avoid
freaking the mundanes."


  Printer-friendly page | Top
Proginoskes
Member since Dec-3-09
210 posts
Jun-24-15, 08:01 PM (EDT)
Click to EMail Proginoskes Click to send private message to Proginoskes Click to view user profileClick to add this user to your buddy list  
12. "RE: Text Transformation Language"
In response to message #0
 
   LAST EDITED ON Jun-24-15 AT 08:29 PM (EDT)
 
Some features I think would be very useful for TTL:

  • Syntactic sugar commands for REPLACE "find" WITH "foo" + "find" + "bar" ONCE and the like. I propose PREPEND "foo" TO "find", APPEND "bar" TO "find", and SURROUND "find" WITH "foo" AND "bar".

  • Speaking of REPLACE ... ONCE, I think that should be the default behaviour. It's certainly what anybody with experience in other text-manipulation languages would expect. How difficult would it be to add a RECURSIVELY word to the language? EDIT: Actually, what would be best would be a behavioral directive, allowing one to put, say, BY DEFAULT REPLACE ONCE at the beginning of a script to add an implicit ONCE to each REPLACE. This has the added benefit that if you later do change the default behaviour, older scripts can be fixed with the single line BY DEFAULT REPLACE RECURSIVELY instead of having to hunt down every REPLACE and add a RECURSIVELY to each one.

  • As a follow-up to the edit above, case-sensitivity directives and modifying words. BY DEFAULT OBSERVE CASE, REPLACE "find" WITH "add" IGNORING CASE, and so on.

  • In general, examine AWK and sed for features that would make sense to steal. In particular, sed has a regular expression language that is simpler than Perl's or PCRE's, but still has most of the really useful features.

As an aside, it's a shame you didn't write this in a descendant of C, because I'd love to hack on the actual source to try to implement these suggestions myself, but I have a slightly irrational fear and loathing of any programming language that has the word "Basic" in its name.


  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jun-25-15, 10:04 PM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
13. "RE: Text Transformation Language"
In response to message #12
 
   Hey, wow, first of all, thanks for taking the time to look over TTL and to put together a thoughtful response!

>Syntactic sugar commands for REPLACE "find" WITH "foo" + "find" + "bar" ONCE and the like. I propose PREPEND "foo" TO "find", APPEND "bar" TO "find", and SURROUND "find" WITH "foo" AND "bar".

I like this idea. And I love that you use the phrase "syntactic sugar." I'm also an Eric S. Raymond fan. I read The Cathedral and the Bazaar just a few years ago after I saw "Revolution OS" on Netflix.

I will implement this soon, it'd be quite easy. I wouldn't need to add any code to the "string manipulation.bi" file since the functionality all exists. I'd just need to add a few words to the "Validate_Command" sub and a few entries in the "Populate_Command" sub in the main "ttl.bas" file.

I know you say that you loathe BASIC, but I'd like to ask you to take a look at one sub and one sub only... "Populate_Command." I really hate the horribly inelegant way "Populate_Command" recognizes TTL syntax. It's a bunch of "If" statements encapsulated in a massive "Select" statement. Any advice there, even just general advice unrelated to BASIC, would be greatly appreciated.

>Speaking of REPLACE ... ONCE, I think that should be the default behaviour. It's certainly what anybody with experience in other text-manipulation languages would expect. How difficult would it be to add a RECURSIVELY word to the language? EDIT: Actually, what would be best would be a behavioral directive, allowing one to put, say, BY DEFAULT REPLACE ONCE at the beginning of a script to add an implicit ONCE to each REPLACE. This has the added benefit that if you later do change the default behaviour, older scripts can be fixed with the single line BY DEFAULT REPLACE RECURSIVELY instead of having to hunt down every REPLACE and add a RECURSIVELY to each one.

I find myself using the recursive version much more often, but since it's the one that could result in an error, I like the idea of making "recursive" the one that you have to specify. I'll definitely implement that.

As for a selector for the default... I'll toy with that idea. I'm not against it, but I need to think on it.

>As a follow-up to the edit above, case-sensitivity directives and modifying words. BY DEFAULT OBSERVE CASE, REPLACE "find" WITH "add" IGNORING CASE, and so on.

This one scares me for the amount of logic I'd need to put in place. I'd need to either look for every possible mixed case version of the "FIND" command or build a separate variable for the input text that's all U- or LCased, and then replicate all the commands on both versions of input text to keep all the characters in the same place... and then do I always replace whatever's found with the case that's in the "Add" sub-string in the script, or do I try to re-create the destination casing?

I'm going to add this to the potential enhancement list, but I don't think I'll add it any time soon.

I appreciate the suggestion, don't get me wrong! I'm just not quite programmer enough to confidently take this one without doing some more research on FreeBASIC.

>In general, examine AWK and sed for features that would make sense to steal. In particular, sed has a regular expression language that is simpler than Perl's or PCRE's, but still has most of the really useful features.

Will do! I only learned about "sed" the other day on Reddit! I suspect if I'd know about "sed" back in '11, TTL might have never come to be. I'll check out "AWK" as well!

>As an aside, it's a shame you didn't write this in a descendant of C, because I'd love to hack on the actual source to try to implement these suggestions myself, but I have a slightly irrational fear and loathing of any programming language that has the word "Basic" in its name.

Also reminding me of Eric S. Raymond! I can't find it now, but I read one of his essays, and it included a brief aside about "real hackers" learning C and not BASIC.

I know, I know, BASIC is more abstracted from the binary executable, which means the compiler can do some really inefficient things on your behalf. But starting with a Commodore64, BASIC has been my programming language literally since I was an 8-year-old desperately trying to re-type the BASICA code that "3-2-1 Contact" magazine published without creating a typo.

I graduated to QBasic when my Dad got us our first PC, and Visual Basic when I started writing Office Macros at work.

I've never done much "serious development," so, while I work with a great many C programmers, I haven't learned any C myself beyond two 1000-level C++ courses in college.

Hey, thanks again for your response, and for looking at TTL. You'll see several of the ideas you've generously provided in the next version of TTL... hopefully this weekend, but maybe the weekend of the 4th.


  Printer-friendly page | Top
zwol
Member since Feb-24-12
299 posts
Jun-26-15, 12:21 PM (EDT)
Click to EMail zwol Click to send private message to zwol Click to view user profileClick to add this user to your buddy list  
16. "RE: Text Transformation Language"
In response to message #13
 
   >I know you say that you loathe BASIC, but I'd like to ask you to take
>a look at one sub and one sub only... "Populate_Command." I really
>hate the horribly inelegant way "Populate_Command" recognizes TTL
>syntax. It's a bunch of "If" statements encapsulated in a massive
>"Select" statement. Any advice there, even just general advice
>unrelated to BASIC, would be greatly appreciated.

I'm not the person this was addressed to, but I am a computer language nerd ;-)

Parsing a "formal language", which is what you're trying to do, is a really well-understood problem in theoretical computer science, and one that comes up all the damn time in industry, but almost no general-purpose programming languages are any good at it. (One of the design goals of Perl 6 was to become good at this. Perl 6 has been vaporware for ~15 years now.) If you look at, say, the handwritten code in GCC that parses C, you'll see that it has the same kind of awkward structure as your parser - lots of functions full of big hairy if-else chains conditioned on what word comes next.

However, there are special-purpose programming languages that are good at this task. The thing you want is called a parser generator. This is a tool that will take a short, readable specification that looks something like your documentation for TTL, and generate a blob of completely unreadable code that does the job of parsing it, which you then combine with the rest of your program. Unfortunately I don't know if there are any parser generators that work with this dialect of BASIC. This one works with VB.NET, though, and it looks like its documentation is written for people who don't already know from formal languages. If that one doesn't suit you, Wikipedia has a long list of more.

(GCC doesn't use a parser generator because it would get in the way of going as fast as possible and generating diagnostics that are as clear as possible. This is a major concern for a production C compiler but not so much for you.)

>Will do! I only learned about "sed" the other day on Reddit! I suspect
>if I'd know about "sed" back in '11, TTL might have never come to be.
>I'll check out "AWK" as well!

sed and awk are certainly good things to crib from, but please do adopt the PCRE regular expression syntax if you're going to have regular expressions at all. Yes, it is cryptic and hard to learn, but basically every popular programming language nowadays supports something close to it, which means everyone only has to learn that cryptic syntax once.

(In particular please don't copy the regular expression syntax from sed -- sed's regex syntax is closely related to PCRE's, but simultaneously harder to read and less powerful.)

>I know, I know, BASIC is more abstracted from the binary executable,
>which means the compiler can do some really inefficient things on your
>behalf. But starting with a Commodore64, BASIC has been my programming
>language literally since I was an 8-year-old desperately trying to
>re-type the BASICA code that "3-2-1 Contact" magazine published
>without creating a typo.

You shouldn't feel bad about preferring BASIC. BASIC got a lot of flak back in the C64 days because it was impossible to write anything in it that wasn't a pile of spaghetti. But modern BASIC has all the same structured-programming conveniences that will be found in other languages, and is reasonably efficient as well.

If you want to learn another programming language, I would recommend not picking C. You will learn more about how to program well by learning Haskell or Scheme, and you will gain more practical ability to make computers do what you want by learning Python, PHP, and/or JavaScript. C nowadays is actually bad at teaching you how the machine works at the low level, for reasons too tedious to get into here; if you want to learn that, have a look at Rust.


  Printer-friendly page | Top
Vorticity
Member since Feb-6-12
110 posts
Aug-01-15, 05:39 AM (EDT)
Click to EMail Vorticity Click to send private message to Vorticity Click to view user profileClick to add this user to your buddy list Click to send message via AOL IM  
21. "RE: Text Transformation Language"
In response to message #16
 
   > (One of the design goals of Perl 6 was to become good at this.
> Perl 6 has been vaporware for ~15 years now.)

Actually Perl 6 Grammars have been very good for 5 years now -- it was necessary to do this part first, because Perl 6 is parsed with Perl 6. Mainly because nothing else is good enough to parse Perl 6 syntax with any level of sanity. One can throw together a little DSL pretty easily: write up a grammar that looks not too different from a spec, add a few actions to the AST, and call a parse file method.

As to the vaporware part, the release date for 6.0.0 is Christmas 2015, with a fixed spec and a compiler that meets that spec. Which is not to say that it's not ready for some purposes now. Grammars, in particular, are a very mature feature. Native arrays... somewhat less so.

That said about the release timing, I should try to fix some Perl 6 compiler bugs this weekend.


  Printer-friendly page | Top
zwol
Member since Feb-24-12
299 posts
Aug-01-15, 07:23 PM (EDT)
Click to EMail zwol Click to send private message to zwol Click to view user profileClick to add this user to your buddy list  
22. "RE: Text Transformation Language"
In response to message #21
 
   >As to the vaporware part, the release date for 6.0.0 is Christmas
>2015, with a fixed spec and a compiler that meets that spec. Which is
>not to say that it's not ready for some purposes now. Grammars, in
>particular, are a very mature feature. Native arrays... somewhat less
>so.

I hope you'll understand when I say that I was super excited by the Apocalypses (particularly the grammar one) when they originally came out ... when was that, 1998? I was still in college, so probably around then ... but after several years of very little progress I kinda stopped paying attention.

I might go back and look at it once the release has happened, though, if only to see how much it lines up with my recollection of what it was originally planned to be.


  Printer-friendly page | Top
Vorticity
Member since Feb-6-12
110 posts
Aug-02-15, 07:06 PM (EDT)
Click to EMail Vorticity Click to send private message to Vorticity Click to view user profileClick to add this user to your buddy list Click to send message via AOL IM  
24. "RE: Text Transformation Language"
In response to message #22
 
   Apocalypse 1 came out in 2001. 1998 is a bit closer to the Absolute Destiny Apocalypse, which now that I think about it might have had an impact on the naming. And yes, 15 years is a bit too long for most people to stay excited.

It probably won't be like your recollection that much. The general concepts have emerged unchanged, but a lot of things have happened in the interim. Lots of input from languages like Haskell and Java through various contributors. And then there was the whole multi-core processor thing, that makes parallel computing a major feature. The original idea was one VM that would run everything, Parrot. Now Parrot is dead (joke complete), and Rakudo Perl 6 runs on 3 VMs (with Javascript coming soon).

If you want a good example of how the grammars turned out, take a look at JSON::Tiny.


  Printer-friendly page | Top
Proginoskes
Member since Dec-3-09
210 posts
Jun-26-15, 04:22 PM (EDT)
Click to EMail Proginoskes Click to send private message to Proginoskes Click to view user profileClick to add this user to your buddy list  
17. "RE: Text Transformation Language"
In response to message #13
 
  
I find myself using the recursive version much more often, but since
it's the one that could result in an error, I like the idea of making
"recursive" the one that you have to specify. I'll definitely
implement that.

As for a selector for the default... I'll toy with that idea. I'm not
against it, but I need to think on it.


The main reason I went back on suggesting that a "recursive" word be required is that changing the default behaviour will break existing scripts. If you implement a default selector, that makes fixing those scripts a matter of adding a single line at the beginning, rather than changing some large fraction of the lines throughout.
>As a follow-up to the edit above, case-sensitivity
directives and modifying words. BY DEFAULT
OBSERVE CASE
, REPLACE "find" WITH "add"
IGNORING CASE
, and so on.

This one scares me for the amount of logic I'd need to put in place.
I'd need to either look for every possible mixed case version of the
"FIND" command or build a separate variable for the input text that's
all U- or LCased, and then replicate all the commands on both versions
of input text to keep all the characters in the same place... and then
do I always replace whatever's found with the case that's in the "Add"
sub-string in the script, or do I try to re-create the destination
casing?


My impression is that generally, case sensitivity only affects the "search" part of "search and replace". But yeah, "where does it end?!" is a valid concern here.

Regarding BASIC:

My dislike of BASIC is, as I mentioned, not completely rational; as Zwol said, modern BASICs are probably as a general rule better to use than Standard C or C++. Modern BASICs no longer force you to write spaghetti code, advancements in compiler state-of-the-art have largely obsoleted hand-tuning critical code, and so on. The only thing that I really have to justify my continued dislike is the ambiguous/context-sensitive constructs. (The code fragment aleph = beth can mean two different things, depending on the surrounding code, and to me, being used to C/Java/PHP syntax, that's just not right.)

All that said, I actually looked into installing FreeBASIC so I could poke at the code, when I discovered a rather bigger problem: both computers I use are Darwin machines, and it's impossible to get a Darwin FreeBASIC compiler. I suppose the thing to do would be to use WINE to run the Windows compiler, but it's a little frustrating.


  Printer-friendly page | Top
dustinian
Member since Sep-18-11
9 posts
Jul-31-15, 11:17 PM (EDT)
Click to EMail dustinian Click to send private message to dustinian Click to view user profileClick to add this user to your buddy list  
20. "RE: Text Transformation Language"
In response to message #17
 
   LAST EDITED ON Jul-31-15 AT 11:23 PM (EDT)
 
Good even, Proginoskes!

It took longer than I thought (everything always does), but I've finally added quite a bit of syntactic sugar.

I implemented your ideas (append, prepend, surround), and also several flavors of "delete" (rather than having replace "x" with "").

I haven't switched the default on "recursive" yet. I'll keep thinking on that one.

I actually think I'm going to take a break on TTL itself (barring bugs) to refocus on the Eyrie Productions ebooks again (hopefully improving the quality of what I've produced so far).

Sorry to hear about your non-starter with FreeBasic! I tried to google "Darwin Machines," and I found two options:

I think it's probably the OS. Let me know if there are any BASIC compilers that work in Darwin OS. Maybe a conversion would be quite easy? In fact, I see that QB64 has a Mac OS version, and TTL actually began its life in QB64 before I made the switch to FreeBasic.

If QB64 works in Darwin OS, let me know. I know exactly how to do that conversion. It would take less than an hour.

Anyway, thanks again for your comments and suggestions. I used them!

Edit: I guess it, uh, helps if I post the hyperlink: TTL


  Printer-friendly page | Top
Proginoskes
Member since Dec-3-09
210 posts
Aug-02-15, 02:52 PM (EDT)
Click to EMail Proginoskes Click to send private message to Proginoskes Click to view user profileClick to add this user to your buddy list  
23. "RE: Text Transformation Language"
In response to message #20
 
   Oops, assumed you'd know what I meant by "Darwin". It's actually the other way around, you know: Darwin is Apple's variant of the BSD Unix kernel, which they use as the base on which Mac OS X and iOS run, the way Gnome and KDE run on the Linux kernel. (iOS is such a walled garden that it's difficult to tell, and I wasn't thinking of it, but it is Darwin at its kernel.)

Lion post-dates Snow Leopard, so it looks like QB64 will work. Thanks!


  Printer-friendly page | Top

Conferences | Topics | Previous Topic | Next Topic

[ YUM ] [ BIG ] [ ??!? ] [ RANT ] [ GNDN ] [ STORE ] [ FORUM ] GOTW ] [ VAULT ]

version 3.3 © 2001
Eyrie Productions, Unlimited
Benjamin D. Hutchins
E P U (Colour)