Category Archives: Using software

Quickie: Upgrading Windows 8 to Windows 8.1 Problems?

Leave a reply

I’ve been using Windows 8 as a dual-boot option on my iMac for a while. When Windows 8.1 came out, I was pretty excited, as Peter Bright’s review on Ars Technica was quite positive about it.

Unfortunately, I was getting errors doing the upgrade it directly from the Microsoft Store, eventually getting Error 0x0073B92 - 0x20009. I was a little stumped about how to get around this.

The solution was actually simple: disconnect all of your USB devices during the installation. (Internet searching seems to indicate that this is related to a problem with a second hard drive).

Advertisement

Sublime Text: Always run a single file in a project no matter what is focused

Leave a reply

When I program, I often try to split up my code into modules. This requires that I use different files. However, I also want to run my code from a “main” or a “master” file as well. It’s pretty annoying in Sublime Text, or any text editor, to have to switch to your main file to run your project.

I’ve talked about how to do this in TextMate but how can you pull this off in Sublime Text? Let me show you how.

  1. Create a Sublime Text project for your program. You can do this by using the menu Project > Save Project As…”.
  2. Open the settings for your project using the menu Project > Edit Project.
  3. You’ll get a mostly empty text window. Modify this window to include a build definition, such as the following one for Python: 
    • "build_systems": [
  {
    "name": "PyProj",
    "cmd": ["python", "-u", "$file"],
    "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
    "selector": "source.python"
  }
]
  4. Modify the line with “cmd” and change “$file” to the master file you always want to run. For example, if your main file is called pyproj.py, then change it to pyproj.py.
Add the "build_systems" JSON field in your project settings file, accessible through the Project menu > Edit project.

Add the “build_systems” JSON field in your project settings file, accessible through the Project menu > Edit project.

That’s it! While you’re editing your project settings, you can make other customizations as well as you see fit.

Obviously, this isn’t limited to just setting a file to be run no matter what file you’re focused on. You could use this to add command-line arguments to a single project, to run project-specific scripts for building and testing, or to otherwise customize the way the “Control-B” (Command-B on Mac OS X) works when you are running the project. You can essentially define your own build system on a per-project basis in a project file’s “build_systems” section and use any information in Sublime Text’s build system documentation to do so. Unfortunately, the build system documentation doesn’t really describe this because it’s focused more toward Package development.

For more information about how project settings work, look at Sublime Text’s documentation for build systems and for projects!

Programming in the console with GNU Screen and vim

Leave a reply

When I program, I use a text editor. Now, even though I recently bought Sublime Text (and I’ll probably do a post on that later), I still use vi a lot – specifically, vim. I also occasionally used GNU screen (mostly for managing remote IRC sessions with the irssi client).

So why use the console to program? Well, the main reason is that typing in the keyboard and not using the mouse is faster if your keyboard skills are good. Given the quality of most laptop trackpads, getting away from the need to use the mouse is actually something of a relief sometimes. The second reason to be familiar with the console is if you’re logging in to a remote session, you often don’t have a mouse available to you at all.

The learning curve is quite steep for console applications, but once you get familiar with them you can get your editing tasks done very quickly because everything’s quite literally right at your fingertips.

Screen: Access Terminal Applications All The Time, Anywhere You Want

GNU Screen is a terminal emulator that’s available on most flavours of Unix-like operating systems, including Linux, BSD, and Mac OS X. Screen runs inside a shell. I think the “killer feature” of screen is the fact that screen is persistent, so you can essentially make your terminal applications run all the time, from anywhere you want.

A terminal window that has screen running inside it.

A typical screen session

What this means is that if you log into a remote machine, and use “screen”, you can start commands that you expect to run for a long time, close the SSH window, and then log back in and resume what you were doing. Everything will be chugging along just as you had it. It’s especially useful for applications that need to run in the background for a long time, like IRC windows or really, really long compilations or analysis.

Another feature of screen is the fact that it has “tabs”, or multiple windows within screen. You can then run multiple terminal programs in the same screen session, resuming them when you want.

A Basic Screen Workflow

To use screen, open a terminal window and then type screen. You will get a splash screen, but then you will be kicked out into what is essentially a shell. From here, you can basically do everything you normally would in a console.

Let’s say you want a new console to work in – a new “tab”, so to speak. In screen, press Control-a, let go of control, then c (c for create). This will create a new console for you to work in. You now have two consoles!

To switch between these consoles, use Control-a + n (n for next) and Control-a + p (p for previous). Try this now: Control-a c, Control-a n, Control-a p.

In screen, every command is preceded by a “Control-a”. It’s kind of the secret shortcut in the program, it’s often abbreviated, so Control-a and a (which means to go to the beginning of the current line) is often written as C-a a.

Let’s say you’re doing a lot of work in screen and now you need to log out, but you want to keep all of your work active. This is a “detach”. If you want to “detach” a screen and do other things in your original shell, you can do that with C-a d. If you’re working in a windowed environment, if you close the program that screen is running in, it’ll also detach for you.

Later that day, you want to log back in to the machine and resume your work. To “attach” a screen, type screen -R -D. That finds a screen for you, detaches it from other consoles if necessary, and resumes it.

The workflow for screen is pretty simple overall: you basically attach a screen when you start, then you create new windows, and detach when you’re done (or if you’re like me, you’ll just lock your computer and reattach it whenever you get around to it from whereever).

I find that using screen in conjunction with vi is extremely useful: you edit it one screen session, then compile/run/test in another.

For more information, take a look at this handy screen command reference.

Making Screen Look Pretty and Useful

You’ll notice that screen on the surface is pretty minimal. However, there are a lot of features built in that make it quite handy as a terminal emulator.

It’s possible to list the windows in your screen at the bottom of the terminal. You can see which one you’re currently working on. You can list the hostname, the current time, and so forth. With a little bit of work, you can make a very simple statusbar that will increase the usability of screen significantly.

To do this, first edit the .screenrc file in your home directory. Then, add the following lines:

hardstatus on
hardstatus alwayslastline
hardstatus string "%{.kw}%-w%{= kR}(%{r}%n %t%{-}%{= kR})%?%{= .kw}%+w%? %{.kw}%=%c %d/%m/%Y" #B&W & date&time

I won’t step through the syntax in its entirety (it’s very cryptic and confusing), but you’ll end up with a handy listing of the current screen windows in the session with the current one visible in red.

I followed the syntax on the GNU Screen manual to customize the string to my liking. %{.kw} colours the string with a black background and white text…. %-w lists all of the windows in screen up to, but not including your current session, %{= kR} makes the next section black and red, ( is a parentheses, %n is the number of the current window, and so forth. It takes a bit of starting at to understand, but it’s not too difficult to customize to your linking.

Vim: Navigate your Documents in an Instant

Vi is a text editor that originated in the mid 70s. One of the most popular modern incarnations is Vi Improved (vim). One of the most powerful features of Vim is the speed at which you can navigate through your documents. You can immediately jump to lines, delete a range of lines, move forward four spaces (or four words), search for words easily, and more, without your hands leaving the keyboard. I use it for much of my small-scale programming work even though I like my GUI-based text editors.

To launch it, open a text editor and type vi.

Configuring vim

I’m really not that much of a vim power user; my .vimrc file is really simple compared to some people who live in vim full-time. I like navigating code in vim and I use it for most of my console-based editing but if I am working on large, multi-project files I end up using a text editor like Sublime Text instead.

Regardless, it’s nice to have a decently usable set of vim defaults. Here’s mine:

syntax on
set number
set smartindent
set tabstop=4
set shiftwidth=4
set expandtab

In this setup, I turn on syntax highlighting, print numbers on the side (considering how dependent vim is on navigating by line numbers, this is essential), I turn on auto-indentation, set the tab size and indentation sizes to 4, and expand tabs to spaces.

A swift introduction to using vi

vim is pretty intimidating to use, because it uses multiple modes. The default mode is a “navigation” mode where you can move the cursor around, and then there’s a “line editor” mode where you actually modify the text on the screen. There’s also a “command” mode where you enter custom commands, too.

The most important command in vi

If you don’t know anything else about vi, learn this:

ESC :q!

Press the escape key, press colon, press q, then press !. This exits the program with no changes! If you have absolutely no idea how to do anything, you can at least get back out to the console.

Related but important as well:

ESC :wq

Press the escape key, press colon, press w, then press q.

This writes the current file, then quits the program. Basically, it’s a “save and quit”.

Navigating

When you start vim, you’re always in navigation mode. You can navigate using the h, j, k, and l keys. These correspond to left, down, up, and right respectively (this was so ingrained into me that I had to actually start vim to check the directions). One reason why this is great is because the controls are on the home row, so navigating in vi is pretty fast.

If you want to jump to a line, you enter the line number and press G. So going to the first line of a file is 1 G whereas going to line 23 is 23 G.

You can move multiple lines and characters up, left, down, and right as well. 3 h moves three characters left. 5 j moves five lines down.

Editing

i to enter “insert” mode. You can then type to enter characters at the current cursor position. You’ll see -- INSERT-- at the bottom of your screen. To stop editing, press Escape.

There are many variations of insert in vi to help people insert things really, really quickly. I (capital I, vi is case-sensitive) inserts at the beginning of the current line, A inserts at the end of the current line, o inserts a new line before the current line, O inserts a new line after the current line. You can even combine these with numbers and the directional hjkl keys to insert multiple lines. That’s probably too much information.

You can also delete quickly vi as well. x is the basic “delete one character”. dd deletes a line.

Finally, u undoes the last command, so if you find yourself deleting the wrong lines, hit u repeatedly.

Conclusion

I hope this serves as a brief introduction to vim and screen for most people. Anyone with a Unix-like computer and the need to write text or markup should consider looking into these kinds of programs because they increase the flexibility of what you can do.

TextMate and Running a Master File

1 Reply

Many people find my blog searching about how to make their multi-file LaTeX projects build in TextMate.

It’s worth nothing that the TM_MASTER_FILE works for almost any programming project, not just LaTeX. So, if you’re working on a Python or a Ruby project and have a main file that you want to run, set the TM_PROJECT_MASTER to point to that and save yourself some effort. Here’s a reminder on how to do that.

  1. Open your project file (or create one if you don’t have one).
  2. De-select any selected files in the TextMate drawer by clicking on empty space.
  3. Click on the “i” in the bottom-right corner of the drawer.
  4. Add a variable named TM_PROJECT_MASTER.
  5. Set the value to the name of your main file (in my Ludum Dare game, it was painterscat.py

Now your “run” ⌘r shortcut will always run your main file with minimal hassle!

Using PyInstaller to make EXEs from Python scripts (and a 48-hour game design compo)

8 Replies

How to Create a Single Windows Executable from a Python and PyGame Project (Summary)

Here’s how you use PyInstaller and PyGame to create a single-file executable from a project that has a data directory that contains resources like images, fonts, and music.

  1. Get PyInstaller.
    • On Windows, you might also need pywin32 (and possibly MinGW if you don’t have Visual Studio).
    • On Mac OS X, you will need XCode’s command line tools. To install the Command Line tools, first install XCode from the App Store, then go to Preferences – Downloads and there is an option to download them there.
  2. Modify your code so that whenever you refer to your data directory, you wrap it using the following function: 
    def resource_path(relative):
    if hasattr(sys, "_MEIPASS"):
        return os.path.join(sys._MEIPASS, relative)
    return os.path.join(relative)

    An example of usage would be

    filename = 'freesansbold.ttf'
myfontfile = resource_path(os.path.join(data_dir, filename)

    This is mostly for convenience – it allows you to access your resources while developing, but then it’ll add the right prefix when it’s in the deployment environment.

  3. Specify exactly where your fonts are (and include them in the data directory). In other words, don’t use font = Font(None, 26). Instead, use something like font = Font(resource_path(os.path.join('data', 'freesansbold.ttf')), 14).
  4. Generate the .spec file.
    • Windows: (You want a single EXE file with your data in it, hence --onefile). 
      python pyinstaller.py --onefile your_main_file.py
    • Mac OS X: (You want an App bundle with windowed output, hence --windowed). 
      python pyinstaller.py --windowed your_main_file.py
  5. Modify the .spec file so that you add your data directory (note that these paths are relative paths to your main directory.
    • Windows: Modify the section where it says exe EXE = (pyz, and add on the next line: 
      Tree('data', prefix='data'),
    • Mac OS X: Modify the section where it says app = BUNDLE(coll, and add on the next line: 
      Tree('data', prefix='data'),
  6. Rebuild your package. 
    python pyinstaller.py your_main_file.spec
  7. Look for your .exe or your .app bundle in the dist directory.

Phew! That took me a long time – the better part of a few hours to figure out. This post on the PyInstaller list really helped.

So why was I trying to package a Python executable file anyway? Read on…

Ludum Dare 26: 48-hour Game Design Compo

This weekend, I decided to participate in a 48-hour game design “competition”. Ludum Dare is a compo that asks you to create a video game from scratch in a 48-hour time period – you have to write your code and create all of your assets in that time period.

This means no reusing graphics, pictures, music, or sound from other projects, for example. You’re also not supposed to reuse code either. I decided to participate on the Thursday the day before. Most people use the previous weekend as a “warmup weekend” to test their tools, get some practice, and so forth. (My entry is located here, by the way).

I’ll do a more detailed compo writeup later, but I just want to concentrate on one thing that kept me up for hours after the competition: getting a Windows executable created from a Python project that uses PyGame and a data directory.

Python, Distribution, and You

I rather enjoy Python as a programming language. The syntax is reasonably concise, the language does a lot of things for you, and it’s well-laid out. There’s also a lot of good support in the form of third-party libraries. I’ve been using Python for various things for the past few years (usually small scripts for data extraction and analysis in research).

One thing I had never thought about before was distributing a Python project as an executable package, and while it was on my mind throughout the entire compo, I didn’t actually learn the process of creating the package until the last hour of the comp before submission. After you submit your primary platform, Ludum Dare allows you around 48 hours to compile for Windows, since the majority of reviewers use Windows.

The ideal submission is a single binary file (an .exe file for Windows) that doesn’t have to extract a lot of data, so that it’s easy for people to download and run your game.

PyInstaller vs. Py2exe vs. Py2app

I went on a wild goose chase trying to find out how to make a single executable file out of a Python project that would include all of my data assets. I first tried py2exe and py2app. py2app mostly worked all right, but py2exe was a pretty big mess.

The end story is that PyInstaller is newer and shinier than py2exe, and that you need to secret sauce code that someone out there on the Internet found before I did. PyInstaller basically runs EXE files by extracting the assets into a temporary data file that has a path _MEIPASS in it ((technical details here). Be sure that you check that every file is loaded in through that wrapper. The Tree() TOC syntax was also confusing, but basically, it’s the relative path of your data files and it will automatically load all of the files in that directory. Make sure it exists in the EXE portion (Windows) or the APP portion (Mac).

There’s a Make/Build cycle in PyInstaller to generate the spec file and build it in a single step as well – I find it easier to do that to generate the spec file and do an initial binary run, then to modify the spec and run PyInstaller again with the spec file as the argument. PyInstaller is pretty smart about rebuilding, and you save a lot of time.

I think in the long run, if you compare py2exe, py2app, and PyInstaller, PyInstaller is the program worth learning. It did have a pretty sharp curve for me – it didn’t help that I was trying to do this late at night after a challenging weekend!

If you do wish to use py2app to build your Mac OS X application bundle, then do keep in mind that you need to have a import pygame._view because of some kind of obscure issue.

Anyway, that’s all there is to this post for now.

Appendix

Here’s the setup.py I used for py2app.

from setuptools import setup

APP = ['painterscat.py']
DATA_FILES = ['data']

OPTIONS = {
    "argv_emulation": False,
    "compressed" : True,
    "optimize":2, 
    # "iconfile":'data/game.icns',        
}

setup(
    app=APP,
    data_files=DATA_FILES,
    options={'py2app': OPTIONS},
)

Creating CHI Video Previews on the Cheap

2 Replies

Our submission to the ACM Conference on Human Factors of Computing was accepted a while ago. As part of that submission, we were also required to create a video preview for the conference.

This presented to us a few logistical problems. First, none of us had camcorders. We had iPhones, which can record video in a pinch, but that can be rather spotty as far as recording goes. Second, we did not have a lot of time in which to arrange to do principal photography or other setups that would require fieldwork. The CHI video is not particularly long – under one minute – but it still requires us to know what we’re shooting, and to ensure that we can do this all in a clean, professional manner. When you have a week, and when you’re also considering the camera-ready version of a paper, trying to direct a video is a lot of work on top of that.

We decided to take a simpler route and use an animated video instead. This presented its own suite of problems. The first one was that no one on our team had computer-based animation experience in Flash or other art tools. Thus, it was up to us to figure out how we could do this nicely with inexpensive, off-the-shelf software.

First, here’s the video…!

And here’s the preprint of the paper! D. Piorkowski, S. D. Fleming, I. Kwan, M. Burnett, C. Scaffidi, R. Bellamy, J. Jordhal. The Whats and Hows of Programmers’ Foraging Diets, ACM Conference on Human-Computer Interaction (CHI), Paris, France, 2013.

Getting a decent video preview together using inexpensive software

If you want to make an animated video and don’t know other animation tools, you can build something reasonable in Apple Keynote and iMovie! With some high-resolution graphics, transitions and Keynote actions, good builds, and a high-resolution export, we were able to create something that, while it’s not going to win us any awards, serves as a presentable video preview.

I used Apple Keynote 5 because it exports high-frame Quicktime movies. I tried to use PowerPoint on various systems, but it exported movies that were not at a high-enough framerate to animate transitions properly.

The main workflow works like this:

  1. Use Keynote to create an animated presentation, then export it to a MOV file.
  2. Record yourself talking about the slides using a headset and Audacity
  3. Use iMovie (included for free in all Macs) to match the talking to the slides

It should be noted that if you do happen to know how to use a professional animation and video tool like Adobe Flash, Adobe Premiere, Apple FInal Cut Pro, etc., you should use those tools instead! This guide is meant for people who don’t have time to learn a real animation suite or don’t have something to field or screen-record.

When you create your presentation

We decided to use an animated figure doing some actions, some text-based transitions, and a voiceover. These won’t get us the “CHI video preview of the year”, but they’ll communicate the idea across! David, in this case, drew a few initial pictures that I used for the basis of the video.

He gave me static images, so the first thing I did was use an image editor to cut out the hands to “animate” them. I believe that’s the extent of the actual hand-drawn animation in the video. The other main animation that you see is the IFT “magnifying glass”, which is animated using Keynote’s build effects.

Here are some tips:

  • Use the effects for moving, building in, and building out liberally. These make your presentation look like an actual video with movement instead of just a set of still slides. Make sure you set them to fire automatically and after an appropriate delay. Here’s some ideas:
    • Swap two images using a really fast (0.1 second) Dissolve effect to get minor animations going.
    • Use a PNG with a partly-transparent background that moves in front of the scene – this is great for magnifying glasses, speech bubbles, and so forth.
    • Use actions such as “move” and “rotate” at the same time to add more dynamic movements.
  • Be sure that the slides advance automatically. Put a lot of “time” between your slides. Even after exporting the times won’t be accurate, they’ll be off by a few fractions of a second, and that’s enough to mess things up. You’re going to need to fix these manually in iMovie.
  • Break up everything into separate slides. If you can afford to have something not animated on the screen (good for static builds), then you can take a still screenshot and insert that into iMovie – it should seamlessly transition.

Even if you get all of your timings right within the presentation itself, the Quicktime movie that Keynote exports will still not have good timings. This is simply due to limitations with frames per second and how fast you can get transitions to fire in Keynote. You’ll want to edit the resulting file in iMovie.

Editing the Audio

My colleague David recorded the audio. I asked that he use a headset and that he record in a quiet room. Doing so will save you a lot of headaches! If possible, you want to record in a reasonably small room without any noises (closets and bathrooms work great in an emergency!). I don’t know how long it took him to record the video, but it was a reasonably clean recording.

I used Audacity, an open-source program, for my audio editing. I personally sliced up the audio into separate sentences, it made it easier to match the beginning of each sentence to the actual transition. I also took the time to edit out any “ums” or “ers” that were in the reading. His voiceover was pretty good overall but I was able to fine-tune it.

Once that is done, you have to match up the audio with the video in iMovie. This is a reasonably tedious, manual process. iMovie is accurate to something like a tenth of a second, and likes to nudge things to the closest 0.4 seconds, so you do want your video to be liberal enough to not go up against these limitations. Use the “clip adjustments” and “audio adjustments” settings. If necessary, slice up the audio into smaller segments and drag and drop them into iMovie at the right points to fine-tune.

Music

I added music to my video. Be aware of licensing arrangements! When you submit a video to the ACM, they require you to have proof of copyright usage of any media, which includes the video and music. In my case, I asked permission from the original author. (The track, by the way, is called World of Snow by DDRKirby(ISQ)).

One thing you don’t want is for your music to drown out your speaker. iMovie almost automates this process, though! There’s an iMovie feature called “ducking” that reduces the volume of the music while someone is talking – this is a common radio technique. For all of your voiceover tracks, set them to “duck other audio tracks” to a small value, like 8%. If you happen to have any gaps in your voiceover, you may need to insert some silence so that your audio doesn’t duck in and out while your speaker pauses.

In Conclusion

I am by no means a professional video editor. No one on our paper-writing team was! That was what led us down this path to creating a short video on the cheap, but the reality is that a lot of people out there aren’t professionals, and many had not ever edited a video clip or tried creating a movie. I think this just underlines the fact that researchers in any discipline have to have a large number of diverse skills, and require the ability to be resourceful and adaptable. We couldn’t afford the time or energy to go shooting video or learning video-editing software, but we also wanted something that was reasonably engaging and that would stand on its own!

I hope that some of the content on this particular post will help some others out there who are working on brief video clips or previews of their own content, be it research papers or other work. Enjoy!

Reading and Writing just got easier with Markdown

Leave a reply

I need to quickly take notes and share them as part of my job. It may be keeping a log during meetings, recording field notes during an observation session, or just doing preliminary notes for a web page I’m posting. However, a lot of the existing tools on computers have limitations.

  • Plain text is easy to write and generally easy to read, but there’s a lack of formatting. You can’t identify headings, create tables, insert images, or easily create hyperlinks. You can keep that information using some kind of markup language…. like HTML.
  • HTML is standard. It’s all over the place, you can format your pages nicely, and it does a reasonably good job at separating information from presentation, especially if you stick with the vanilla tags. Everyone can view HTML on their browser as well, so it’s very potable. Unfortunately, HTML is also really verbose and difficult to write. You can’t easily add linebreaks or headings. Miss a close tag and you’re toast. You have to type a LOT. You can ease this by using a WYSIWYG editor, but then your HTML markup becomes very ugly, and you’re committed to using that editor.
  • LaTeX is also very flexible but is mostly limited to PDF output (yes, you can output to HTML, but that requires a non-trivial amount of commitment to installation and setup).
  • Microsoft Word is easy to use, but is not portable to systems without Word. It exports to HTML but also has reasonably ugly HTML output. It also takes a really long time to launch and is difficult to check into a version control system.

So what is the solution? Well, I’ve discovered Markdown a few months ago, which is a lightweight markup language that is readable as plain text, but allows easy conversion to outputs like HTML. So what this means is that you can write your documents in a plain text editor, read them in a plain text editor, but also convert them easily into a web-viewable format. I’m beginning to take notes and make initial drafts of documents in Markdown because you can easily go from Markdown to almost anything else by either copy-and-pasting it, or by converting it to your target output format.

On Mac OS X, I use a command-line tool called MultiMarkdown by Fletcher Penny which is easily installable if you have Homebrew (http://mxcl.github.com/homebrew/). Then, you can either convert your documents using the command line, or use one of many text editors set up to compile your plain text into something else, like HTML.

I personally use the TextMate bundle that’s supplied by the author of MultiMarkdown. I can easily output my text file as HTML with a keyboard command, and then I can move that onto a website or similar if needed. Just today, I set up TextMate to write a timestamp when I press “ll<tab>” (that’s two lowercase letter Ls followed by a tab) so that I can easily take notes during a user study. Now, I don’t have to watch the time and try to write it in. And, since it’s in Markdown, the time shows up as bold text underneath some nice headers. If I want, I can copy and paste my original notes into Excel, post them online, or just look at them in the plain text editor without seeing all of the ugly HTML associated with it.

I’m not sure what the Windows equivalent is, but there are a lot of Markdown text editors out there and even Javascript client-side applications that allow you to write Markdown and show you a real-time preview of the Markdown in HTML! That is pretty nifty.

There are a lot of Markdown tutorials on the web, but here’s a brief command reference just to emphasize its simplicity.

Writing Headings

# This is a Level 1 heading.

## This is a Level 2 heading.

### This is a Level 3 heading. 

You can also Underline your headings.
===============================

Creating lists

* List item
* List item
    * Nested List Item
1. Numbered item
2. Numbered item

Links

Here is some [text of the link](http://example.com "Example")

You could also use reference-style [labels][1].

[1]: http://example.com "Example"

Images

![alt text](/path/to/image.jpg "Title")

Code spans

This is a Unix command: `find . -type d -exec chmod 755 {} \;`

You can also include larger code blocks by indenting code with 4 spaces or 1 tab.

Bold/Italics

**Bold text** __Also bold text__ *Italic text* _Also Italic text_

MultiMarkdown also has syntax for tables and equations that I haven’t yet had a need for (and therefore haven’t learned), but I suspect that could come in handy as well.

In any case, I am feeling already that Markdown is going to make my life easier because it’s lightweight, you can write and read it anywhere, and it is easily transformed to useful output formats. I like to keep my thinking about formatting separate from content, and this enables me to do so in a lightweight way. While LaTeX is nice for print documents and Word is good for collaborating with others, having a simple system for quick notes is really convenient as well.

Transcoding video for import into iMovie

Leave a reply

I’ve been getting a bit into the habit of taking camcorder footage of things that I go to now, editing them in iMovie’09, and then posting them on Facebook and YouTube.

However, I’ve had some issues trying to transfer video from an HD camera I’ve borrowed, the JVC Everio GZ-HD7U. It’s a handy camera with a large hard drive and a decently-long lasting battery, but it’s not very compatible with iMovie — if you connect it to a Mac and try to import the movies, it simply doesn’t let you import them directly.

I’ve tried a few things, mostly transcoding with ffmpegX, but every time I did so it resulted in really bad motion artifacts that band across my movie. Today, I finally discovered a good compromise and the solution is actually much easier than I would have thought.

The short answer:

ffmpeg -i infile.mpg -target ntsc-dv outfile.dv

The expanded answer is that even though the camera records internally as MPEG-TS, it’s in some kind of compressed format and needs to be transcoded. The way that I do this is:

  • Plug the camera into my computer using the USB Cable.
  • Turn on the camera.
  • Using the camera’s options, select “Back up onto PC”. The camera will mount itself as a drive on the computer. The drive layout will probably vary based on what you’ve been recording, but in general it’s laid out with a bunch of directories named PRG000, PRG0001 and so on.
  • Within these directories, copy all of the .TOD files onto your hard drive.
  • Once the files are on your hard drive, export them to DV format using the ffmpeg command: ffmpeg -i infile.mpg -target ntsc-dv outfile.dv

I use the following shell script to do this.

#!/bin/sh

suffix=.TOD
files=`find . -name "*.TOD"`

for movie in $files; do

infile=`basename $movie`
outfile=`basename -s $suffix $infile`.dv
ffmpeg -i $infile -target ntsc-dv $outfile

done

If you don’t have ffmpeg installed on your Mac, you can do it with MacPorts (type sudo port install ffmpeg and ports should do the rest*).

I hope this helps some of you!

NOTE:
Using this method converts stuff into standard NTSC resolution (apparently, I don’t know enough about video transcoding). Now, the “original” movies are filmed at 1920×1080 (HD 1080) resolution, but if I copy the TOD files onto my computer and play them, I get nice funny artifacts. This leads me to suspect that the camcorder’s compression kind of plays tricks on playback or something – makes it seem like they’re HD1080 when the files are in fact not even close to that. So the issue of course is that when you use the DV output format with ffmpeg, you end up scaling your movies down to 720×480. The odd thing for me is that the movies are still widescreen. I don’t know how this works or if some programs are fooled and all I truly do know is that the movies look better. But, this leads me to believe that if you buy a camcorder that says “full HD” it might actually not be full HD even if it claims that it can film in 1920×1080 resolution.

Just my two cents, if anyone can help fill me in on video formats please let me know.

* I did have a problem with installing from ports, and it was due to an outdated gettext Perl module. What I ended up needing to do was to delete all of my ports and recompile. I am suspecting the issue was because I installed all of my ports on Leopard, and upgrading to Snow Leopard without recompiling all of the ports caused problems.