Saturday, April 26, 2008

HVR-950 Todo list

So basically the driver works, and Mauro submitted a Git pull to Linus for the changes. Wohoo. However, there is still work to be done. Here is a list of things I still need to do:

  • IR support - it looks like most of the components are there already (tveeprom support, IR i2c commands, etc). I will need to setup the driver for inclusion in the em28xx driver properly and debug/test.

  • Analog/digital tuner locking - Because the HVR-950 is a hybrid device that supports both analog and digital reception, I need to work out some locking between DVB and V4L to handle cases where one tries to attach while the other is in use. Right now, there is no locking and connecting while the device is in use results in breaking the currently running stream.

  • S-Video/composite testing - I need to test/debug input switching and signal reception for the other ports.

  • Power management - Right now, the device never goes into low power mode from the time you connect it to the USB. Under Windows or Mac, the device is in low power mode until you startup your video application. While this doesn't affect MythTV users, who pretty much always have the backend connected, it would be helpful for more casual users who use it to record audio/video or watch live TV with a standalone application. Besides, the device runs hot already, so it's probably for the best that we power it down when it's not in use.

Aside from the actual device support, I want to get Kaffeine working with ATSC in general. It's annoying there is no standalone application for ATSC reception (Kaffeine supports DVB only). Shouldn't be too hard, but the project doesn't look very active. It would also be nice to have a real test application that does tuning so I don't have to combine mplayer with the command line azap tool.

Centrifugal Force

Friday, April 25, 2008

Debugging fun Solaris Perl crash

Can you spot what's wrong with this picture (truss output of Perl on Solaris)?

/10: llseek(12, 0, SEEK_CUR) = 180224
/10: llseek(12, 0, SEEK_CUR) = 180224
/10: brk(0x0281D3F0) = 0
/10: brk(0x0281F3F0) = 0
/10: read(12, " $ d { " b u s i n e s s".., 8192) = 8192
/10: llseek(12, 0, SEEK_CUR) = 188416
/10: llseek(12, 0, SEEK_CUR) = 188416
/10: brk(0x0281F3F0) = 0
/10: brk(0x028213F0) = 0
/10: llseek(12, 0, SEEK_CUR) = 188416
/10: llseek(12, 0, SEEK_CUR) = 188416
/10: llseek(12, 0, SEEK_CUR) = 188416
/10: brk(0x028213F0) = 0
/10: brk(0x028233F0) = 0
/10: llseek(12, 0, SEEK_CUR) = 188416
/10: brk(0x028233F0) = 0
/10: brk(0x028253F0) = 0
/10: llseek(12, 0, SEEK_CUR) = 185518
/10: brk(0x028253F0) = 0
/10: brk(0x028273F0) = 0
/10: llseek(12, 0, SEEK_CUR) = 185518
/10: llseek(12, 0, SEEK_CUR) = 185518
Here's a hint:

llseek man page
http://docs.sun.com/app/docs/doc/805-8005-03/6j7hcvh5p?l=ru&a=view

Wednesday, April 23, 2008

Putting MythTV's News service to good use

I finally got my MythTV box setup. What better way to take advantage of the customized news than to subscribe to the company's announcement page for "Fruity People" (people who pay 3 dollars a week for Eric to bring in fruit to enjoy during working hours).



This way when I wake up in the morning and turn on the TV, I immediately know whether or not to pick up a muffin on my way into the office!

Sunday, April 20, 2008

Sunday

Victoria came out to New York Friday night. We had a nice lunch, picked up some new sneakers (animal free!), and took a walk through Little Italy. I like Spring...

Got some work done. Trying to get caught up on some stuff that I haven't been able to find a block of time to concentrate on when people aren't interrupting me. Interruptions are part of my job, but it does make it hard to focus sometimes.

Also cleaned up some issues with my HVR-950 driver. Installed the latest MythTV packages for Ubuntu today and did some testing. Works pretty good, but I think there's a locking issue in my driver when going between analog and digital mode. Also, the infrared receiver doesn't work yet, so I can't use my remote control. I started to look at the IR code, but it looks like the device doesn't say it has an IR receiver like other Hauppauge products (there's usually a field in the EEPROM). Will hopefully figure it out over the next few nights this week... There's also some issue with ATSC scanning where the scanning itself works fine but the signal strength and SNR are not reported properly in the MythTV GUI.

I also discovered that apparently Nvidia dropped support for XvMC starting in their GeForce 8xxx series. So with no MPEG2 hardware acceleration during decoding, the CPU is pretty heavily taxed rendering 1080i HDTV which is annoying.

I'm thinking about doing some work on Kaffeine now. It's really surprising that I couldn't find a test application that supports both viewing a stream *and* ATSC tuning. People use Kaffeine with DVB but it doesn't support ATSC, or they use run azap from the command line and then run mplayer to view the stream.

Saturday, April 19, 2008

Fun with routers...

A couple of months ago I spent an evening playing with Transmission (the BitTorrent client for Mac), and I had made a number of changes to my home network in order to get it working. This included updating the firmware in my router, putting my DSL modem into bridging mode, enabling uPNP, disabling the built-in firewall on my Macbook. And the situation got a little better in that uPNP was working, but a couple of days ago I realized I couldn't successfully do an Mercurial update of the source code from www.linuxtv.org without the tool hanging indefinitely in mid download. In fact, I discovered that I couldn't even download the tarball from there without it hanging in Firefox. I confirmed it was not the linuxtv.org website by "borrowing" some wireless from my neighbor, and confirmed it was not any particular workstation since it happened both on my Linux desktop and my Mac laptop.

Tonight, I had had enough, so I sat down and started to troubleshoot the problem. If you don't enjoy fun stores of network troubleshooting, you might as well stop reading.

My basic setup is I have a Westell 6100 DSL modem, which has a built-in NAT and firewall, and I am connected to that with my Belkin F5D7230-4 wireless router, behind which sits my various PCs. I started by looking at the logs in the Belkin router since that is the closest to the PCs. Here's what I found:



Hmmm.... For some reason the router is renewing it's DHCP address very frequently. In fact, it does it exactly every 30 seconds. And why does it think the DHCP lease is only 300 seconds? AND, why does it do a DHCP release followed by a renew, instead of just a DHCP renew? I logged in to the DSL modem and confirmed the internal DHCP server is set to issue leases of 1 day (51000 seconds).

Well, let's start by seeing if there is any new firmware. I updated the firmware two months ago to a release from July, so I guess it's possible that there's new firmware. In fact, release 9.0.10 came out in November, and guess what the release notes say in regards to the fixes:



Yeah, duh. Ok. So I updated the firmware, and while it's not renewing every 30 seconds, it now renews every 46 seconds. I guess we're moving in the right direction at least.



Now I'm starting to wonder about that 300 second lease time, and those of you familiar with the RFC 2131 would know that the "lease time" is actually a worst case before the address is issued to another party, and that client implementations are actually supposed to renew more frequently. Section 4.4.5 recommends a renewal time of (lease * 0.5), but implementations may vary. So, I break out an old 10Mb hub, stick it between the DSL modem and the router, and turn on Wireshark:



Yup, the router really is doing a release/renew, and the server really is issuing a 300 second lease.

From my time doing development on 3Com's routers, I have seen cases where routers that have both DHCP implementations and NAT in some cases will flush out the NAT table entries associated with a given IP if the client does a DHCP release. They do this to ensure that one PC doesn't get the NAT entries associated with a previous PC bound to that IP address. A side effect though is that if it is the same client then all of the state info is missing for sessions in progress.

Also, you might notice that the address issued isn't a private address on the 192.168.2.x network? It looks like a real public network address. In fact, it is. So back to the Westell DSL modem interface. It looks like even though I have DHCP configured for a 1-day lease, this doesn't apply to the DMZ. I had moved the router into the DMZ when I was playing with Transmission because I figured the NAT was flaky and I wanted to just use the modem as a bridge.

Turn off the DMZ, and like magic the lease time jumps back to 51000 and it stops renewing every 46 seconds. Mecurial can download source code. Firefox doesn't hang on downloads. All is well in the world. :-)

So why does the router do a release/renew instead of just a renew, causing the IP to be released and an interruption of service? Well, probably because it's stupid.

Why doesn't the Westell modem follow it's DHCP configuration with addresses in the DMZ like it does with private addresses? Well, probably because it's stupid.

In the end I found a known firmware bug in the router, a protocol violation in the router's DHCP, and what appears to be a configuration bug in the DSL modem's DHCP server. I was really expecting it to be one relatively simple problem.

New lobby furniture

So I came into my building last night and found that apparently we got some new furniture for the lobby.

To give some frame of reference, before we had a small loveseat and two dark covered chairs around a coffee table.

Here's what we have now:

This first picture is what you see when you enter the lobby:

Yes, that's a goose that doubles as a bowl for lemons. How functional! And notice the dog. That's not a real dog, of course, but one made of leather.


Yeah, don't those chairs look comfortable? Notice how low to the floor they are, if you're more than 48" tall, good luck getting up out of one of them.

Wednesday, April 16, 2008

The TURBO button

Did more work on my HVR-950 code tonight. Because of some broken patches I received from Mauro, I couldn't focus on figuring out why switching between digital and analog mode doesn't work. So instead I looked at why firmware loading took forever.

The Xceive 3028 is a programmable digital tuner, and one of the things you have to do is load microcode into the device. For some reason, it was taking 13 seconds to load the firmware, which is only about 60 KB. Bear in mind that this is a tuner chip that normally transfers data at around 19 Mb/s.

I thought perhaps there was some limitation on the speed of the i2c bus in the em28xx bridge, or perhaps there was some magic register you had to set to change the bus speed. In the end, the answer was MUCH simpler:

From: Devin Heitmueller

em28xx-core.c:
- Remove sleep in i2c message routine which slows down i2c by a factor
10x. Load time for BASE firmware went from 13s to .973s

Signed-off-by: Devin Heitmueller

diff -ur a/v4l/em28xx-core.c b/v4l/em28xx-core.c
--- a/v4l/em28xx-core.c 2008-04-16 00:34:32.000000000 -0400
+++ b/v4l/em28xx-core.c 2008-04-17 01:48:04.000000000 -0400
@@ -153,7 +153,6 @@
ret = usb_control_msg(dev->udev, usb_sndctrlpipe(dev->udev, 0), req,
USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
0x0000, reg, bufs, len, HZ);
- msleep(5); /* FIXME: magic number */
kfree(bufs);
return ret;
}

Yup, somebody stuck a 5 millisecond sleep into a low level function that sends commands onto the USB bus. While .005 seconds doesn't sound like much, try doing that around 2600 times, and you magically end up with around 13 seconds...

Fun with Perl....

Check out this gem Packy found today:

sub _app_run_args
{
my $runfile = shift;

my %server_args = do {
-r $runfile
or throw Error( "Un-readable version file: '$runfile'" );

-s _
or throw Error( "Empty version file: '$runfile'" );

# failing would be really wierd at at this point,
# but there is no telling...

open my $fh, '<', $runfile
or throw Error( "Failed open: '$runfile', $!" );

map
{
m{ ^\s* -(\w) \s* (\S+) \s+ }x
? ( $1 => $2 )
: ()
}
<$fh>
};

my %parms = ();

@parms{ qw( device log_path cfg_path app_dir dbname ) } =
@server_args{ qw( d e c M s ) };

return %parms;
}
If you're looking at this right now and thinking, "What the fuck?", well me too.

Let's look at some of the cute things here:

-s _
or throw Error( "Empty version file: '$runfile'" );

Nope, that underscore isn't a typo. Essentially this code says see if the previous file we were working on is greater than zero in size, and if not throw an error. But for readability's sake, why wouldn't you make that an if statement?

And how about this one:

map
{
m{ ^\s* -(\w) \s* (\S+) \s+ }x
? ( $1 => $2 )
: ()
}
<$fh>
Well, if you happened to know the file looked like the following, you could deduce that it's scanning over the contents of $fh and parsing out the arguments:

/sybase-stuff/sybase/ASE-12_5/bin/dataserver \
-d/sybase-stuff/sybase/master_device \
-e/sybase-stuff/sybase/ASE-12_5/install/SUNHAK.log \
-c/sybase-stuff/sybase/ASE-12_5/SUNHAK.cfg \
-M/sybase-stuff/sybase/ASE-12_5 \
-sSUNHAK \
Oh, and the results of the map are what is stored in %server_args, because that's the last statement in the block.

HVR-950 work continues

Spent the last few nights going back and forth with Mauro and Aidan on my HVR-950 work. While I did "get it to work", I now have to get the edge cases worked out relating to switching back and forth between analog and digital capture mode (which requires enabling/disabling certain chips and reprogramming the firmware). I'm making progress and finding/fixing bugs along the way, but it's harder than I thought it would be.

Since I was planning on doing some work on the Kaffeine Media Player next anyway, getting to know the V4L API better can only help me. It's actually pretty cool stuff.

It's actually pretty convenient that Mauro is two hours ahead of me in Brazil, since that means I have fresh patches to try out when I get home from work and we're not stepping all over each other.

Speaking of work, things have been a little crazy for me there recently. Trying to push through my bug queue and finish the major functionality I am responsible for. It's all good stuff, but there just aren't enough hours in the day...

Sunday, April 13, 2008

Weekend

Went home sick on Friday. Ick.

On Saturday, went to Philly to meet Victoria, and then we drove out to New Jersey for Jayden's 2nd birthday. He's such a cute kid, and I can't get over how smart. I don't think I've met a two year old who points out the window at the fence and says "The squirrel went under the gate."

Went out with Mom this morning to the Route 70 flea market. I haven't been there in over a decade, but it hasn't really change much. Lots of crap I don't need: knives, porn, old books, furniture, and more porn. After the flea market, we grabbed some lunch at the All Seasons and went to the movies to see 21. Not bad, but pretty predictable since I saw a documentary on the Discovery Channel about the MIT blackjack team a couple of years ago.

Got home late this afternoon, and worked through some issues over email with the V4L guys regarding my HVR-950 support. We've been working on a new unified driver that supports digital tv for various devices in the same family. Hoping to see it stabilize over the next couple of days. We missed the merge window for Linux kernel 2.6.25, but it should get in there for 2.6.26. Pretty cool.

Oh, and yesterday I bought a Pinnacle PVTV Pro HDTV because it was supposed to be an EmpiaTech based device and I wanted to try it with my new driver, and found out that Pinnacle came out with a brand new hardware design, but didn't change the product name. So there are now two totally different devices with the same name. I can't blame Pinnacle too much, since from an end user standpoint there isn't any difference, but it makes it harder to tell which driver you need because the packaging is identical (they have different USB IDs, but you can't find which it is until you have opened the box and plugged it in).

To make matters worse, the device doesn't work with the Elgato EyeTV software I use with my Mac either.

Saturday, April 12, 2008

Liberty Science Center trip

Last Saturday Victoria and I celebrated our anniversary by going to the recently reopened Liberty Science Center. Yeah, we're such nerds...



Here are the other pics:

http://pics.livejournal.com/devinjh/gallery/00015bw7

Thursday, April 10, 2008

Today's Perl cat test

Here is the Perl /bin/cat implementation for today's candidate:

open $file;
write $file, >;

Yeah, I don't think we're going to hire this one...

GDB debugger crashing...

Don't you just love it when you go to debug some crashing process and the debugger itself crashes?

bash-3.00# /usr/local/bin/gdb /foo/myprogram /foo/core
GNU gdb 6.0
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "sparc-sun-solaris2.9"...
Core was generated by `/foo/dagent'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libsocket.so.1...done.
Bus Error (core dumped)
bash-3.00#

At least it generated a core file.... Next stop:

bash-3.00# /usr/local/bin/gdb /usr/local/bin/gdb ./core

Monday, April 7, 2008

HVR-950 ATSC Linux support - SUCCESS!

Finally spent a few hours tonight to try to figure out why I couldn't get a signal lock on the xc3028 for my HVR-950 USB tuner. Turns up I had copied some code from the cx23885 driver which loaded the OREN538 scode into the device before tuning to the specified frequency. All I had to do was comment out that one line of code, and I started getting tuner lock.

ATSC scanning now works. Azap works. AND... I can now do "mplayer /dev/dvb/adapter0/dvr0" and I am able to view HDTV streams over ATSC.

Wohoo. About time.


(It may not be obvious from the screen capture above, but bear in mind my desktop is at 1920x1200).

I still have to do some cleanup, and the tuning is still pretty slow because of a bunch of debug code I still have in there. It's hard to imagine why a 50ms sleep after every i2c command might slow things down. :-)

On a separate note, I really should have gone to bed two hours ago.

Wednesday, April 2, 2008

#pragma pack() across multiple platforms

Been doing more AIX porting work. Found out how badly different compilers handle the #pragma pack directive, and I'm looking for a solution that works with all of them:

First, let's look at gcc:

#pragma pack(1)
...
#pragma pack()

Simple enough. I can set the packing to 1 byte, do some stuff, and then set it back to the previous value with an empty argument.

Under AIX, the xlC compiler throws a warning if you specify an empty value with the above code:

1540-1404 (W) The parameter "" specified for "pragma pack" is not valid. The pragma is ignored.

So under xlC, setting it back requires me use the "pop" keyword such as the following:

#pragma pack(1)
...
#pragma pack(pop)

It's treating it like a stack, which is fine. However, if you use this code under gcc, you get the following error:

warning: #pragma pack (pop) encountered without matching #pragma pack (push, <n>)

Hmmm... So, let's explicitly push the value on when I set change the packing in the first place:

#pragma pack(push, 1)
...
#pragma pack(pop)

Except that doesn't work because xlC doesn't recognize the push keyword. What kind of compiler knows about has a "pop" without a corresponding "push"? IBM's compiler, apparently

So in summary, xlC always treats it like a stack, GCC only treats it like a stack if you explicitly say "push", but xlC doesn't support the "push" keyword. Argh.

Here's gcc definition for #pragma pack()

http://gcc.gnu.org/onlinedocs/gcc/Structure_002dPacking-Pragmas.html

MSVC appears to support the same feature set as gcc, but it also supports optional identifiers.

http://msdn2.microsoft.com/en-us/library/2e70t5y1(VS.80).aspx

xLC actually claims to support "#pragma pack()" with no arguments in it's documentation (and in fact has it in one of it's examples, except it doesn't actually work..

I guess I'm going to have to do a #define for xlC's compiler

Tuesday, April 1, 2008

Interesting support ticket

Here is a rather odd support ticket filed against Fedora Core.

https://bugzilla.redhat.com/show_bug.cgi?id=439858

The fun part is when you happen to notice who reported the problem.

Design by committee

I've been in some discussions with people for the last day or two about the best architecture approach for some new functionality.

It's a frustrating experience to try to get some form of consensus among a group, especially when I wonder about the motivations of those involved. Are we really doing what's best for the product and customer, or are some parties just trying to do the least work possible to claim to have met the requirements?

The term "design by committee" immediately came to mind, which prompted me to wonder how Wikipedia defined the topic:

http://en.wikipedia.org/wiki/Design_by_committee

**grrrr***

Ah, the things that keep me from sleeping....