Wednesday, November 26, 2008

I really don't like remote debugging

Last week I sent out a notice to the v4l mailing list saying I would debug Linux support for any em28xx based devices for the cost of shipping the device to me for a week. I restricted it to ATSC/NTSC based devices since I don't have ready access to DVB and PAL. I also restricted it to users in the US, to avoid the customs issues.

Well, I got quite a few responses, but of them:
  • All of them were out of the country
  • *Almost* all of them were for DVB and PAL
  • The one user reporting an issue with an NTSC device was in Canada.
So now I'm in a difficult position - do I attempt to debug and get working a bunch of devices I don't have physical access to? Even dealing with experienced linux-dvb/v4l developers takes an extraordinary amount of time. Dealing with end-users is an outright nightmare.

No disrespect to any users I'm dealing with who might have just read the above sentence.

Obviously, I can't afford to go out and buy all these devices. Kworld, Artec, Asus, Dvico - Even at the low cost of $40-$50 each, there are just too many of them. Not to mention that I only need them for a relatively short period of time to get them working (a few days) and then they join the pile of tuners I already have which I don't need.

On the upside, the em28xx driver in general is continuing to improve. Things are getting cleaned up, edge cases are being ironed out.

Tuesday, November 25, 2008

Tuesday

Two more patches got merged into the v4l-dvb mainline today:

em28xx: cleanup XCLK register usage
9744 http://linuxtv.org/hg/v4l-dvb/rev/0ad0eccb9e51
em28xx: Cleanup GPIO/GPO setup code
9745 http://linuxtv.org/hg/v4l-dvb/rev/d80d3da52f90

Watched House.

Spent pretty much the whole evening debugging my xc5000 patches with mkrufky over AIM. Fixed the cx88 race condition we found on Sunday. There's still some issue with that XREG_BUSY register though, and changing the timing from 100ms to 5ms broke analog support on the Dvico FusionHDTV 7. It's *so close* to working though.

That's turning into a descent size patch series (10 so far).

Pinnacle was kind enough to introduce me to someone over at Xceive who I am talking to next week. I'm hopeful that we can work something out with the firmware redistribution rights for xc5000, xc3028, and xc3028L. This will make it *so* much easier for Linux users as the firmware would be bundled with the distributions. They would be able to just plug in their device and start watching TV.

Monday, November 24, 2008

Monday

I shouldn't go into it in a public forum, but let's just say I had a really crappy day.

Submitted three more patches tonight. Watched a little TV. Tried to get caught up a little on my backlog.

Yeah, not much else to say today.

Confidential email

I often see blocks of text at the bottom of emails such as the following:
This e-mail and any attachments are confidential and may be subject to legal or some other professional privilege. They are intended solely for the attention and use of the named addressee(s). If you are not the named addressee(s) you must not use, disclose, retain or reproduce all or any part of the information contained in this e-mail or any attachments. Any unauthorized use or disclosure may be unlawful. If you have received this e-mail by mistake, please inform the sender immediately and delete it and all copies from your system and destroy any hard copies of it.

This makes me wonder: If you *do* accidentally transmit my confidential information to the wrong party, can I sue you for gross negligence? You are knowingly transmitting confidential information across an insecure network and hoping that a little block of text at the footer of the email is going to relieve you of any responsibility if it falls into the wrong hands?

Suggesting it "may be unlawful" for me to retransmit the stuff you sent me is especially egregious.

Even worse is when I see morons post to public mailing lists with such text at the bottom of their emails. After all, a Gmail or Yahoo account is *so* expensive.

P.S. - for the record, the above example email text was taken from such an email posted to a public forum, and any assertion of copyright would be countered with a textbook example of fair use doctrine.

Sunday, November 23, 2008

End of the weekend...

Had a nice weekend. Victoria's sister and sister's boyfriend came out to the City on Friday night. Had a nice dinner. Yesterday was pretty quiet. Took Victoria's laptop to the Apple Store while she went and saw her friend in Brooklyn. We stayed in and made dinner (tomato soup and salad).

Today, we played some Scrabble. Took it easy most of the day. After Victoria left I went out to mkrufky's house and tried to debug the xc5000 problems he reported. He's merging my series of cleanups and performance improvements, and it looks like there was a merge bug in addition to uncovering a bug in the cx88's state machine that resulted in a race condition.

I guess I should have known that doing proper power management would uncover all sorts of fun bugs in surrounding drivers.

My backlog of em28xx stuff is growing as people have been reporting their problems and sending me debugging information. I really need to get that situation under control. Also continuing to do cleanup and fix bugs with Mauro and Douglas, which is pretty time consuming.

Thursday, November 20, 2008

Almost Friday

Got a few more patches into the v4l-dvb mainline over the last couple of days:

9644 http://linuxtv.org/hg/v4l-dvb/rev/714aedd7fc6d
9648 http://linuxtv.org/hg/v4l-dvb/rev/33eeb5e389d9
9657 http://linuxtv.org/hg/v4l-dvb/rev/1e51fd09353a
9658 http://linuxtv.org/hg/v4l-dvb/rev/09ffe8162ba1
9674 http://linuxtv.org/hg/v4l-dvb/rev/9470671cf3d8

Still working various audio issues with Mauro and Douglas. Also, a few people have reached out to me in response to the announcement I made on the video4linux mailing list soliciting people with em28xx problems, so I am starting to work through those.

Victoria is coming to New York tomorrow. Yay! Haven't seen her in two weeks.

Sunday, November 16, 2008

Productive week/weekend

I actually got quite a bit done this week. More stuff got pushed into the Linux-dvb mainline today:

9628 http://linuxtv.org/hg/v4l-dvb/rev/4f696c1de931
9629 http://linuxtv.org/hg/v4l-dvb/rev/ed1b7332fd9b
9631 http://linuxtv.org/hg/v4l-dvb/rev/5fc27f83d5a7
9632 http://linuxtv.org/hg/v4l-dvb/rev/7e6a049bc31b
9633 http://linuxtv.org/hg/v4l-dvb/rev/5fba1d2a1ed2
9634 http://linuxtv.org/hg/v4l-dvb/rev/f7bf1c4482c4
9639 http://linuxtv.org/hg/v4l-dvb/rev/561b447ade77

This afternoon was pretty quiet. Debugged a bit of code. Went for a walk. Browsed around at the Strand. Nice and relaxing. Going to make some ravioli for dinner.

Saturday, November 15, 2008

Coding weekend

With Victoria in Florida this weekend, I spent the entire day writing code. Well, not the entire day: I did go out for a bagel and cup of coffee in the morning.

Submitted four patches to linux-dvb today, and supported mkrufky as he tried to parse through my batch of proposed xc5000 patches. I did all the original work with the Pinnacle 801e. He gave me a HVR-950q a couple of weeks ago, and I didn't get around to testing it. Turns out it exposed two bugs in my series of nine patches. Crap.

I hate looking like an idiot.

On the bright side, I did fix the audio on the HVR-950, found a power management bug in the dvb framework, resubmitted my fix for the s2api being completely broken for ATSC, and found another power management issue in s5h1411.

I also started to look at the the analog video quality in em28xx, using a pristine video source (in this case four episodes from season 3 of the Office on DVD). I feel like there is definitely something wrong, but I can't put my finger on whether it's some sort of scaling problem, the interlacing, or something else. Perhaps I will do a dump against the Windows driver and compare how the registers are setup.

Man Submits Drawing Of Spider Instead Of Payment For Overdue Account

This is a pretty entertaining read:

http://www.geekologie.com/2008/11/good_idea_man_submits_drawing.php

Don't worry, it's pretty short.

Thursday, November 13, 2008

Thursday

Wrote a keymap for the ATI TV Wonder 600 USB's remote control, and submitted a patch tonight. Haven't heard back from Mauro about my em28xx IR patch yet.

Disassembled my Pinnacle PCTV HD Ultimate. I'm still waiting for the datasheets from ATI and NXP before I can really get started on the driver code.

Patrick got back to me about the dib0700's IR changes in firmware 1.20, which I tried to implement tonight and failed miserably. The data just doesn't match how he's telling me it's supposed to work. :-(

Watched the Office and ER (which I probably haven't watched in five or six years).

Victoria is in Orlando at a conference.

Wednesday, November 12, 2008

Night of IR

In response to some discussion with Mauro, I finally broke down and took a serious look at the infrared remote control support in the em28xx driver. I've known for a while that it didn't work with any of the devices I added support for, but just hadn't gotten around to doing anything about it.

Tonight, I did a bunch of refactoring/cleanup, including adding support for the new em2874's IR registers.

Successfully tested the remotes to my HVR-950, Pinnacle 800e, and Pinnacle 80e. I've still got my AMD TV Wonder 600 USB to do, but I need to write a keymap for the remote control first. I've also got a Pinnacle HD Ultimate Stick that uses the em2882, but I haven't done the other driver work yet.

Tuesday, November 11, 2008

em2874 support merged

The em2874 support got checked in to the mainline tonight:

9580 http://linuxtv.org/hg/v4l-dvb/rev/e94926cdc504
9581 http://linuxtv.org/hg/v4l-dvb/rev/82020a39a85b
9582 http://linuxtv.org/hg/v4l-dvb/rev/07e1799676fd
9583 http://linuxtv.org/hg/v4l-dvb/rev/535830ac72ae
9584 http://linuxtv.org/hg/v4l-dvb/rev/6941f46ad815
9585 http://linuxtv.org/hg/v4l-dvb/rev/3c9853f2e924
9586 http://linuxtv.org/hg/v4l-dvb/rev/17c8a84501d9
9587 http://linuxtv.org/hg/v4l-dvb/rev/4fb54f171511
9588 http://linuxtv.org/hg/v4l-dvb/rev/dc22320bb695
9589 http://linuxtv.org/hg/v4l-dvb/rev/9529469979f0
9590 http://linuxtv.org/hg/v4l-dvb/rev/3b840c0a6975

Now just need to finish getting the Micronas drx-j licensing straightened out and the Pinnacle 80e will be fully supported.

Monday, November 10, 2008

Pinnacle 80e merge prep

Committed a couple of fixes into Kaffeine tonight for ATSC scanning support.

http://websvn.kde.org/?view=rev&revision=882635
http://websvn.kde.org/?view=rev&revision=882634

While I'm waiting to get the licensing straightened out with Micronas, started breaking up my em28xx changes into a quilt series. I've got ten patches so far and I haven't gotten to the Pinnacle 80e specific stuff yet.

Looks like my s5h1411 tuning and power management fixes went into 2.6.27 stable a couple of days ago. Now if I can just get the xc5000 to be taken upstream then I'll be home free.

Wednesday, November 5, 2008

I've decided: not everyone at the FCC is a moron...

You know all that crap about the cable company telling you that you had to get a cable box to continue getting the channels you are paying for? Well, apparently it finally caught the FCC's attention.

http://arstechnica.com/news.ars/post/20081105-fcc-launches-probe-into-possible-cable-pricing-shenanigans.html

Also, it seems like the FCC is onto that crap the cable companies are telling people about how they're being forced to do it by the FCC as a result of the "DTV transition".

Jury Duty: Part Deux

In my second day of jury duty. The upside is that if I make it through today then I am done.

Spent most of last night investigating Kaffeine's buffering model, since I am trying to figure out why the video is so crappy. A few obvious missing exception handling cases aside, it looks like this if a more fundamental problem.

Kaffeine has three threads of execution that are relevant here:
  • Thread 1 reads the /dev/dvb/adapter0/dvr0 device file and inserts the MPEG packets into a local buffer
  • Thread 2 reads the local buffer and pushes the data into a pipe (and optionally writes it to disk to)
  • Thread 3 is the instance of Xine that reads the pipe and renders the stream to the display
Note that in the main data path, if you are watching live tv, the data is never written to the disk. So if the system stalls for even a short period, you will lose packets.

After adding a bunch of comments, I can see two basic failure cases:
  1. In some cases the thread reading the device file gets overflow errors from the kernel, indicating that it is not reading the device file fast enough
  2. In some cases, the buffer pool populated by the device thread fills up before the thread that services the pipe can read it out. As a result, the packets are not inserted into the buffer.

Obviously, making the buffer pool bigger might reduce the risk of overflow, but the first condition above is more troubling. It suggests that a thread whose sole responsibility is to read the device file cannot keep up with the data flow.

The thread in question actually works as follows:
Poll call on device fd
if (data available)
{
read(fd, buf, 188)
copy data from buf to memory buffer of size (188x8)
if (memory buffer full) {
announce (8x188) bytes to second thread
}
}

Now, if you do the math, you might appreciate why this might be a bit disturbing. If you do one read call per mpeg packet, on a 19Mb stream that amounts to almost 13,000 read() calls per second. Of course there can be multiple elementary streams within that, so it might only be around 6,000 read() calls per second. That's still a hell of a lot of read calls.

It's also possible that there is some other sleep() call or some equivalent that stalls the pipeline, but I haven't found one yet.

Monday, November 3, 2008

Coffee

Did I mention there is no food or drink allowed in the Jury room, including *coffee*? For those of you who know me, I don't function well without a continuous stream of coffee throughout my day.

"True Administration of Justice is the Firmest Pillar of Good Government"

... At least that's what it says on the outside of the building.

Yeah, so I'm stuck in Jury duty. It feels like it was just yesterday that I was doing this in New Jersey, except it was actually more than four years ago.
  • Laptops? - Encouraged
  • Wifi? - Provided
  • Electrical outlets? Not a single one to be found

I am at lunch right now. I got sucked into a Jury selection pool where I'm #4. This would suggest a good chance of the next seven or eight days sucking if I get selected. It also means I don't get to sit around and play with my laptop, since I actually have to pay attention.

Oh yeah, and we only get one channel on the TV: Fox News.

Ugh.

Sunday, November 2, 2008

Weekend

Spent the weekend in Philadelphia with Victoria. Went to a couple of Halloween parties. Had a nice time.

Came home to find that my Pinnacle 808e Ultimate sample unit had arrived in the mail. Yay for free hardware.

For those of you who might be waiting on a Linux driver for the Pinnacle 80e Mini, I'm still working on the legal issues with Micronas. Hopefully will be straightened out in the next week or so.

Empia sent me the datasheet last night for the em2880, which I am currently resolving the differences against the em2874 datasheet so I can do a bit of driver cleanup.

Did some work tonight to prep my em2874 changes to merge, and managed to accidentally trash out the first byte in the eeprom of my HVR-950. Doh. Fortunately, since it was only the first byte and the device still loaded, it was pretty easy to reprogram back to the original value.

On Thursday night I also found a couple of nasty bugs in Kaffeine that were causing stream corruption, blocking and pixelation. I've got a few preliminary patches but they're not ready to push upstream yet. There's still one nasty bug that I want to isolate before I submit them.