Conformance tests for drivers?

We have certain issues that are driving us mad, and it seems to be driver bugs at the root cause.

Does anyone know of any good conformance test programs for drivers?

We have 4 machines, 2 of them the exact same except one has a ATI Radeon 9600 PRO, and the other one has a Nvidia GF4 Ti 4200. The 9600 Pro machine crashes/locks up the machine, and the 4200 never locked up the machine. We have tried Cats 3.5-3.9 and we can’t seem to find a stable version that don’t crash/lockup. Before you ask, yes, we have swapped the cards between those 2 machines, and the 9600 still has these crash/lockup issues. Even tried a new 9600 with same results. Also note that we do NOT modify the drivers in any way, whatever ATI has them set for by default is what we use.

I have read around, and it seems that ATI’s openGL drivers are piss poor, people talking about other openGL games crashing and or locking up also.
Is this the consensus of most people here also?

Please no flame wars, we just need facts.

I’ve been coding OpenGL-apps and games on a GF4-Ti4400 before I switched to a Radeon9700 about 8 months ago, and I can assure you that ATI’s GL-Drivers aren’t any worse than the ones from Nvidia.
But Nvidia allows you to do some things that may crash on ATI, since that are things you shouldn’t do. I’ve noticed that on older Catalyst-Drivers (not happening since 3.7 anymore) an app will crash if you forget to close a glBegin…glEnd-Block with an glEnd. But besides this issue that has been resolved some time ago, I’ve never had any problems with ATI’s GL-Implementation and I’m coding OpenGL-Apps day by day.

The GL has an error mechanism for a reason. Failing to call glEnd() should result in an OpenGL error, not in app termination. The only excuse the drivers should have for crashing is when you pass them bad pointers.

I don’t use Radeons very frequently, but I’ve found at least six (confirmed) bugs in ATI’s drivers in the past year. None of them were in any way related to programmer errors, and all of them were resolved by driver upgrades. My main card is an FX, and the drivers never crashed on me except in the aforementioned bad pointer scenario.

ATI is very quick in resolving issues, though, so if you have problems with Cats 3.5 all the way through 3.9, you should really try to narrow down the cause of the problem and tell them about it.

– Tom

are you using sone special extensions ? my expirience is that usually the newest extension implementations tend to crash regradless which HW is used.
i have written recently some arb_vertexprogramms which run & compile perfectly on ATI and on nvidia with Detonator45.xx but crashes with the newest release of the Nvidia-driver.
(in some vp’s it appears to support only 2 instead of 4 texture-units on a gf4!, and in some cases i have to make some syntax corrections to get it compiled! 'couse NVidias compiler seems to have more restrictions - or the ATI’s has less i’m not sure about that at this point. i just discovered it)
…but on the other side, any other Catalyst release than 3.4 seems to crash on my workstation.

I have recently released a game, and we run hardware-compilance-tests on a veriety of hardware configurations (we actually hired a hardware-testing lab for this tendious task… )
My conclusion of the tests was: if you want to have 100% crash-proof applications you have not to use any extension. And in some special cases it is even needed to check for particular hardware/driver versions and to disable some features in this case.
If you still decide to use some extensions you must create some way for the user to disable them.(it always better to give the user some options to play around with, before he calls your hotline… )

Dan,
where does it crash? I hope it’s not glCombinerInputNV …

i was able to crash my system with my gf2mx about once a month… now i’m able to do this with my radeon9700pro again…

but somehow i think its me

I think I read about a problem in the 9600 Pro, in that you need to have fastwrite support ON, that was in the NWN forums. I also think that haveing that ON was causing other issues also, but you may wish to toggle that on/off to see if it helps.

For what it is worth, I would say that Nvidia’s drivers are better for most things I have tried so far in terms of stability. I can’t think of a time when I had a BSOD with the Det 40+ drivers, and using openGL.

I did have a few issues though, and since my event log only keeps the last 30 day events, I do not recall what the error message in the EV was when I did have issues. I am sure I posted it here though, and did get a reply.

Oh, are you using 1.1, 1.2, 1.3, or 1.5 functions? (Speaking of 1.5, I wonder who will be first to have them available to use?)

The new GL1.5 functionality is VBO, occlusion queries, and EXT_shadow_funcs. I don’t think ATI has EXT_shadow_funcs in their drivers, but VBO and occlusion queries have been there for a while. I’m sure NVidia has had all of them for a while.

Or are you refering to shader objects? According to the spec they are not part of the 1.5 core, merely “important adjuncts”.

Yeah, shader objs… “important adjuncts” indeed.

Oh, forgot to say, in the ARB meeting notes, someone did say that they should release conformance tests to the public, but I don’t see any more info on this.

glDan,

I don’t know of a good conformance test. I know that the graphics part of SPECMark uses OpenGL, but that’s not conformance per se.

Regarding crashing on one card and not another, that happens all the time, and it’s a different card/issue each time. As far as I can tell, it’s that way because the PC hardware business is young, and relentlessly price driven, which means that “good enough” will have to do. Compare the interop of car stereos (almost any stereo will work in any car) with computer parts, and cry.

I’d suggest isolating what it is that’s crashing, and either fixing your app if it’s indeed a bug of yours, or reporting it to ATI if it isn’t. Tools that would help include WinDbg and turning on mini-dumps in Windows XP recovery options.

Yes, we are using extensions, nothing vendor specific though, ARB stuff mainly, and 1.2 and some 1.3 also.
As far as where it crashes, that is rather difficult to determine, since it is not in the same place everytime, and once the machine is locked, I can’t do much with the debugger in VC. It almost seems like some kind of timing issue with the drivers. Guess we need to invest in a kernel level debugger, or having not to support ATI cards.

I will check out WinDbg. Oh, working on win2k, not XP yet.

Thanks for the info guys.

I found a conformance test, it is at microsoft’s site, download
“DCT DX9 QFE (Windows XP SP1, Windows Server 2003, Windows Me, Windows 2000)”
Link is : http://www.microsoft.com/whdc/hwtest/device/default.mspx?area=displayadapter

It is about 88MB big, and it has DX tests (among others)in it also. I think it only checks 1.1 maybe 1.2 openGL calls though.

[This message has been edited by Elixer (edited 11-23-2003).]

Forgot to mention, if you do all the tests, better take the day off, and turn off the monitor to save on all the refresh rate checks.

Glean - OpenGL conformation test
“glean is a suite of tools for evaluating the quality of an OpenGL implementation and diagnosing any problems that are discovered. glean also has the ability to compare two OpenGL implementations and highlight the differences between them.” http://glean.sourceforge.net/

sources can be founded in old suse linux distribution(7.x).

But I really don’t know why glean is not supported any more - may be Cass or Brian Paul will tell something?

I wouldn’t say Glean is completely unsupported. If you post to the Glean mailing list you’d probably get a response.

It’s been a while since anyone’s contributed new tests to Glean, however.

-Brian