mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > GPU Computing

Reply
 
Thread Tools
Old 2012-12-18, 17:54   #1
Graff
 
Graff's Avatar
 
Jul 2006
USA (UT-5) via UK (UT)

22·59 Posts
Default CUDA driver disappeared after patch installation + kernel update

Unbelievable. I had a running CUDA installation on one of my machine
for about two days. Then I installed some patches and a new Linux kernel.
After rebooting, mfatkc didn't run. After digging around a bit, I tried
nvidia-smi -a and got:

FATAL: Module nvidia not found.
NVIDIA: failed to load the NVIDIA kernel module.
NVIDIA-SMI has failed because it couldn't communicate with NVIDIA driver. Make sure that latest NVIDIA driver is installed and running.

!!!! It was there before I installed the new kernel.

# dir /dev/nvidiactl
dir: cannot access /dev/nvidiactl: No such file or directory

I assume I need to reinstall CUDA. Has anyone else experienced this?

Gareth
Graff is offline   Reply With Quote
Old 2012-12-18, 19:11   #2
ckdo
 
ckdo's Avatar
 
Dec 2007
Cleves, Germany

2·5·53 Posts
Default

Happens with every kernel update on Ubuntu.
ckdo is offline   Reply With Quote
Old 2012-12-18, 19:14   #3
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3×29×83 Posts
Default

For Ubuntu at least, there's a repository you can add that will auto-update drivers via the package manager. I'll try and find it.

Edit: http://www.ubuntuupdates.org/ppa/ubuntu-x-swat
Code:
sudo add-apt-repository ppa:ubuntu-x-swat/x-updates 
sudo apt-get update
sudo apt-get install nvidia-current

Last fiddled with by Dubslow on 2012-12-18 at 19:21
Dubslow is offline   Reply With Quote
Old 2012-12-18, 19:54   #4
Graff
 
Graff's Avatar
 
Jul 2006
USA (UT-5) via UK (UT)

22×59 Posts
Default

Quote:
Originally Posted by Dubslow View Post
For Ubuntu at least, there's a repository you can add that will auto-update drivers via the package manager. I'll try and find it.

Edit: http://www.ubuntuupdates.org/ppa/ubuntu-x-swat
Code:
sudo add-apt-repository ppa:ubuntu-x-swat/x-updates 
sudo apt-get update
sudo apt-get install nvidia-current
Thanks. That worked. My GPU is functioning again. The installed driver
is now 304.64, which is older than the 310.19 I was running earlier. But
I guess that isn't a problem.
I assume I'll need to do all three commands after each kernel "upgrade"?

Gareth

Last fiddled with by Graff on 2012-12-18 at 20:01
Graff is offline   Reply With Quote
Old 2012-12-18, 20:24   #5
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

2·2,909 Posts
Default

Quote:
Originally Posted by Graff View Post
Thanks. That worked. My GPU is functioning again. The installed driver
is now 304.64, which is older than the 310.19 I was running earlier. But
I guess that isn't a problem.
I assume I'll need to do all three commands after each kernel "upgrade"?

Gareth
Just the last one should do.
The first command was adding a repository. The second was downloading the lists from that repository.
henryzz is offline   Reply With Quote
Old 2012-12-19, 00:25   #6
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

11100001101012 Posts
Default

Quote:
Originally Posted by Graff View Post
Thanks. That worked. My GPU is functioning again. The installed driver
is now 304.64, which is older than the 310.19 I was running earlier. But
I guess that isn't a problem.
I assume I'll need to do all three commands after each kernel "upgrade"?

Gareth
Not quite.

`sudo apt-get upgrade` will tell your package manager to update all the packages on your system.

If you're in Ubuntu (I'm guessing you are) then the graphical interface is called "Update Manager". Any updates to the kernel, drivers, or anything else, will be handled by the Update Manager. Since the drivers are now managed by the package system, you shouldn't need to do anything at all after any kernel upgrades, since those are also handled by the package manager. Any updates to the driver itself will appear on the list of packages that need updating, whenever the Update Manager pops up like that.

In other (simpler) words, the drivers are now a part of the same system that updates the kernel, and all other installed packages on your system. It will make sure that all packages work after any updates.
Dubslow is offline   Reply With Quote
Old 2012-12-27, 21:52   #7
Graff
 
Graff's Avatar
 
Jul 2006
USA (UT-5) via UK (UT)

3548 Posts
Default

Quote:
Originally Posted by Dubslow View Post
If you're in Ubuntu (I'm guessing you are) then the
Yes, Ubuntu 12.04 LTS.

Quote:
Originally Posted by Dubslow View Post
In other (simpler) words, the drivers are now a part of the same system that updates the kernel, and all other installed packages on your system. It will make sure that all packages work after any updates.
Thanks for that info, I hope that is the case.

I've just had to restart both of my GPU-equipped machines and upon reboot
mfaktc fails to run on both machines:

Code:
./mfaktc.exe
mfaktc v0.19 (64bit built)
...
CUDA version info
  binary compiled for CUDA  4.20
  CUDA runtime version      0.0
  CUDA driver version       4350.57
ERROR: CUDA runtime version must match the CUDA toolkit version used during compile!
nvidia-smi -a tells me:

Code:
NVIDIA: could not open the device file /dev/nvidiactl (No such file or directory).
NVIDIA-SMI has failed because it couldn't communicate with NVIDIA driver. Make sure that latest NVIDIA driver is installed and running.
Attempting to reinstall the driver tells me I already have the latest
driver. lshw indicates that the GPU card is using the nvidia driver:

Code:
configuration: driver=nvidia latency=0
printenv | grep cuda shows the correct entries in LD_LIBRARY_PATH
and PATH.

So what am I missing? Why isn't my CUDA setup being maintained
across reboots/power cycles? Is this really normal behavior????

Gareth
Graff is offline   Reply With Quote
Old 2012-12-27, 21:58   #8
Graff
 
Graff's Avatar
 
Jul 2006
USA (UT-5) via UK (UT)

22×59 Posts
Default

Quote:
Originally Posted by Graff View Post
So what am I missing? Why isn't my CUDA setup being maintained
across reboots/power cycles? Is this really normal behavior????
Just tried another reboot. This time ran the nvidia-smi -a command with sudo.
Normal output resulted! Was able to get mfaktc running.
Will now try this on the other machine.

Same thing, no joy until I ran sudo nvidia-smi -a.

Gareth

Last fiddled with by Graff on 2012-12-27 at 22:05 Reason: Tried it on the other machine...
Graff is offline   Reply With Quote
Old 2012-12-27, 22:08   #9
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

23·3·5·79 Posts
Default

Quote:
Originally Posted by Graff View Post
So what am I missing? Why isn't my CUDA setup being maintained
across reboots/power cycles? Is this really normal behavior????
Sadly, yes.

I very recently had a similar situation. I upgraded the kernel on one of my CentOS-64 installations, and suddenly mfaktc failed.

Trying to access the nVidia Unix drivers via lynx and wget failed.

Thankfully I had another system I could use, and a flash drive, on site. So I was able to download the latest driver and run the installation script which compiled the latest driver against the just installed kernel.

This would be funny if it wasn't so sad....
chalsall is online now   Reply With Quote
Old 2012-12-27, 22:10   #10
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

11100001101012 Posts
Default

Huh, yes that is really bizzare behavior. Another thing to try is `sudo apt-get upgrade`, though I'm not sure that would help.

I have no idea why the drivers seem to disappear, or why `sudo nvidia-smi -a` would fix it (but not without the sudo).
Dubslow is offline   Reply With Quote
Old 2012-12-28, 03:01   #11
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

23·3·5·79 Posts
Default

Quote:
Originally Posted by Dubslow View Post
Huh, yes that is really bizzare behavior. Another thing to try is `sudo apt-get upgrade`, though I'm not sure that would help.
Actually, that's not really bizzare.

The nVidia drivers are proprietary code. So you are suppose to download them yourself each and every time. Then run the script to recompile the driver against your current kernel.

Welcome to freedom... Even though you payed for all the hardware, you still have to jump through hoops to run said hardware using free software....
chalsall is online now   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
What happened to moregimps.it? It seems to have disappeared... Stargate38 Operation Billion Digits 2 2017-05-08 18:53
NVIDIA TITAN 320.59 driver with cuda 5.5 Manpowre GPU Computing 43 2013-08-22 12:28
Ubuntu 10.10 Maverick and CudaLucas 1.2 - Fatal kernel update moebius Linux 1 2012-02-20 03:17
A new driver? (or type of driver?) 10metreh Aliquot Sequences 3 2010-02-15 15:57
smilies disappeared from post composition page! mdettweiler Forum Feedback 3 2008-02-21 09:20

All times are UTC. The time now is 03:26.

Thu Mar 4 03:26:44 UTC 2021 up 90 days, 23:38, 1 user, load averages: 2.07, 1.85, 1.75

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.