[QUOTE=xx005fs;528806]Interesting, I thought kaggle would offer more free storage than 5G since a lot of AI training datasets easily exceeds that. [/QUOTE]

Training datasets are readonly, yes? They can go into /kaggle/input. Working folder is for outputs (which are retained forever). Naturally they would want to limit that. You are also free to create other folders outside /kaggle/working without space constraint, I believe.

Factors found?
 

Has anyone found a factor yet by Colab or Kaggle computing?
I haven't yet, and am no longer trying P-1 since the selftest failed.
Dozens of TF no factor found, but no factors found yet. Need a bigger sample size.

[QUOTE=kriesel;528911]Has anyone found a factor yet by Colab or Kaggle computing?
[/QUOTE]

I have found a few factors on Colab/Kaggle, the most recent being a factor for [URL="https://www.mersenne.org/report_exponent/?exp_lo=97653289&full=1"]M97653289.[/URL]
I also had found a 114.266 bit factor of [URL="https://www.mersenne.ca/exponent/3331309"]M3331309[/URL] using mprime’s ECM on a colab instance.

[QUOTE=kriesel;528911]Has anyone found a factor yet by Colab or Kaggle computing?[/QUOTE]I have found 3: [M]97464457[/M], [M]97474501[/M], and [M]97502947[/M] have factors. Out of 134 runs in that range (and some in the DC range, but that chart is broke.)

[QUOTE=kriesel;528911]Has anyone found a factor yet by Colab or Kaggle computing?
I haven't yet, and am no longer trying P-1 since the selftest failed.
Dozens of TF no factor found, but no factors found yet. Need a bigger sample size.[/QUOTE]


I found 2 in 219 runs for
[URL="https://www.mersenne.org/report_exponent/?exp_lo=147494717&full=1"][COLOR=#000000]147494717[/COLOR][/URL]
[URL="https://www.mersenne.org/report_exponent/?exp_lo=848999959&full=1"][COLOR=#000000]848999959[/COLOR][/URL]

[QUOTE=axn;528865]Training datasets are readonly, yes? They can go into /kaggle/input. Working folder is for outputs (which are retained forever). Naturally they would want to limit that. You are also free to create other folders outside /kaggle/working without space constraint, I believe.[/QUOTE]

Sorry I meant the checkpoint files. I know that sometimes they can get absolutely huge. For GIMPS purposes though I don't think that much storage is necessary.

[QUOTE=kriesel;528911]Has anyone found a factor yet by Colab or Kaggle computing?
[/QUOTE]

I found a couple of TF factors in the 97M and 65M ranges, and an ECM factor for some exponent in the 200K range.
I am using Colab only.

[QUOTE=xx005fs;528922]Sorry I meant the checkpoint files. I know that sometimes they can get absolutely huge.[/QUOTE]

Absolutely.

It then comes down to the programmer(s) to manage the nuances of the resources, and ensure that the work unit is done "Just In Time".

Edit: Actually, modeling it for a bit... Since checkpoint files larger than 5 GB won't survive a "kill", there's no point in generating them. Might save a few seconds per run.

Introducing tf1G.py v0.04:
Changes relative to the last version: Now the code will stop fetching work if there is no work available with the given parameters and it will also keep track of the number of exponents factored and the number done:


[CODE]#Script to automate trial factoring of Mersennes above 1G, using mfaktx.
#Requires wget, curl and grep.
#version history:
#v0.01 - Testing version.
#v0.02 - First public release.
#v0.04 - Stop trying to fetch if there are no assignments of that type.
#- Count number of exponents tested and factored.

#import needed packages
import sys, os, subprocess, signal, time
#set path to mfaktx
mfaktx_path = "C:\\Users\\Dylan\\Desktop\\mfaktc\\mfaktc-0.21\\"

#names of executables, change if needed
MFAKTX = 'mfaktc.exe'
WGET = 'wget.exe'
CURL = 'curl.exe'
GREP= 'C:\\cygwin64\\bin\\grep.exe'

#specify certain parameters for later, when we go and fetch assignments:
TF_LIMIT = str(71)
TF_MIN = str(68)
MAXASSIGNMENTS = str(1)
BIGGEST = str(1)

#changes should not be needed below
print ("---------------------------------")
print ("This is tf1G.py v0.04, a Python script to automate Mersenne trial factoring for exponents above 1 billion.")
print ("It is copyleft, 2019, by Dylan Delgado.")
print ("---------------------------------")

#run checks to see if we have the paths correct
if not os.path.exists(mfaktx_path):
print("The path for Mfaktx does not exist. Check your setting for mfaktx_path.")
sys.exit()
else:
#do we have mfactx?
if not os.path.exists(mfaktx_path + MFAKTX):
print("Mfaktx does not exist. Check your path or name of your executable.")
sys.exit()
#Now we define our URL
URL = "https://www.mersenne.ca/tf1G.php?download_worktodo=1&tf_limit=" + TF_LIMIT + "&tf_min=" + TF_MIN + "&max_assignments=" + MAXASSIGNMENTS + "&biggest=" + BIGGEST
print(URL)

#delete a file (code courtesy of Brian Gladman)
def delete_file(fn):
if os.path.exists(fn):
try:
os.unlink(fn)
except WindowsError:
pass

#submit work to mersenne.ca
def submit_results():
print("Submitting work...")
subprocess.run([CURL, "-F", "results_file=@results.txt", "https://www.mersenne.ca/bulk-factors.php"])
delete_file(mfaktx_path + "results.txt")
#analyze a file for factors for statistics
def analyze_for_factors(file):
exp_with_factors = 0
#use grep and look for the phrase "has a factor:. Push that to the file "exp_factors.txt"."
f = open("exp_factors.txt","w+")
f.write(str(subprocess.run([GREP, "has a factor:", file])))
exp_with_factors = sum(l for line in f)
f.close()
delete_file("exp_factors.txt")
return exp_with_factors
#main loop - fetch work with wget, run mfaktx, and submit results
exp_tested = 0
exp_factored = 0
while(True):
#check if we have a worktodo.txt
if not os.path.exists(mfaktx_path + 'worktodo.txt'):
print("No work to do, fetching more work in 5 seconds...")
time.sleep(5)
#No need to check for factors since this is presumably the first time we are running the script...
submit_results()
subprocess.run([WGET, URL, "-Oworktodo.txt"])
if os.stat(mfaktx_path + "worktodo.txt").st_size == 0: #no assignments were fetched
raise Error("No assignments were fetched, check your parameters for TF_LIMIT, TF_MIN or MAXASSIGNMENTS.")
#check if worktodo.txt is empty
elif os.stat(mfaktx_path + "worktodo.txt").st_size == 0:
print("No work to do, fetching more work in 5 seconds...")
time.sleep(5)
exp_with_factors = analyze_for_factors(mfaktx_path + "results.txt")
exp_tested = exp_tested + int(MAXASSIGNMENTS)
exp_factored = exp_factored + int(exp_with_factors)
print("Number of exponents tested: " + str(exp_tested))
print("Number of exponents factored: " + str(exp_factored))
submit_results()
subprocess.run([WGET, URL, "-Oworktodo.txt"])
if os.stat(mfaktx_path + "worktodo.txt").st_size == 0:
#no assignments were fetched
raise Error("No assignments were fetched, check your parameters for TF_LIMIT, TF_MIN or MAXASSIGNMENTS.")
#run mfaktc
subprocess.run([mfaktx_path + MFAKTX])[/CODE]

Introducing tf1G.py v0.05:
The previous code for analyzing for factored exponents didn't update the number of factored exponents. This is because subprocess.run returns a CompletedProcess object when the process is done. That code has been rewritten so it will work (and it doesn't require grep).
Also some spelling mistakes were fixed (only in comments, thankfully).


[CODE]#Script to automate trial factoring of Mersennes above 1G, using mfaktx.
#Requires wget and curl.
#version history:
#v0.01 - Testing version.
#v0.02 - First public release.
#v0.04 - Stop trying to fetch if there are no assignments of that type.
#- Count number of exponents tested and factored.
#v0.05 - subprocess.run returns a CompletedProcess object, so the number of factored exponents was not updating. So I fixed the analyze_for_factors function.
#- some smelling mistakes fixed.

#import needed packages
import sys, os, subprocess, signal, time
#set path to mfaktx
mfaktx_path = "C:\\Users\\Dylan\\Desktop\\mfaktc\\mfaktc-0.21\\"

#names of executables, change if needed
MFAKTX = 'mfaktc.exe'
WGET = 'wget.exe'
CURL = 'curl.exe'

#specify certain parameters for later, when we go and fetch assignments:
TF_LIMIT = str(71)
TF_MIN = str(68)
MAXASSIGNMENTS = str(1)
BIGGEST = str(1)

#changes should not be needed below
print ("---------------------------------")
print ("This is tf1G.py v0.05, a Python script to automate Mersenne trial factoring for exponents above 1 billion.")
print ("It is copyleft, 2019, by Dylan Delgado.")
print ("---------------------------------")

#run checks to see if we have the paths correct
if not os.path.exists(mfaktx_path):
print("The path for Mfaktx does not exist. Check your setting for mfaktx_path.")
sys.exit()
else:
#do we have mfaktx?
if not os.path.exists(mfaktx_path + MFAKTX):
print("Mfaktx does not exist. Check your path or name of your executable.")
sys.exit()
#Now we define our URL
URL = "https://www.mersenne.ca/tf1G.php?download_worktodo=1&tf_limit=" + TF_LIMIT + "&tf_min=" + TF_MIN + "&max_assignments=" + MAXASSIGNMENTS + "&biggest=" + BIGGEST
print(URL)

#delete a file (code courtesy of Brian Gladman)
def delete_file(fn):
if os.path.exists(fn):
try:
os.unlink(fn)
except WindowsError:
pass

#submit work to mersenne.ca
def submit_results():
print("Submitting work...")
subprocess.run([CURL, "-F", "results_file=@results.txt", "https://www.mersenne.ca/bulk-factors.php"])
delete_file(mfaktx_path + "results.txt")
#analyze a file for factors for statistics
def analyze_for_factors(file):
exp_with_factors = 0
#use re and look for the phrase "has a factor:" in file. If it exists, increment exp_with_factors and print the line.
g = open(file,"r")
to_match = ["has a factor:"]
for line in g:
for each_to_match in to_match:
if each_to_match in line:
print(line)
exp_with_factors += 1
continue
return exp_with_factors
#main loop - fetch work with wget, run mfaktx, and submit results
exp_tested = 0
exp_factored = 0
while(True):
#check if we have a worktodo.txt
if not os.path.exists(mfaktx_path + 'worktodo.txt'):
print("No work to do, fetching more work in 5 seconds...")
time.sleep(5)
#No need to check for factors since this is presumably the first time we are running the script...
submit_results()
subprocess.run([WGET, URL, "-Oworktodo.txt"])
if os.stat(mfaktx_path + "worktodo.txt").st_size == 0: #no assignments were fetched
raise Error("No assignments were fetched, check your parameters for TF_LIMIT, TF_MIN or MAXASSIGNMENTS.")
#check if worktodo.txt is empty
elif os.stat(mfaktx_path + "worktodo.txt").st_size == 0:
print("No work to do, fetching more work in 5 seconds...")
time.sleep(5)
exp_with_factors = analyze_for_factors(mfaktx_path + "results.txt")
exp_tested = exp_tested + int(MAXASSIGNMENTS)
exp_factored = exp_factored + int(exp_with_factors)
print("Number of exponents tested: " + str(exp_tested))
print("Number of exponents factored: " + str(exp_factored))
submit_results()
subprocess.run([WGET, URL, "-Oworktodo.txt"])
if os.stat(mfaktx_path + "worktodo.txt").st_size == 0:
#no assignments were fetched
raise Error("No assignments were fetched, check your parameters for TF_LIMIT, TF_MIN or MAXASSIGNMENTS.")
#run mfaktx
subprocess.run([mfaktx_path + MFAKTX])[/CODE]

[QUOTE=Fan Ming;528390]So why my attachment disappeared without notification... Did it violate any rules? Here I reupload it in zip.[/QUOTE]
How did you get that to build? When I tried make gpuowl I got
[CODE]g++ -MT GmpUtil.o -MMD -MP -MF .d/GmpUtil.Td -Wall -O2 -std=c++17 -c -o GmpUtil.o GmpUtil.cpp
GmpUtil.cpp:3:10: fatal error: gmp.h: No such file or directory
#include <gmp.h>
^~~~~~~
compilation terminated.
Makefile:30: recipe for target 'GmpUtil.o' failed
make: *** [GmpUtil.o] Error 1[/CODE]