(archive)MISFIT - Page 9

swl551 · 2012-12-07, 16:02

This weekend I'll be adding the file locking mechanism to MISFIT that bdot uses in mfaktO. Once completed the possibility of I/O collision between MISFIT and mfaktO should be zero.

mfaktC does not implement a file locking system so there will not be any benefit for that user base.

Dubslow · 2012-12-07, 20:55

Quote:

Originally Posted by swl551

This weekend I'll be adding the file locking mechanism to MISFIT that bdot uses in mfaktO. Once completed the possibility of I/O collision between MISFIT and mfaktO should be zero.

mfaktC does not implement a file locking system so there will not be any benefit for that user base.

CUDALucas 2.04 uses the same file locking as mfakto. (The code was quite literally copied.)

swl551 · 2012-12-08, 20:05

Quote:

Originally Posted by Dubslow

CUDALucas 2.04 uses the same file locking as mfakto. (The code was quite literally copied.)

I will implement file locking into MISFIT CuLu at a later time.

swl551 · 2012-12-08, 20:08

VERSION 2.2.0 is a real milestone in the life of MISFIT.

1. Implemented mfaktO style '.lck' file locking system.
The following MISFIT actions execute file-lock-management (detection, retry, assertion, cleanup)
UPDATE STATS
ADD WORK
ASSIGN WORK
BALANCE WORK
EXPORT RESULTS
EDITING FILES (BUILT IN EDITOR ONLY)
SEARCHING WORKTODO
WORKTO REPORTS
RESULTS REPORTS

In a typical MISFIT installation there is more than one directory containing files that need file-lock-management therefore whenever executing an action that requires file-lock-management MISFIT treats the entire set as one. If any one of the files has a lock collision or hard error the entire set is rolled back out of the locked state. (think database transactions /commit /rollback) and MISFIT aborts further processing.

An example would be you have 8 directories and you want to assign work out to them. MISFIT starts locking each workToDo.txt in each directory. If directory 6 encounters a lock collision and the wait_timeout is exceeded trying to apply a lock to workToDo (in directory 6) then MISFIT rollsback (unlocks) all the workToDo files it succesfully locked in directories 5,4,3,2,1 and aborts further processing.

LOCKING always executes a retry loop if there is an existing lock on a file (LOCK_COLLISION). The retry loop last up to 10 retries with SLEEP(100ms * RETRY_COUNT) between each retry. If the counter reaches 10 then LOCK_COLLISION is terminal and MISFIT aborts further processing. If the locks is released by the "other program" during the retry loop MISFIT will then assert the lock and continue processing.

Menu items for LOCK management are included to forcibly apply/remove/report on systemwide locks.

On a special note: Currently ONLY MFAKTO implements the '.lck' file locking system.

How to test with MFAKTO:
Stop your MFAKTO instances.
Use the menu item to "Apply systemwide file-locks"
Start MFAKTO
Observe MFAKTO's response to the file-locks
Use the menu item to "Remove systemwide file-locks"
Observe MFaKTO's response once the locks are removed.

Unlike MFAKTO which will wait forever for a lock to be released, MISFIT is a UI driven application and waiting forever is unsuitable as MISIT has many tasks to perform at any given time. If a process is aborted due to a lock collision it can be re-executed at a later time. Remember most lock conditions last about .25 seconds so allowing MISFIT to wait a max of 5.5 seconds for a lock to be released is very tolerant and suitable to a UI.

Enjoy.

Get from http://mersenneforum.org/misfit/

Bdot · 2012-12-09, 11:24

Quote:

Originally Posted by swl551

VERSION 2.2.0 is a real milestone in the life of MISFIT.

1. Implemented mfaktO style '.lck' file locking system.
...
Enjoy.

Get from http://mersenneforum.org/misfit/

Great that locking is being used, thanks! Really nice to see MISFIT growing up so fast!

sonjohan · 2012-12-10, 13:21

Sorry that I'm not commenting anymore, as I'm not doing LL tests, but only TF on a single PC, I haven't updated nor tried out (m)any of the new versions.

swl551 · 2012-12-13, 00:36

This is the MISFIT version to support CUDALucas

VERSION 1.0.2
1. Added GHZDayToDo calculation
2. Added GhzDzDone calculation including the FFT component as found in the results line of the file
3. Enhanced MISFIT to further prevent unwanted/premature purging of checkpoint files under abnormal conditions by:
a. increasing the inital delay to 60 minutes (from 30)
b. After all stalled process are cleared there is now a 60 minute delay before the next purge cycle will run

Get from http://mersenneforum.org/misfit/

flashjh · 2012-12-13, 05:17

Quote:

Originally Posted by swl551

This is the MISFIT version to support CUDALucas

VERSION 1.0.2
1. Added GHZDayToDo calculation
2. Added GhzDzDone calculation including the FFT component as found in the results line of the file
3. Enhanced MISFIT to further prevent unwanted/premature purging of checkpoint files under abnormal conditions by:
a. increasing the inital delay to 60 minutes (from 30)
b. After all stalled process are cleared there is now a 60 minute delay before the next purge cycle will run

Get from http://mersenneforum.org/misfit/

Working great and looking good. Thanks.

swl551 · 2012-12-16, 22:18

Is this the calm before some storm.

kracker · 2012-12-16, 22:25

Quote:

Originally Posted by swl551

Is this the calm before some storm.

Boom BOOOOOOOOOOOOMM!!

...
Hmm, Did I miss something?

...

swl551 · 2012-12-16, 22:29

Quote:

Originally Posted by kracker

Boom BOOOOOOOOOOOOMM!!

...
Hmm, Did I miss something?

...

I don't think you missed anything. I'm just wondering where everyone is... So quiet...

2012-12-07, 16:02	#89
swl551 Aug 2012 New Hampshire 2³×101 Posts	Next release of MISFIT will implement file locking This weekend I'll be adding the file locking mechanism to MISFIT that bdot uses in mfaktO. Once completed the possibility of I/O collision between MISFIT and mfaktO should be zero. mfaktC does not implement a file locking system so there will not be any benefit for that user base. Last fiddled with by swl551 on 2012-12-07 at 16:03

2012-12-08, 20:08	#92
swl551 Aug 2012 New Hampshire 2³·101 Posts	MISFIT 2.2.0 (WITH FILE LOCKING) [non CuLu version] VERSION 2.2.0 is a real milestone in the life of MISFIT. 1. Implemented mfaktO style '.lck' file locking system. The following MISFIT actions execute file-lock-management (detection, retry, assertion, cleanup) UPDATE STATS ADD WORK ASSIGN WORK BALANCE WORK EXPORT RESULTS EDITING FILES (BUILT IN EDITOR ONLY) SEARCHING WORKTODO WORKTO REPORTS RESULTS REPORTS In a typical MISFIT installation there is more than one directory containing files that need file-lock-management therefore whenever executing an action that requires file-lock-management MISFIT treats the entire set as one. If any one of the files has a lock collision or hard error the entire set is rolled back out of the locked state. (think database transactions /commit /rollback) and MISFIT aborts further processing. An example would be you have 8 directories and you want to assign work out to them. MISFIT starts locking each workToDo.txt in each directory. If directory 6 encounters a lock collision and the wait_timeout is exceeded trying to apply a lock to workToDo (in directory 6) then MISFIT rollsback (unlocks) all the workToDo files it succesfully locked in directories 5,4,3,2,1 and aborts further processing. LOCKING always executes a retry loop if there is an existing lock on a file (LOCK_COLLISION). The retry loop last up to 10 retries with SLEEP(100ms * RETRY_COUNT) between each retry. If the counter reaches 10 then LOCK_COLLISION is terminal and MISFIT aborts further processing. If the locks is released by the "other program" during the retry loop MISFIT will then assert the lock and continue processing. Menu items for LOCK management are included to forcibly apply/remove/report on systemwide locks. On a special note: Currently ONLY MFAKTO implements the '.lck' file locking system. How to test with MFAKTO: Stop your MFAKTO instances. Use the menu item to "Apply systemwide file-locks" Start MFAKTO Observe MFAKTO's response to the file-locks Use the menu item to "Remove systemwide file-locks" Observe MFaKTO's response once the locks are removed. Unlike MFAKTO which will wait forever for a lock to be released, MISFIT is a UI driven application and waiting forever is unsuitable as MISIT has many tasks to perform at any given time. If a process is aborted due to a lock collision it can be re-executed at a later time. Remember most lock conditions last about .25 seconds so allowing MISFIT to wait a max of 5.5 seconds for a lock to be released is very tolerant and suitable to a UI. Enjoy. Get from http://mersenneforum.org/misfit/ Attached Thumbnails Last fiddled with by swl551 on 2012-12-08 at 20:21

2012-12-13, 00:36	#95
swl551 Aug 2012 New Hampshire 328₁₆ Posts	MISFITculu 1.0.2 This is the MISFIT version to support CUDALucas VERSION 1.0.2 1. Added GHZDayToDo calculation 2. Added GhzDzDone calculation including the FFT component as found in the results line of the file 3. Enhanced MISFIT to further prevent unwanted/premature purging of checkpoint files under abnormal conditions by: a. increasing the inital delay to 60 minutes (from 30) b. After all stalled process are cleared there is now a 60 minute delay before the next purge cycle will run Get from http://mersenneforum.org/misfit/

2012-12-16, 22:18	#97
swl551 Aug 2012 New Hampshire 1450₈ Posts	Ok, you guys are too quiet! Is this the calm before some storm.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
questions about misfit	jschwar313	MISFIT	1	2016-02-02 00:32
Help with Misfit	TheMawn	MISFIT	18	2015-06-21 22:00
Misfit feature request.	Aramis Wyler	MISFIT	15	2015-03-04 21:13
MISFIT 2.9	swl551	MISFIT	44	2014-12-15 12:46
MISFIT for MFAKTx	swl551	MISFIT	186	2014-02-05 23:27

2012-12-10, 13:21	#94
sonjohan May 2003 Belgium 2×139 Posts	Sorry that I'm not commenting anymore, as I'm not doing LL tests, but only TF on a single PC, I haven't updated nor tried out (m)any of the new versions.