[QUOTE=swl551;325384]What do you want the userAgent string to be: "MISFITWorkFetcher" ?[/QUOTE]

Sure, that would work. Something unique is all we need.

[QUOTE=swl551;325384]It should be noted that the existing version could be wrapped in a bat file that checks the errorlevel (result) and initiates another call pointing to a different config file thereby providing a secondary request.[/QUOTE]

Is not the whole point of WinBlows to isolate the user(s) from needing to know about [I]scary[/I] things like scripts?

[QUOTE=chalsall;325387]Is not the whole point of WinBlows to isolate the user(s) from needing to know about [I]scary[/I] things like scripts?[/QUOTE]

Yes it is. Hardly anyone knows how to do Windows [B]command shell scripting[/B] anymore..... The old days are gone.

Fallback to What Makes Sense configuration example
 

Since I am adding Fetching directly into MISFIT here is how it will look.

[QUOTE=swl551;325391]Since I am adding Fetching directly into MISFIT here is how it will look.[/QUOTE]

Looks reasonable.

Just wondering though... What does the checkbox labeled "Replace placeholder with YYYY-MM-DD" do?

[QUOTE=chalsall;325392]Looks reasonable.

Just wondering though... What does the checkbox labeled "Replace placeholder with YYYY-MM-DD" do?[/QUOTE]

turns Factor=N/A,577757,68,69
into Factor=2013-01-21,577757,68,69

From GIMPS N/A is a GUID.

Usefull to people who fondle their assignments a lot: FLASHJH being a prime example!

[QUOTE=swl551;325393]Usefull to people who fondle their assignments a lot: FLASHJH being a prime example![/QUOTE]

Again, reasonable.

Dates given in YYYYMMDD order can be sorted using simple command-line programs. (I've never understood why almost everyone except geeks use any other, and almost every other possible, order.... :smile:)

VERSION 2.3.2
 

[QUOTE=swl551;325385]You should be seeing the non-blocking MISFIT alert panel when automation detects an error condition instead of a modal dialog box.

However a file-lock cleanup can be automated just by looking at the age of the lock file. If > 10 mins..... delete.[/QUOTE]



VERSION 2.3.2
1. Added code to release a lock from a file if the ".lck" file is > 15 mins old. This will prevent stale locks from hindering unattended operations.

****************

Still working hard on v 2.4 which puts full gpu72 fetching directly into MISFIT. Hopefully I'll be ready for beta testing by this weekend. The GPU72 fetch code is simple, refitting MISFIT to handle full configuration for two providers (GIMPS, GPU72) and streamlining the code-base is the complicated part.

[QUOTE=swl551;325385]You should be seeing the non-blocking MISFIT alert panel when automation detects an error condition instead of a modal dialog box.

However a file-lock cleanup can be automated just by looking at the age of the lock file. If > 10 mins..... delete.[/QUOTE]
I haven't been home much lately, but when something like a stalled instance occurs and I'm not there to fix it (or I don't log in and catch it), MISFIT doesn't run the fetch code nor does it distribute anything that was already there which causes all the work to run out.

I wish I could say nothing else will go wrong, but for now I'll just watch for another scenario and hopefully take some screen shots.

[QUOTE=swl551;325424]VERSION 2.3.2
1. Added code to release a lock from a file if the ".lck" file is > 15 mins old. This will prevent stale locks from hindering unattended operations.
[/QUOTE]Thanks.
[QUOTE=swl551;325388]Yes it is. Hardly anyone knows how to do Windows [B]command shell scripting[/B] anymore..... The old days are gone.[/QUOTE]
You'd love the one I put together to do what you coded for G72 fetching ;)

[QUOTE=flashjh;325426]I haven't been home much lately, but when something like a stalled instance occurs and I'm not there to fix it (or I don't log in and catch it), MISFIT doesn't run the fetch code nor does it distribute anything that was already there which causes all the work to run out.
[/QUOTE]

FLASHJH. I just did a code review to see if there is a blocking condition related to a stalled instance. There is not one. The inability to apply a lock is a blocking condition. So if a lock is applied and then a network error occurs and the lock is orphaned the next time a lock is requested there will be collision. So the 2.3.2 PURGE_STALE_LOCK will solve that.


Code snippet showing exactly how the main methods are called.
[CODE]
if (timer1UpdateStatsCounter >= globals.cfg.settingUpdateStatsInterval)
{
UpdateGridStats();
AssignWorkViaAutomation();
BalanceWorkViaAutomation();

}

[/CODE]

[QUOTE=flashjh;325426]
You'd love the one I put together to do what you coded for G72 fetching ;)[/QUOTE]

Same here. We are old school guys, and I believe the most of the members of this forum can handle very well some batch making thingies (*nix or win), or string processing toys like perl, etc. I personally do windoze stuff.
[ATTACH]9156[/ATTACH]

[QUOTE=LaurV;325432]Same here. We are old school guys, and I believe the most of the members of this forum can handle very well some batch making thingies (*nix or win), or string processing toys like perl, etc. I personally do windoze stuff.
[ATTACH]9156[/ATTACH][/QUOTE]

I'm hardly old school, and am proficient in *nix scripting... :razz:

[QUOTE=Dubslow;325433]I'm hardly old school, and am proficient in *nix scripting... :razz:[/QUOTE]

Stick near us kid. You'll do OK.... :wink:

[QUOTE=Dubslow;325433]I'm hardly old school, and am proficient in *nix scripting... :razz:[/QUOTE]


Chalsall's original assertion is that Windows user rely less on scripting than *nix users due to the Windows UI. I concur. In my old DOS days scripting was part of the job. IT departments could roll out entire Windows 3.x installations via scripting. Write menu systems like Laurv did and just about everything else. Today our IT department has to come to corporate development groups to show them how to use xcopy.

So scripting is good and no insult to anyone uses it.

[QUOTE=swl551;325435]Chalsall's original assertion is that Windows user rely less on scripting than *nix users due to the Windows UI.[/QUOTE]

That's not exactly what I asserted. But it's not too far off the mark.

Bill Gates et al got rich by enabling those who perhaps shouldn't be using computers to pretend to do so. Anyone else here find themselves employing someone who was using a spreadsheet as a desktop publishing program?

"Math class is tough!" -- Barbie

I back to running 2.3.1 for a while because 2.3.2 kept telling me there was a stall condition (even though everything was working fine). It kept fetching but couldn't assign work, so I have about a weeks worth of TF for now :smile:

I'll see if t's something I was doing...

[QUOTE=flashjh;325476]I back to running 2.3.1 for a while because 2.3.2 kept telling me there was a stall condition (even though everything was working fine). It kept fetching but couldn't assign work, so I have about a weeks worth of TF for now :smile:

I'll see if t's something I was doing...[/QUOTE]


I have seen false stalls with 0.20 running bit levels of 68,69. 0.20 does not output an ETA for a small range and therefore may not be creating .ckp files even though the duration of the run may exceed the checkpoint file interval. So if you processing 20 small bit ranges in a row... STALL alarm is triggered.

We might need to discuss with Oliver.

[QUOTE=swl551;325481]So if you processing 20 small bit ranges in a row... STALL alarm is triggered.[/QUOTE]

Please forgive me if this is a stupid question, but what about simply monitoring the worktodo.txt file as well (if you aren't already)?

[QUOTE=chalsall;325483]Please forgive me if this is a stupid question, but what about simply monitoring the worktodo.txt file as well (if you aren't already)?[/QUOTE]

Well the stall indicator is based on .ckp files changing which typically had a predicatable rate of "change". For wide range, like 68,74 and stages=off the workToDo won't change for 4 to 8 hours depending on your speed making them less useful in stall identification and the reason I didn't code to workToDo.txt(s)


If the run takes 4 minutes and checkpoint is set to 30 seconds there should be checkpoints being created. I think the lack of CKP files needs to be directed to Oliver.

[QUOTE=swl551;325484]For wide range, like 68,74 and stages=off the workToDo won't change for 4 to 8 hours depending on your speed making them less useful in stall identification and the reason I didn't code to workToDo.txt(s)[/QUOTE]

I understand that. But I see no downside (other than a [U]tiny[/U] amount of additional processing time) to monitor both.

[QUOTE=chalsall;325486]I understand that. But I see no downside (other than a [U]tiny[/U] amount of additional processing time) to monitor both.[/QUOTE]

Yes I can do it... The only downside is that auto-fetching and work balancing can modify the worktodo.txt(s) so the "lastWriteTime" could be recent, but the mfaktX instances is actually stalled. A fringe case for sure but worth mentioning.

[QUOTE=swl551;325488]A fringe case for sure but worth mentioning.[/QUOTE]

Monitor the file size as well as the last modified date. It's still meta-data (as in, you don't need to open the file; i.e. it's fast).

[QUOTE=chalsall;325492]Monitor the file size as well as the last modified date. It's still meta-data (as in, you don't need to open the file; i.e. it's fast).[/QUOTE]


Right. I'm onboard. Now finding more time...... I still want to check with Oliver on the assumed base condition of .ckp not being written. Then I'll decide if I need to compensate or if he'll address it in his code or my assumptions are totally off and something else is going on.

I'm processing 61*M range up to 73. It takes about an hour per exponent, so it should be writing ckp files. 2.3.1 just did it too, so it's likely something on my end. I'll run it down when I have time. For now I'll make sure I have enough assignments to prevent too many assignments.

0.20 is writing ckp files even for small bit ranges
 

However there is a long duration where the completed ckp file is deleted and the new ckp file is NOT yet created. MISFIT's stall detection should be tuned to compensate.

Set the check frequency to 5 mins and the number of failed tests to at least 3. 5 is better.. if any ckp file is detected during that window the ckp aging is reset and therefore less prone to false alarms.

K, I'll let you know.

[QUOTE=swl551;325500]Set the check frequency to 5 mins and the number of failed tests to at least 3. 5 is better.. if any ckp file is detected during that window the ckp aging is reset and therefore less prone to false alarms.[/QUOTE]

My apologies if I'm telling you how to chew gum... But please don't forget that the last modified date of the directory might also be useful to you.

I'm imagining a temporal harmonic where (for short work) the time between MISFIT checking for a .ckp file aligns with just after mfaktX has completed its current assignment.

[QUOTE=flashjh;325501]K, I'll let you know.[/QUOTE]
Jerry are you factoring low bit levels?

[QUOTE=swl551;325503]Jerry are you factoring low bit levels?[/QUOTE]

No, I'm doing what makes sense from G72, so 61*M range up to 73.

[QUOTE=chalsall;325502]My apologies if I'm telling you how to chew gum... But please don't forget that the last modified date of the directory might also be useful to you.

I'm imagining a temporal harmonic where (for short work) the time between MISFIT checking for a .ckp file aligns with just after mfaktX has completed its current assignment.[/QUOTE]

in a gpu farm like flash has seeing into remote processes across pcs is complicated. So generic .ckp file testing seemed best case to evaluate.

[QUOTE=swl551;325505]So generic .ckp file testing seemed best case to evaluate.[/QUOTE]

Has it turned out that way? Now that you have empirical data?

There is an old saying in the software development industry which few heed: "Plan to throw the first one away. You will anyway."

We appreciate what you're doing.

Keep doing it.

[QUOTE=flashjh;325504]No, I'm doing what makes sense from G72, so 61*M range up to 73.[/QUOTE]

[QUOTE=flashjh]I'm processing 61*M range up to 73. It takes [B]about an hour[/B] per exponent, so it should be writing ckp files. 2.3.1 just did it too, so it's likely something on my end.[/QUOTE]

The bug is obvious.

[QUOTE=chalsall;325506]Has it turned out that way? Now that you have empirical data?

There is an old saying in the software development industry which few heed: "Plan to throw the first one away. You will anyway."

We appreciate what you're doing.

Keep doing it.[/QUOTE]

You are boxing me into incriminating myself:no:. Damn you'd be a good lawyer.
......
I feel that the existing method works and since the parameters are user adjustable the condition is solved without redesign. Sure it could have been done another and I often do rewrite code as requirements change. I don't see this area needing rewrite yet.

If the need arises to rewrite or modify stall detection I will take your suggestion of including monitoring of the workToDo(s). So do feel like you didn't help out!

[QUOTE=chalsall;325509]The bug is obvious.[/QUOTE]

Do tell as it is not obvious to me!

[QUOTE=swl551;325510]You are boxing me into incriminating myself:no:. Damn you'd be a good lawyer. [/QUOTE]

I advise lawyers...

[QUOTE=swl551;325511]Do tell as it is not obvious to me![/QUOTE]

If your program only checks for the existence of .ckp files immediately after mfaktX completed its assignment then it might get confused.

[QUOTE=chalsall;325512]I advise lawyers...[/QUOTE]

Are you the devil? LOL

[QUOTE=TObject;325514]Are you the devil? LOL[/QUOTE]

Yes.

Alright,

The problem is on my end. I got home and checked and 5 of 6 systems we not running mfaktc. I don't know why, but I think it may have happened this morning before I left as the systems were getting low on assignments, I made sure the exponents got assigned, but I never checked to make sure they were running. So the stalled warnings all day were valid. If I would have checked... oh well. I'll track tonight and see if any more errors pop up.

[QUOTE=flashjh;325524]Alright,

The problem is on my end. I got home and checked and 5 of 6 systems we not running mfaktc. I don't know why, but I think it may have happened this morning before I left as the systems were getting low on assignments, I made sure the exponents got assigned, but I never checked to make sure they were running. So the stalled warnings all day were valid. If I would have checked... oh well. I'll track tonight and see if any more errors pop up.[/QUOTE]
Don't forget u can do a global status inquiry using the remote manager features.

and... Phew... Now I can get back to 2.4

[QUOTE=swl551;325525]Don't forget u can do a global status inquiry using the remote manager features.

and... Phew... Now I can get back to 2.4[/QUOTE]

I know, and it was stupid not to. The display showed each worker still had at least 1 assignment, but it hadn't updated. From now on I'll make sure to check before I leave for the day. And hopefully there won't be a next time.

Thanks again.

VERSION 2.4.B2 (beta) prep build for GPU72 fetching
 

VERSION 2.4.B2
[B]Major[/B] overhaul of code base to support fetching from GIMPS, GPU72 or External file. [B]GPU72 fetch is NOT implemented yet[/B], but so much has been changed it is important to regression test MISFIT against current functionality. The preparation for implementing GPU72 support forced me to rename many configuration variables to clarify if they were GIMPS specific, GPU72 specific or generic Work Fetching values. Currently the GPU72 work fetch button is disabled since it is not ready.

An improvement to the encryption key generation was done which will make MISFIT unable to decrypt your existing SMTP, GIMPS passwords. Once re-encrypted with 2.4.x you cannot go back to 2.3.x without reconfiguration.


As stated many configuration changes are in 2.4.x so please backup your existing MISFITconfig.txt before running 2.4.x and after you run it re-check ***ALL*** your configuration values as many will be set back to defaults.



Under the Process Control menu you can now "Force an Automated Work Fetch" which bypasses all conditional checks and executes the fetch using the configured values. [if you fetch via External OS command the command will be executed].




Get from [url]http://mersenneforum.org/misfit/[/url]

I've tested just about all the manual things, and so far it's all good except external fetching from G72.

I'm set for an automatic upload @ 1640 MST, I'll let you know how that goes.

The external fetch from G72 doesn't work as configured before. I keep getting a 100. If I run it from he cmd prompt, it runs fine. The batch file didn't work either from within MISFIT.

Lots of good updates...

[QUOTE=flashjh;325695]I've tested just about all the manual things, and so far it's all good except external fetching from G72.

I'm set for an automatic upload @ 1640 MST, I'll let you know how that goes.

The external fetch from G72 doesn't work as configured before. I keep getting a 100. If I run it from he cmd prompt, it runs fine. The batch file didn't work either from within MISFIT.

Lots of good updates...[/QUOTE]

If the GPUfetcher is in the same directory as MISFIT you don't need a bat file at all. If not in the same directory the paths must be fully specified. Your bat file may need to execute a CD/Chdir to the fetcher's directory before calling the fetcher. In other words the working directory for MISFIT is the MISFIT directory..

Keep me posted.

thx

Everything is in the same directory as misfit.exe. I didn't change anything for 2.4b2 except updating misfit.exe and running the configuration again. I went back to the old config and 2.3.2 and everything works fine. I tried 2.4b2 again and I get a 100...

Edit:
I just manually assigned the work I got in the cmd test above. MISFIT put the work into the workers, but it's not showing the additional number in FactorRows. I tired closing and restarting and also removing the blank line in the worktodo file. Nothing changed...

[QUOTE=flashjh;325702]Everything is in the same directory as misfit.exe. I didn't change anything for 2.4b2 except updating misfit.exe and running the configuration again. I went back to the old config and 2.3.2 and everything works fine. I tried 2.4b2 again and I get a 100...

Edit:
I just manually assigned the work I got in the cmd test above. MISFIT put the work into the workers, but it's not showing the additional number in FactorRows. I tired closing and restarting and also removing the blank line in the worktodo file. Nothing changed...[/QUOTE]


I'll walk through the code tonight.

Thanks for testing!

I have another update:

I have ~6500 GHz days of work in my files in 274 lines. MISFIT is set to the settings in the attachment. Since earlier, every hour MISFIT is still running the external fetcher is running even though it shouldn't be.

I thought it wasn't getting assignments because it's returning a 100, but [B]it is[/B] getting assignments (I have it set to 4 right now). But, it's not putting them into the stage file nor is it assigning/balancing them.

After I manually grabbed the assignments given to me throughout the day and put them in add work and assigned work, it put them into the files as expected. It also found 4 duplicates that I had checked. But, for some reason it missed 4 more duplicates (G72 showed 274, MISFIT showed 278). Once I clicked Balance All, it found the other 4.

Let me know if you have any tests you need me to run.

Edit:
I updated the picture because I think I may see an issue? The circled areas seem to conflict with each other now, maybe? I'm expecting MISFIT to only fetch assignments when below 375GHz Days AND MISFITWorktodo is below 200, but it's doing OR, right? I've set work source to disabled, for now ;)

One item at a time.
 

I have an answer for the 100 error message.
MISFIT is locking MISFITworkToDo.txt before it kicks off Gpu72WorkFetcher.exe at which time Gpu72WorkFetcher tries to lock MISFITWorkToDo.txt and cannot and Error 100 is generated.
FIX... [B]MISFIT should not lock the file when running external fetcher[/B]. It is up to the external fetcher to lock the file since MISFIT is not actually doing the file IO. No IO log is created in the Gpu72WorkFetcher log directory because nothing was fetched. [EASY FIX...]

other items to note:
Gpu72WorkFetcher has a config file with a value [B]ReloadWhenBelow[/B]:xxx if the destination file has more than this number of rows Nothing will get fetched regardless of the MISFIT configuration.


[B]Draining [/B]work from MISFITworkToDo is based on the remaining GHZDays as calculated from all the workToDo.txt files. [B]Fetching[/B] work is based on the number of rows in MISFITWorkToDo. They are mutually exclusive, but can be harmonized via configuration to ensure MISFITworkToDo.txt always has enough work to service any Drain event. I recommend MISFITWorkToDo.txt have several days of reserve work in it in-case there is a prolonged inability to fetch new work. (totally my opinion).

[B]EDIT:[/B]
Duplicate detection only occurs during work balance, either automated or manual. It is not part of any other event.
Fetching via Gpu72WorkFetcher.exe won't be needed much longer for MISFIT users, but this testing did find a bug so good job and don't worry too much about how the two will interact long term since they won't need to.

Let me fix the file-locking conflict and resume from there. Stand by 30 mins!

VERSION 2.4.B3
 

VERSION 2.4.B3
Fixed bug where MISFIT was locking MISFITworkToDo.txt before launching external work fetcher. The external work fetcher will be performing the IO and is responsible for locking the file, not MISFIT. Because MISFIT locked the file before the external fetcher was started there was a lock collision and the external fetcher was forced to quit with error 100.


get from [url]http://mersenneforum.org/misfit/[/url]

Before, per the screen shot of 2.3.2, all actions (draining and fetch) were based on GHz days only. It's separated now in 2.4x, so it behaves differently. I'll take a look at everything end retest once I have it configured.

[QUOTE=flashjh;325733]Here is my G72 config file:
[CODE]StagingFile:MISFITWorkToDo.txt
ReloadWhenBelow:500
FetchCount:4
Low:0
High:100000000
Pledge:73

#OPTION
#WhatMakesSense = 0
#LowestTFLevel = 1
#HighestTFLevel = 2
#LowestExponent = 3
#OldestExponent = 4
Option:1

ReplaceNAwithDate:true[/CODE]
Reload below is set to 500. Is it checking worktodo or the staging file?[/QUOTE]

It is counting rows in [B]MISFITWorkToDo.txt[/B] and if below 500 a fetch will occur and populate [B]MISFITWorkToDo.txt[/B] with any returned assignments.

[QUOTE=flashjh;325733]Before, per the screen shot of 2.3.2, all actions (draining and fetch) were based on GHz days only. It's separated now in 2.4x, so it behaves differently. I'll take a look at everything end retest once I have it configured.[/QUOTE]

Yes they were combined before, but normalizing the approach to fetching work split them into distinct operations. Sorry I did not sufficiently clarify that in my release notes.

No need to apologize, I just realized why it wasn't working as I expected.

Got the fetcher working. Now I just need to burn through 6000 GHz Days of work so it will start getting assignments again.

So far everything else seems to be on track. I'll let you know if anything else comes up.

[QUOTE=flashjh;325745]I just need to burn through 6000 GHz Days of work[/QUOTE]
How many hours? 5? 6? 30 minutes? :razz:

[QUOTE=LaurV;325751]How many hours? 5? 6? 30 minutes? :razz:[/QUOTE]

He's probably almost done now!

I wish, it's 2.5 days :smile:

VERSION 2.4.B4. improved remote control panel
 

Minor changes to the remote control panel greatly improved its usability

When do you sleep? ;)

[QUOTE=flashjh;325793]When do you sleep? ;)[/QUOTE]
Between work and a "Learning to Program with C#" class.:geek:

I was wondering, along with the submit on a schedule, can you have an option for MISFIT to email a recap of what is submitted/remaining. Nothing fancy but something like:

Exponents Done: xxxx
GHzDays Done: xxxx
Factors Found: xxxx
Exponent Rows: xxxx
GHzDaysToDo: xxxx

Since I never see the submission anymore and I'm not around when it does, I'd like to know what's getting done each day. Since I only submit once day, I'll only get one email, but I suppose others might get more?

Edit: BTW - After I got everything reconfigured, the external fetcher is working perfectly. It hasn't actually needed to "drain" the results yet, so I'll let you know how that goes in a couple days.

[QUOTE=flashjh;325867]I was wondering, along with the submit on a schedule, can you have an option for MISFIT to email a recap of what is submitted/remaining. Nothing fancy but something like:

Exponents Done: xxxx
GHzDays Done: xxxx
Factors Found: xxxx
Exponent Rows: xxxx
GHzDaysToDo: xxxx

Since I never see the submission anymore and I'm not around when it does, I'd like to know what's getting done each day. Since I only submit once day, I'll only get one email, but I suppose others might get more?

Edit: BTW - After I got everything reconfigured, the external fetcher is working perfectly. It hasn't actually needed to "drain" the results yet, so I'll let you know how that goes in a couple days.[/QUOTE]

I think the easier solution is to remove all the automation features from MISFIT so you can get back in touch with your environment. :no:

It'll be a scheduled item titled "summary_email" so you decide when it goes out.

[QUOTE=swl551;325874]I think the easier solution is to remove all the automation features from MISFIT so you can get back in touch with your environment. :no:

It'll be a scheduled item titled "summary_email" so you decide when it goes out.[/QUOTE]
I thought about that, but I wasn't sure if it needed to be tied to the submission to capture before/after info. Thanks.

BTW - No rush, I thought about it a while back and just got around to asking.

[QUOTE=flashjh;325875]I thought about that, but I wasn't sure if it needed to be tied to the submission to capture before/after info. Thanks.

BTW - No rush, I thought about it a while back and just got around to asking.[/QUOTE]

How about a phone app instead showing the real-time stats?

[QUOTE=swl551;325879]How about a phone app instead showing the real-time stats?[/QUOTE]

:smile:

[QUOTE=flashjh;325880]:smile:[/QUOTE]

Phone app.....:ignore::ignore::ignore:

I don't know how to write one, but I figured I'd jerk your chain for fun!

[QUOTE=swl551;325881]Phone app.....:ignore::ignore::ignore:

I don't know how to write one, but I figured I'd jerk your chain for fun![/QUOTE]
Believe me, I'd thought about that a while ago... before MISFIT I was looking into ways to publish the results.txt and workdoto.txt files to a website so I could monitor everything.

[QUOTE=flashjh;325882]Believe me, I'd thought about that a while ago... before MISFIT I was looking into ways to publish the results.txt and workdoto.txt files to a website so I could monitor everything.[/QUOTE]
Ask Chalsall if he'll build a page for us. MISFIT can POST the current values to the page and you can view them from there. He builds nice to view pages that is for sure.

[QUOTE]MISFIT can POST the current values to the page and you can view them from there.[/QUOTE]

What format/data are you thinking?

[QUOTE=flashjh;325888]What format/data are you thinking?[/QUOTE]
POST values on the page would be
[CODE]
ExponentsDone
GHzDaysDone
FactorsFound
ExponentRows
GHzDaysToDo
[/CODE]

MISFIT executes a POST to the submission page. The data is stored in a table associated with your account (like all your other stats). A page for viewing presents the info.

Decided to give a try to misfit. Made a VBox installation of XP, until I can see the sources (well, you know, I am a bit paranoid) and eventually compile them by myself.

It seems to work as promised, except it is always telling me that I have no "no factor" lines in the results file, and I will "have problems" reporting only "factor found" lines (that's the trick with PrimeNet sometimes confusing the algorithms used to find the factors, if no "no factor" line is present in the result file).

That is a very nice feature to notify me,.... except that the reality was that I only had "no factor" lines, and no "factor found" lines (that is, vice-versa!).

[edit: this also happens when I try to export the results in a new file, and generally, misfit does not recognize the format of my lines in the result files]

However, I use an output format with "username/computername" as P95 outputs (this is an option of mfaktc, how to output the results) because that helps me with batch reporting, and there is no way around it, i.e. I will not change that, as I continue to report/fetch by hand/batch files, not yet trusting misfit so much to input passwords into it, again paranoid, sorry.

Is this output format the reason why Misfit does not recognize the "no factor" lines? Can the behavior be changed from the settings? Or the internal parser need to be reviewed?

Any other suggestion?

[QUOTE=LaurV;325913]Decided to give a try to misfit. Made a VBox installation of XP, until I can see the sources (well, you know, I am a bit paranoid) and eventually compile them by myself.

It seems to work as promised, except it is always telling me that I have no "no factor" lines in the results file, and I will "have problems" reporting only "factor found" lines (that's the trick with PrimeNet sometimes confusing the algorithms used to find the factors, if no "no factor" line is present in the result file).

That is a very nice feature to notify me,.... except that the reality was that I only had "no factor" lines, and no "factor found" lines (that is, vice-versa!).

[edit: this also happens when I try to export the results in a new file, and generally, misfit does not recognize the format of my lines in the result files]

However, I use an output format with "username/computername" as P95 outputs (this is an option of mfaktc, how to output the results) because that helps me with batch reporting, and there is no way around it, i.e. I will not change that, as I continue to report/fetch by hand/batch files, not yet trusting misfit so much to input passwords into it, again paranoid, sorry.

Is this output format the reason why Misfit does not recognize the "no factor" lines? Can the behavior be changed from the settings? Or the internal parser need to be reviewed?

Any other suggestion?[/QUOTE]

Since your results lines start with "username/computername" MISFIT does not find the expected .StartsWith phrases and considers the lines invalid.

No work around sans code change.

Emailing of global stats before upload
 

Flashjh see screenshot.

This is TOO simple. We should have done this long ago.


I'll have this in a build later today. GREAT IDEA.

[QUOTE=swl551;325961]Flashjh see screenshot.

This is TOO simple. We should have done this long ago.


I'll have this in a build later today. GREAT IDEA.[/QUOTE]

And here is the output send via email.
[CODE]
MISFIT Global Work Status Report
****** REMAINING WORK INFORMATION ******
GhzDzToDo:4,262.346
ExpRows:50
Staged Work:6
****** COMPLETED WORK INFORMATION ******
GhzDzDone:89.854
ExpsDone:8
FactorsFound:0
Results Bytes:608

Sent by MISFIT 2.4.b4 from computer THOR at 1/26/2013 11:19:55 AM
[/CODE]

Perfect

That is very cool.

:tu:

VERSION 2.4.b5
 

VERSION 2.4.b5
Added LLDC stats to the Leaderboard

Added option to email Global Work Stats prior to Auto Results Uploading

Get from [URL]http://mersenneforum.org/misfit/[/URL]

*** in this version clicking the [B]Export Results[/B] button will also send the email, but that will be removed in final version ***
This will allow you to generate the email without waiting for automated upload, but since you actually looking at the screen it doesn't need to be emailed to you. (that why it will be removed)

Is it possible to reduce upload time minimum from 4 hours to let's say, 2?

Other than that... latest working great :smile:

My daily upload just occurred and the email was perfect, thanks!

[QUOTE=kracker;326028]Is it possible to reduce upload time minimum from 4 hours to let's say, 2?

Other than that... latest working great :smile:[/QUOTE]

Yes, configure scheduled uploads to manually achieve an aggressive upload frequency.

2.4.b8
 

Added calculation of GhzDays from the MISFITworkToDo.txt file. This information also now in the Emailed Global Stats report. This should help you in determining if you are fetching work in suitable quantities to have a several day reserve. (imagine how mad you will be if you fetch so frequently that you have a near real-time dependency on fetching, and somehow access to to your work source is cut-off for an extended period of time) [hint hint, several day reserve!]


Added file-locking to all I/O scenarios where MISFITworkToDo.txt is involved. A few conditions didn't have locking and that was OK when it was only possible to load the MISFITworkToDo.txt via the UI. Now with randomly timed fetching threads adding work to the file it was at risk of an un-handled IO collision. A few new file handling methods were written to resolve this so it is definitely BETA code.

Enjoy.

EDIT:$100.00 says FLASHJH asks the work fetching trigger be changed to use calculated GhzDays from MISFITworkToDo.txt instead of number of lines in the file. (If he does ya'll pay up!)

[QUOTE=swl551;326208][hint hint, several day reserve!][/QUOTE]

An excellent hint.

At least for the next week or so... (Hint, hint... :wink:)

[QUOTE=swl551;326208]EDIT:$100.00 says FLASHJH asks the work fetching trigger be changed to use calculated GhzDays from MISFITworkToDo.txt instead of number of lines in the file. (If he does ya'll pay up!)[/QUOTE]

Well, since YOU mentioned it ;)

[QUOTE=flashjh;326230]Well, since YOU mentioned it ;)[/QUOTE]
It does make a lot of sense so I'll have that update avail later tonight.

2.4.b9 (last release for the weekend)
 

VERSION 2.4.b9
Changed the determination of when to fetch work from the number of rows in MISFITworkToDo.txt to the calculation of GhDz remaining in said file. This provides a much better method for calibrating the balance between work consumption and work fetching.

This should help you in determining if you are fetching work in suitable quantities to have a several day reserve. (imagine how mad you will be if you fetch so frequently that you have a near real-time dependency on fetching, and somehow access to to your work source is cut-off for an extended period of time) [hint hint, [U]several day reserve![/U]]

Get from [url]http://mersenneforum.org/misfit/[/url]

When all these improvements start to settle down I think I will look into becoming a MISFIT...when I take an extended trip this spring I want my system to be in automated fetch/report mode.

[QUOTE=Chuck;326331]When all these improvements start to settle down I think I will look into becoming a MISFIT...when I take an extended trip this spring I want my system to be in automated fetch/report mode.[/QUOTE]


As a weekend warrior frequent releases are what we get until:
1. There is no more functionality to add
2. I get bored

However 2.4's next build (eta is 2/3) will have native GPU72 fetching completed. Now is the time to get on-board.

After that I might take a few weekends off.

[QUOTE=swl551;326349]As a weekend warrior frequent releases are what we get until:
1. There is no more functionality to add
2. I get bored[/QUOTE]

That's my spirit :razz:

[QUOTE=swl551;326349]As a weekend warrior frequent releases are what we get until:
1. There is no more functionality to add
2. I get bored

However 2.4's next build (eta is 2/3) will have native GPU72 fetching completed. Now is the time to get on-board.

After that I might take a few weekends off.[/QUOTE]

You may have left one off:

3. Jerry stops asking for changes

As 3 is easily trumped by 2, however, 3 really isn't all that important ;)

DEFECT in 2.4b8 & b9
 

PROBLEM: MISFITworkToDo.txt has more rows than expected and the extra rows are duplicates.

CONDITIONS: MISFIT drains work from MISFITworkToDo.txt and if there is [B]left-over[/B] work (not assigned out) the left-overs are appended to MISFITworkToDo.txt instead of replacing MISFITworkToDo.txt

ROOT CAUSE: MERGE_WORK method accidentally calling the file.AppendWithLock instead of the file.ReplaceWithLock method. Problem introduced after refactoring some methods and then calling the wrong method.

SOLUTION: SHORT TERM: STOP MISFIT....

STAND BY........ MORE INFORMATION.... 60 MINUTES.....

<ALL STOP>

A newbie question
 

I tired the MISFIT today for the first time.

After the initial configuration I pushed “Add Work”

It reserved 25 exponents.

I pushed “Assign Work”

I went to Prices Control – Start Batch, and mfaktc started cranking. But when it was done with the 25 exponents, nothing happened.

How do I configure MISFIT to get more work and submit results automatically without my intervention?

Thank you

URGENT FIX 2.4.b12 (required for 2.4b8 and 2.4b9)
 

[B]NOT REQUIRED IF YOU RUNNING VERSIONS PRIOR TO 2.4b8[/B]

VERSION 2.4.b12 (beta)
Fixed bug where MISFIT drains work from MISFITworkToDo.txt and if there is left-over work (not assigned out) the left-overs are appended to MISFITworkToDo.txt instead of replacing MISFITworkToDo.txt.

How to fix if you have duplicates: (visually inspect MISFITworkToDo.txt in an editor, external editor is best since it can go full screen)

You can manually edit MISFITworkToDo.txt and try to remove the duplicates, but some of them may already be assigned out to WorkToDo files.
or
1.) DOWNLOAD 1.4.B12 and start it up.
2.) From PROCESS CONTROL MENU select SUSPEND AUTOMATION (10 minutes)
3.) Click the ASSIGN WORK Button and assign out ALL work in one shot. (duplicates may be detected which is what we are looking for)
4.) From the OPERATIONS menu select FORCE WORK BALANCE (even if you have only one mfaktX instance). All duplicates will be tossed!
5.) Now you can either leave as is or you can reclaim work from the WorkToDo.txt (with an editor) and add it back into MISFITWorkToDo by using the ADD WORK button.
6.) UN-SUSPEND AUTOMATION... you back up and running.


Get from [url]http://mersenneforum.org/misfit/[/url]

[QUOTE=TObject;326415]I tired the MISFIT today for the first time.

After the initial configuration I pushed “Add Work”

It reserved 25 exponents.

I pushed “Assign Work”

I went to Prices Control – Start Batch, and mfaktc started cranking. But when it was done with the 25 exponents, nothing happened.

How do I configure MISFIT to get more work and submit results automatically without my intervention?

Thank you[/QUOTE]

Go through all the configuration options. The defaults should work fine but I think you will need to get more than 25 assignments since GIMPS is assigning out a lot of 66,68 bit levels recently and they will process VERY fast. You might need hundreds to last just a few hours or you could set the [B]Re-write TO bit level to 2^[/B] = 71 and they will process to higher levels, taking more time and giving you the appropriate additional credits.

A reasonable number of GhDzToDo for one day of processing is 450 (for a GTX 570). You should have work assigned out and work staged in enough quantity that MISFIT can just keep things going. Knowing your cards output and your system capabilities are key to making advanced decisions on MISFIT configuration.

I hope that helps.

Thanks :smile:

Jump from beta 9 to 12?

[QUOTE=kracker;326420]Thanks :smile:

Jump from beta 9 to 12?[/QUOTE]

Yes, internal build numbers that were [B]secret[/B]:stirpot:

What is “draining?”

If I leave “Drain MISFITworkToDo…” setting at “0” it does not affect automatic work fetching, does it? Or do I need to set this to get the things rollong?

Thank you.

What does “Export Results” do? I clicked it and it said I had nothing to export even though I have many lines in my results.txt.

Thank you