|
Post by Sniffles on Jul 14, 2015 20:29:37 GMT
About the cell loadding. The game uses spawn lists for generating NPCs and items and for creating containers contents (in the ones that are resetted). The trick here, is that after the first time, the list is generated from the last generated item instead of from the initial list. This causes two problems, first, the game won't ever erase the reference of you having entered the cel and second, if a wolf follow you from windhelm otskirts to your home in whiterun, the wolf (or whichever creature in it's list) will spawn in your home each time aftewards, until the wolf exit the cell again. That list generating a list confuses the heck out of me. Couldn't that eventually go on indefinitely? A List item generating a list item generating a list item generating a list item etc? Also!! Right now I have a screw up going if you could explain, please? When PC enters a new cell where the autosave goes off, after a certain period of time the game will suddenly drop to desktop crash as long as I am running after the autosave. Sometimes just a few seconds but often after 5 or 10 minutes of game play. But if I stop everything and make a manual save everything will be okay. What is going on there? For example, I left Riften and was walking towards Falkreath, somewhere, completely at random the game drops to desktop. It's inevitable, always before getting to Alchemists shack. But if I stop anywhere along the way, make a save then continue on, it keeps running just fine. OR, I leave Lakeview and do some smithing just outside, within about 3 to 5 minutes it drops to desktop. If I do a save manually there's no problem. BUT!!! Sometimes the manual save also gets fooked up and drops to desktop anyway. In that case I restart, load the last saved game, make another manual saved game, exit the game then load that latest manual save, it will work fine with no crash to desktop until the next autosave then it's back at it again. What is causing this?
|
|
|
Post by Cybersquirt on Jul 15, 2015 4:23:04 GMT
Just a quick note: in testing that so-called 30 day trick, my saves did drop 10k after 10 days but overall there is no "30 day trick". At least, there isn't anymore (on PC).. hmm, unless there's more to it than just standing there and advancing the clock 30 days..
|
|
|
Post by amgepo on Jul 15, 2015 14:59:49 GMT
About the cell loadding. The game uses spawn lists for generating NPCs and items and for creating containers contents (in the ones that are resetted). The trick here, is that after the first time, the list is generated from the last generated item instead of from the initial list. This causes two problems, first, the game won't ever erase the reference of you having entered the cel and second, if a wolf follow you from windhelm otskirts to your home in whiterun, the wolf (or whichever creature in it's list) will spawn in your home each time aftewards, until the wolf exit the cell again. That list generating a list confuses the heck out of me. Couldn't that eventually go on indefinitely? A List item generating a list item generating a list item generating a list item etc? That's just me not explainning things propperly. There is not a list generating another list. It's just a reference being picked up from a list of references. The first time, it's generated on the spawn point, but afterwards it's spawned where the previously generated item is cleanned. When saving, the CPU and disk loads are increased. That affects performance and stability. When you save manually, you tend to do it while not doing any other thing, specially if you save from the menu. Autosaves though, tend to happen when you are doing something *and* when the system is already busy with things like cell changes. It's the worst posible time to do it. In other games, that's not a big deal, but as TES and Fallout 3+ are pretty unstable on their own, reducing stability further, can easily lead to CTDs. There is an aditional factor here too. The cell buffer, which stores info from cells you visit recently, to load them quickly if you return to them. It increase load speed, but also memory load, which is a real problem, considering the game can only handle 4 Gb assuming you are using a 64 bit operating system (Thant's in theory, in practice I think it crashes before reaching 3 Gb), regardless the ammount of memory you have installed. When the RAM reach it's max allowed usage, the game crashes. Saving a game, tends to flush the buffer, but a more reliable method is to bring the console and purge the cell buffer, typying PCB Just a quick note: in testing that so-called 30 day trick, my saves did drop 10k after 10 days but overall there is no "30 day trick". At least, there isn't anymore (on PC).. hmm, unless there's more to it than just standing there and advancing the clock 30 days.. The 30 day trick will work on each cell once you return to it after the 30 day period. If you never return, the cell won't be ever resetted.
|
|
|
Post by Sniffles on Jul 15, 2015 21:43:32 GMT
Thanks, Amgepo! Combining that info with what my tech has been saying is starting to make sense to my terminally softened brain.
I've been doing what he recommended with saving. Find a location without any activity going on like monsters roaming, go into 1st person camera, look at a close static object, not a distant high LOD, do the manual save and don't touch keyboard or mouse for 5 seconds. This also contributes to what I call player vertigo where we load a saved game and find the PC staring at a patch of road or a rock and you have to realign the camera and usually open the map to figure out where the heck you are.
He talked about the difference between Linux and Windows and how Windows actually promotes crashes by the way it saves and spools data. It sounds like the ultimate game stability mod is in order. Force the game and Windows to emulate the Linux 'DB dump'. Would that be possible?
So with a really safe game save would it be best to do the PCB then do the save?
|
|
|
Post by amgepo on Jul 16, 2015 9:17:39 GMT
Maybe a SKSE pluggin could change the way the engine saves games, but the problem is more how unstable the game is, than how much saving increase unstability. Not touching anything while saving is a good advice, better yet if you ensure the engine itself won't be processing meanwhile (just not using quick saves. Just use the menu save, as the menu stops game activity).
Using PCB along saving won't help. Both needs to be done when things are calm (no combat, no cell changing, no quick looking around with fast mouse movements...) And both did purge the buffer. Propperly done, both increase game stability by ensuring the buffer didn't overflow and unpropperly used (being used when things aren't calm) both could crash the game.
The best stability advices related to saving game is deactivating autosave, not using quicksaves (Also exiting the game before loadding a save, to avoid save corruption. which could eventually crash the game between other nasty things), using the menu for game saving and not touching anything while the game saves.
|
|
|
Post by Sniffles on Jul 16, 2015 12:39:26 GMT
Maybe a SKSE pluggin could change the way the engine saves games, but the problem is more how unstable the game is, than how much saving increase unstability. Not touching anything while saving is a good advice, better yet if you ensure the engine itself won't be processing meanwhile (just not using quick saves. Just use the menu save, as the menu stops game activity). Using PCB along saving won't help. Both needs to be done when things are calm (no combat, no cell changing, no quick looking around with fast mouse movements...) And both did purge the buffer. Propperly done, both increase game stability by ensuring the buffer didn't overflow and unpropperly used (being used when things aren't calm) both could crash the game. The best stability advices related to saving game is deactivating autosave, not using quicksaves (Also exiting the game before loadding a save, to avoid save corruption. which could eventually crash the game between other nasty things), using the menu for game saving and not touching anything while the game saves. Thanks very much, Amgepo. It's making sense now. My tech mentioned something additionally. That the computer needs to be given time to get data out of the virtual memory and if the virtual memory is on a badly fragmented hard disk the longer the time it should be given.
|
|
|
Post by amgepo on Jul 16, 2015 21:03:05 GMT
Must admit I'm not sure how Windows handle virtual memory ussage when RAM is missused. One would think that it wouldn't use virtual memory at all, but that would be giving too much credit to MS. Most probably it will use it extendedly, as with any aplication able to all the system's RAM, making what your tech say something to consider when saving and loadding. Anyway, the saved game itself does use the disk and can indeed be fragmented. So, the question is if the advice is good or very good.
|
|
|
Post by Sniffles on Jul 17, 2015 6:20:25 GMT
Must admit I'm not sure how Windows handle virtual memory ussage when RAM is missused. One would think that it wouldn't use virtual memory at all, but that would be giving too much credit to MS. Most probably it will use it extendedly, as with any aplication able to all the system's RAM, making what your tech say something to consider when saving and loadding. Anyway, the saved game itself does use the disk and can indeed be fragmented. So, the question is if the advice is good or very good. From techy: " Re your virtual memory use Q. There is no simple answer but I will give a rough idea. Windows always reserves memory for itself. It has two primary basic allocation methods. First it always loads itself and some critical tools into memory. It also reserves some memory for other ciritcal or important tools it might need. This is the OS itself. To this it adds some additional memory to be kept in reserve. This it set and unalterable when the system boots. Next it sets aside another block of memory for control functions of programs. Some TSR programs get loaded here, and various reference pointers to other programs. The system clock, audio, systray and so on. These are variable programs that aren't critical to the operating system but are given higher priority and memory allocations. For example, while running your Skyrim game the desktop, the clock, audio manager, antoi virus and so forth are still running and in memory. When a program crashes the desktop and all these support programs instantly come back without hard drive retrieval required.
So a huge amount of your computer memory is either always in use or reserved at all times. Running a memory check program will return a certain amount of memory free but this in not the case. It is free, but still reserved for the systems axillary operations.
Then you start up a large program like your game. Windows will not allocate all the available memory to this program, only a given percentage. This is to prevent memory full crashes. When the OS sees memory usage exceeding a certain amount it automatically spools some of the RAM memory to the virtual memory. How much and how often can not possibly be predicted. It all depends on whatever else Windows is running in it's two primary allocations.
So your safest guess as to whether virtual memory is being used is yes it is. All the time. It's inevitable. Even if RAM is available Windows won't permit it's use. It always keeps a buffer in case of memory becoming near full. Remember, full memory plus 1 bit and the computer crashes. It's this memory control and allocation, perfected over the years and various versions of the Windows OS that has lent the greater degree of system stability, drastically reduced the instances of BSOD, but much more often you get program crashes and drops to desktop.
Before the memory fills up which will cause the entire computer to crash and it is suspected the virtual memory will not be able to spool fast enough the OS simply terminates the major program that is active when that critical near full and no end in sight state is detected." And: " Right now Microsoft is banking on solid state drives. Their future OS that expects a solid state drive is already on the drawing board. The old conventional drive is future history. The main reason it is being phased out as fast as possible is this virtual memory allocation. Right now Windows does as it always has, spraying data at the C: drive. Most computers are set up for virtual memory to use this drive as well, finding room when and where it can. Defragmentation remains a critical factor. With solid state drives this becomes irrelevant. The drive can access any memory bit anywhere at maximum speed which is much faster that a conventional drive ever could perform. When this happens, solid state drives becoming the norm and the operating system no longer having to bow to the slow mechanics of the old drives, you could have 20 or 50 GB of virtual drive and the system will never have memory full allocation problems ever again. Give it another 2 years and people will be breaking down their old hard drives for the rare earth magnets." I hope that makes sense to somebody. It is going to take me a week to digest it.
|
|
|
Post by Cybersquirt on Jul 17, 2015 22:24:46 GMT
ssd - drool! I've yet to have a corrupt save - knocks on wood - but I only run a handful of mods and have a reliable system in general. Old, but reliable.
|
|
|
Post by amgepo on Jul 18, 2015 0:34:07 GMT
Must admit I'm not sure how Windows handle virtual memory ussage when RAM is missused. One would think that it wouldn't use virtual memory at all, but that would be giving too much credit to MS. Most probably it will use it extendedly, as with any aplication able to all the system's RAM, making what your tech say something to consider when saving and loadding. Anyway, the saved game itself does use the disk and can indeed be fragmented. So, the question is if the advice is good or very good. From techy: " Re your virtual memory use Q. There is no simple answer but I will give a rough idea. Windows always reserves memory for itself. It has two primary basic allocation methods. First it always loads itself and some critical tools into memory. It also reserves some memory for other ciritcal or important tools it might need. This is the OS itself. To this it adds some additional memory to be kept in reserve. This it set and unalterable when the system boots. Next it sets aside another block of memory for control functions of programs. Some TSR programs get loaded here, and various reference pointers to other programs. The system clock, audio, systray and so on. These are variable programs that aren't critical to the operating system but are given higher priority and memory allocations. For example, while running your Skyrim game the desktop, the clock, audio manager, antoi virus and so forth are still running and in memory. When a program crashes the desktop and all these support programs instantly come back without hard drive retrieval required.
So a huge amount of your computer memory is either always in use or reserved at all times. Running a memory check program will return a certain amount of memory free but this in not the case. It is free, but still reserved for the systems axillary operations.
Then you start up a large program like your game. Windows will not allocate all the available memory to this program, only a given percentage. This is to prevent memory full crashes. When the OS sees memory usage exceeding a certain amount it automatically spools some of the RAM memory to the virtual memory. How much and how often can not possibly be predicted. It all depends on whatever else Windows is running in it's two primary allocations.
So your safest guess as to whether virtual memory is being used is yes it is. All the time. It's inevitable. Even if RAM is available Windows won't permit it's use. It always keeps a buffer in case of memory becoming near full. Remember, full memory plus 1 bit and the computer crashes. It's this memory control and allocation, perfected over the years and various versions of the Windows OS that has lent the greater degree of system stability, drastically reduced the instances of BSOD, but much more often you get program crashes and drops to desktop.
Before the memory fills up which will cause the entire computer to crash and it is suspected the virtual memory will not be able to spool fast enough the OS simply terminates the major program that is active when that critical near full and no end in sight state is detected." And: " Right now Microsoft is banking on solid state drives. Their future OS that expects a solid state drive is already on the drawing board. The old conventional drive is future history. The main reason it is being phased out as fast as possible is this virtual memory allocation. Right now Windows does as it always has, spraying data at the C: drive. Most computers are set up for virtual memory to use this drive as well, finding room when and where it can. Defragmentation remains a critical factor. With solid state drives this becomes irrelevant. The drive can access any memory bit anywhere at maximum speed which is much faster that a conventional drive ever could perform. When this happens, solid state drives becoming the norm and the operating system no longer having to bow to the slow mechanics of the old drives, you could have 20 or 50 GB of virtual drive and the system will never have memory full allocation problems ever again. Give it another 2 years and people will be breaking down their old hard drives for the rare earth magnets." I hope that makes sense to somebody. It is going to take me a week to digest it. More or less what I suspected, except the SSD and rare earth hard drives. I guess MS is relying on SSD being more dinamic in near future. Certainly, using rare earth magnets, would allow smaller hard drives than ferromagnetic ones and the lesser use of materials could compensate at least to some extent it's expensiveness, but wouldn't different memory cells have too much magnetic influence each other? I find quite intriguing how they plan to compensate that part. I think I will make some browsing on this matter on the web. EDIT: Just being stupid. Just maintaining the distance between smaller magnets is more than enough for avoiding problems, I was depicting a sice decrease as a whole, as the only posibility. ssd - drool! I've yet to have a corrupt save - knocks on wood - but I only run a handful of mods and have a reliable system in general. Old, but reliable. Probably you are not performing actions prone to cause save corruption like loadding without exiting the game, or not reverting to a save prior to a mod instalation before uninstalling it (this is disastrous for many mods )
|
|
|
Post by Cybersquirt on Jul 18, 2015 0:48:56 GMT
Probably you are not performing actions prone to cause save corruption like loadding without exiting the game, or not reverting to a save prior to a mod instalation before uninstalling it (this is disastrous for many mods ) I used quick save and load, I load saves w/o exiting, and I have auto-save enabled. However, I didn't run any mods for quite a while. Dunno if it matters, except for general game performance: I have oodles of HD space, including on c:\ and have a windows managed swap file (currently 25gb).. maybe the i7-920 and 24gb of memory (or 4gb of video memory) don't hurt either. Like I said though: knocking on wood. edit: Oops! Except for Vilja -- Vilja and the Divorce thingy were the only mods.
|
|
|
Post by Sniffles on Jul 18, 2015 4:33:52 GMT
From Techy: "What is a corrupt save? What isn't? Data is data. That's an intrinsic part of any program negine's job: keeping the data sorted and streaming properly. Refer back to the earliest forms of programming where traps, error levels, were used intentionally as a normal part of the program. I seriously doubt any working file like a saved game is entirely error free and even if it was, the CPU, buffers, supporting devices, disk drives and GUI architecture certainly is far from pixel perfect. Checksums, CRCs and on out correct and control the errors and the programs go on their merry way.
Re Solid State Drives. SRAM has been around for about 40 years. At first discounted due to it being much slower than DRAM but with the perfection of silicon devices and nanotechnology it has established itself as a critical part of any modern computer system. It wasn't given the development and lags about 10 years behind DRAM but it may eventually catch up and RAM as we know it, along with mechanical hard drives, will become things of the past. A computer with a bank of several terabytes of DRAM speed SRAM could conceivably operate at close to the speed of the CPU. Imagine a computer that displays the desktop the second it is turned on and any given program regardless of size loads instantly. The computer itself would have only three basic components, power supply/motherboard/CPU, memory block and GPU-user interface."
"Future computer architecture. First and foremost, power and heat has always been an issue. The CPU used to be a power hog, disk drives would warm the room, the power supply itself came in it's own cabinet and RAM was constantly sucking on the juice. With todays computer we can see the trend with ease. The CPU voltage and power consumption is a fraction of the old 8088+, SSDs only use power on demand and the same for SRAM. I built up a 12 KVA power supply long ago to kick a HP mini in the ass. Now that thing powers 18 computers and I have to give it an artificial load to get it to stabilize. The largest power hog in the modern computer is the GPU, using well over half of the power. Most likely they will, like the CPU, get lower and lower voltage requirements. A new ass kicking Nvidia is out that only burns 45 watts and many more improvements are on the horizon. 100 watt microvoltage power supplies that sit on the corner of the motherboard, tiny CPU fans or even tiny solid state cooling devices, ram and memory storage that uses no power at all when idle. It's all coming."
Amgepo: More or less what I suspected, except the SSD and rare earth hard drives. I guess MS is relying on SSD being more dinamic in near future. Certainly, using rare earth magnets, would allow smaller hard drives than ferromagnetic ones and the lesser use of materials could compensate at least to some extent it's expensiveness, but wouldn't different memory cells have too much magnetic influence each other? I find quite intriguing how they plan to compensate that part. I think I will make some browsing on this matter on the web. EDIT: Just being stupid. Just maintaining the distance between smaller magnets is more than enough for avoiding problems, I was depicting a sice decrease as a whole, as the only posibility.
Techy: "They are already using rare earth magnets, Neodymium, in most HDs. I heard something similar a little while ago about DRAM that they will never be able to get more than 64K, 16M, 64M, 256M on one silicon substrate. It's no longer an issue. They are approaching quadrillions of xsistrs on a single substrate at present. 8-Core Xeon Nehalem-EX has 2 trillion+."
|
|
|
Post by amgepo on Jul 18, 2015 15:44:30 GMT
Cibersquirt, if you haven't experienced save corruption while you load without exiting the game, either you have been lucky with your saves, or you are doing or not doing something that does influence in avoiding the corruption in your files being a problem. As Sniffles techy says, corruption is simply data that isn't in the ranks it should be. Thus, it's difficult to know how to cope with it, besides avoiding such moving out of range, but it seems you have been doing it somehow. A RAM speed HDD does sound good. I suppose we'll need to wait a lot for that, but it does sound really good
|
|
|
Post by Sniffles on Jul 18, 2015 23:16:22 GMT
I turned off all autosaves yesterday and never used quick saves. I only used the autosaves as a reminder to manually save and that turned out to be stupid. The a manual save after an autosave has the autosave corruption. As Amgepo said, manual save, exit game, restart and load the manual save. Then if you made mucking fesses like I did, repeat. No more CTDs and loading times are reduced.
So I went crazy and did as tech suggested, buying a USB memory stick and put the save games, the entire My Documents folder actually, on that drive. Now the game loads a LOT faster.
|
|
|
Post by amgepo on Jul 19, 2015 8:23:28 GMT
Glad everything works fine, but as I understand, you are exiting the game after each saving, isn't it? if so, you only need to exit the game before loadding a save, not after saving.
This is because the game didn't reset every variable unless you exit it and so, loadding a game without exiting, can put some variables the way they were before loadding, instead of the way they are supposed to be in the loadded saved game.
|
|