External drive for BISS finding

[Francescone]

I'm using a 1TB SSD in my old computer having only SATA2 (not SATA3). With my actual setup I find BISS keys in 3 to 4 minutes after Crypt8 was found. Now I'm preparing to build new computer. For various reasons, I would like to connect an external SSD via Thunderbolt 3 port instead an inside drive. Do you think I can maintain the same rate? Which external box you would suggest? Thank you.

 

[dog-man]

No idea what a Thunderbolt 3 port is, but I used to use an external sata port to connect various SSD's with tables on them and it was as fast as yours to find a key or not.

 

[kebien]

The big difference is the reading and writing speed.
With this said,if the program is in the same drive as tables,is faster,so the sata port speed woudn't matter much.
It only matters when program reside in different drive than tables.

 

[CANADA]

It only matters when program reside in different drive than tables.

what program do U mean? - windows os or program of Colibri?

 

[Francescone]

It only matters when program reside in different drive than tables.

I hope you refer to Colibri's program, not Windows! If so, it would be a good idea that I did not evaluate before. Thank you for your advice.

 

[Francescone]

No idea what a Thunderbolt 3 port is, but I used to use an external sata port to connect various SSD's with tables on them and it was as fast as yours to find a key or not.

Thunderbolt 3 is a super fast connection implemented by Apple on mac computer at first, but now it begins to be used on some Windows machine, too. I believe the standard was released and copyrighted by Intel but I'm not sure. Transfer speed is about 40 GB/s (far faster than USB 3), but connector is a common USB-C one. My original question was posted because maybe I will need an Apple computer or a "closed" (not OEM) Windows one. In both cases I would work in Windows with CSA v1 (obviously, no mac version.....). Video card solution was just found, so no problem to include any NVIDIA model in an external box connected via Thunderbolt 3 to the computer. My issue is SSD now, I hope to make some test soon. Any other suggestion on this issue is greatly appreciated. Thanks.

 

[dale_para_bajo]

I'm using a 1TB SSD in my old computer having only SATA2 (not SATA3). With my actual setup I find BISS keys in 3 to 4 minutes after Crypt8 was found. Now I'm preparing to build new computer. For various reasons, I would like to connect an external SSD via Thunderbolt 3 port instead an inside drive. Do you think I can maintain the same rate? Which external box you would suggest? Thank you.

1rst I am no expert my experience in RBT is less than 1 year. I learn from marawan32 and kebien. Thanks go to both of them.

You did not say what type of table you are using V1 or V2. But as your time is 3-4 minutes I will assume V1.

The limiting factor is Momentum not speed of HDD. I will explain, Imaging you push a Car Tire in one direction. Then wile moving you try to move it back in the opposite direction. It takes more effort than the first time. Inside the HDD you have Disk and a Head in an arm mounted just like old Plastic record player use for music many years ago. Well it take effort to move the Head in the arm to different positions. This is the main cause in the delay.

So see that the delay has nothing to do with how the HDD is connected instead is an internal Physical problem. Just by this conclusion I will expect almost same result on any type of disk connection. Now SSD have no physical arms nor head. It is created using ONLY memory, That is how it can achieve both speed in big chunk of data transfer as well Input/Output Operations Per Second (IOPS).

If you search for a drive what you are looking for is IOPS, Why? Becouse the program is not requiring High amount of data transfer. Instead it is requesting millions of 16byte read request from the table.

Se a comparison of my OLD HDD vs a 64gb Sandisk HighSepped USB. For this I use a a program called CrystalDiskMark 5.1.2. Look for it it is free.

But see how a USB2 Memory can be 4 time more faster interns of IOPS than a Internally connected HDD!!!

You are looking for Random Read

Random Read 4KiB (Q= 32,T= 1) : 0.699 MB/s [ 170.7 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 2.848 MB/s [ 695.3 IOPS]

695.3 / 170.7 amost = 4 times

C: Drive

-----------------------------------------------------------------------
CrystalDiskMark 5.1.2 x64 (C) 2007-2016 hiyohiyo
                           Crystal Dew World : [url]http://crystalmark.info/[/url]
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

   Sequential Read (Q= 32,T= 1) :    98.434 MB/s
  Sequential Write (Q= 32,T= 1) :    99.588 MB/s
  Random Read 4KiB (Q= 32,T= 1) :     0.699 MB/s [   170.7 IOPS]
 Random Write 4KiB (Q= 32,T= 1) :     1.250 MB/s [   305.2 IOPS]
         Sequential Read (T= 1) :    93.323 MB/s
        Sequential Write (T= 1) :    96.272 MB/s
   Random Read 4KiB (Q= 1,T= 1) :     0.524 MB/s [   127.9 IOPS]
  Random Write 4KiB (Q= 1,T= 1) :     1.237 MB/s [   302.0 IOPS]

  Test : 1024 MiB [C: 59.6% (118.3/198.5 GiB)] (x5)  [Interval=5 sec]
  Date : 2016/09/30 16:25:38
    OS : Windows 7 Ultimate [6.1 Build 7600] (x64)
	
	
	
USB

-----------------------------------------------------------------------
CrystalDiskMark 5.1.2 x64 (C) 2007-2016 hiyohiyo
                           Crystal Dew World : [url]http://crystalmark.info/[/url]
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

   Sequential Read (Q= 32,T= 1) :    28.837 MB/s
  Sequential Write (Q= 32,T= 1) :    18.508 MB/s
  Random Read 4KiB (Q= 32,T= 1) :     2.848 MB/s [   695.3 IOPS]
 Random Write 4KiB (Q= 32,T= 1) :     1.443 MB/s [   352.3 IOPS]
         Sequential Read (T= 1) :    28.729 MB/s
        Sequential Write (T= 1) :    18.246 MB/s
   Random Read 4KiB (Q= 1,T= 1) :     2.865 MB/s [   699.5 IOPS]
  Random Write 4KiB (Q= 1,T= 1) :     1.366 MB/s [   333.5 IOPS]

  Test : 1024 MiB [E: 62.9% (72.7/115.7 GiB)] (x5)  [Interval=5 sec]
  Date : 2016/09/30 16:47:45
    OS : Windows 7 Ultimate [6.1 Build 7600] (x64)

USB HARDDRIVE

-----------------------------------------------------------------------
CrystalDiskMark 5.1.2 x64 (C) 2007-2016 hiyohiyo
                           Crystal Dew World : [url]http://crystalmark.info/[/url]
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

   Sequential Read (Q= 32,T= 1) :    29.179 MB/s
  Sequential Write (Q= 32,T= 1) :    27.603 MB/s
  Random Read 4KiB (Q= 32,T= 1) :     0.554 MB/s [   135.3 IOPS]
 Random Write 4KiB (Q= 32,T= 1) :     1.508 MB/s [   368.2 IOPS]
         Sequential Read (T= 1) :    28.943 MB/s
        Sequential Write (T= 1) :    27.471 MB/s
   Random Read 4KiB (Q= 1,T= 1) :     0.550 MB/s [   134.3 IOPS]
  Random Write 4KiB (Q= 1,T= 1) :     1.519 MB/s [   370.8 IOPS]

  Test : 1024 MiB [J: 26.2% (255.5/976.6 GiB)] (x5)  [Interval=5 sec]
  Date : 2016/09/30 17:11:23
    OS : Windows 7 Ultimate [6.1 Build 7600] (x64)
  
  ==================
  
  -----------------------------------------------------------------------

Conclusion;
Please not that that test is for 4kib but CSA RBT only read 16 bytes. So Actual Speed is worst.

So the limiting factor is not "How is Conected" but "How is Build" memory or a physical mechanisim

 

[Francescone]

Very interesting post! Thank you very much for your explanation. Yes I knew differences between hdd and ssd, but I had no such details. I'm using V1 tables, you are right. So, if I understand correctly your post, common SSDs inside computer or connected via USB 3 or Thunderbolt 3 have more or less same performances during RBT v1 usage. This is an important point to reach my goal. Thank you again.

 

[dale_para_bajo]

...So, if I understand correctly your post, common SSDs inside computer or connected via USB 3 or Thunderbolt 3 have more or less same performances during RBT v1 usage....

Yes I am ONLY suggesting as I had not used my self a SSD to test. And The info or conclusion ONLY apply to CSA RBT due to the high amount of read request vs very little amount of data per read ( 16byte).

Clearly if you work on a different situation like copying Movies back and forth. Then the opposite apply. Notice that in my example of Movie Copy, the movie will occupy a continuous block so Head Arm does not have to move almost at all. Result then no delay due to IOPS and then ONLY the data transfer thru USB/SATA is what hold speed down.

Now OS is a problem too. OS keep reading all the time. So if you have no choice at least use a Second different normal HDD with CSA Table ONLY. In that way Win7 will be slow for a few times as it keep scanning the new HDD. But soon all scan should end and that HDD will ONLY have to do IOPS for CSA. Making it some how faster.

I am paranoid. In my case I have and extreme solution. I had strip or stop 85% of all services from win7, so less unecesary reads by OS.

And I do not have Cuda so I build my own OpenCL AMD version. I can achive 1-3 minutes with 2 pendrives. and 1-5 minutes and extra regular HDDs. Please note that I have no money to buy SDD even when I have been really tempted as we can find Small SDD for $50 USD.

 

[Francescone]

Very very interestin suggestions you know. Thanks! Please another question: how can CSA v1 work without cuda? As I said earlier, I must include my NVIDIA card in an external box to add cuda features because my new computer will have not inside. Are you telling me I do not need this? Please explain better. Thanks again.

 

[kebien]

I mean Colibri program,yes
When is installed in the same SSD as the tables,sata speed do not matter.
I use it in an external USB SSD,and searching time is the same in every PC I plug it into.It reads tables from same SSD
If I run the program installed in the PC,then it takes a very long time,since the tables are read through USB port.

 

[Francescone]

I mean Colibri program,yes
When is installed in the same SSD as the tables,sata speed do not matter.
I use it in an external USB SSD,and searching time is the same in every PC I plug it into.It reads tables from same SSD
If I run the program installed in the PC,then it takes a very long time,since the tables are read through USB port.

Very good. Now it's totally clear. So the SSD solution is simpler than I thought. Now I must understand cuda problem. If cuda is not really needed it would be more simple, but it sounds strange to me. Any new information would be appreciated here. Thanks.

 

[C0der]

@kebien:
It shouldn't matter what drive the program is on.
If so, its either a bug or some odd setup.

 

[dale_para_bajo]

,,,how can CSA v1 work without cuda? ...

The only public program (in public domain) to work for CSA RBT are colibris programs. ALL of them where build to be run using Nvidia GPU ( CUDA ). So the answer is you need a Nvidia GPU ( Cuda).

But if you are willing to build your program. I guess it could even be build without GPU.
I will explain. The process has 2 mayor fuctions.
*A GPU is used to create a 0x1000 chain for the crypt8 you are looking for,
*Then the endvalues are compare with the RBT Table and StartValues are obtain. ( this is what takes 50 minutes) Normally there are 100,000 posible solutions.
*Then a GPU is use to crate the 0x1000 chains of the start values. Mean while we keep an eye in each iteration to see if the crypt8 is found, Problem is that you have to do about 100,000 of this.


Well if you are willing to program you can do the GPU work in CPU!. Clearly it will take some time but it can be do,

Now in my case I do not have Nvidia GPU. Instead I have AMD GPU. But as powerfull as CUDA. But Cilibri programs do not work on it.

 

[Francescone]

Yes, I understood. Unfortunately I'm not able to write code, so I need an NVIDIA GPU. Thank you very much for your advice.

 

[kebien]

@kebien:
It shouldn't matter what drive the program is on.
If so, its either a bug or some odd setup.

No,I tried different setups,3 to 5 minutes search if the tables and program are in the same SSD.My SSD is not one those really fastest ones,though.
If program is installed in different drive than tables then it takes way longer,given the tables are in USB or SATA drives,it does show this behavior in my setup.
It does seem to me accessing tables directly from the same drive is the optimal,and not discounting I could also have a problem,but the search time for me in this way is fast enough for me

 

[Martin.Wigston]

No,I tried different setups,3 to 5 minutes search if the tables and program are in the same SSD.My SSD is not one those really fastest ones,though.
If program is installed in different drive than tables then it takes way longer,given the tables are in USB or SATA drives,it does show this behavior in my setup.
It does seem to me accessing tables directly from the same drive is the optimal,and not discounting I could also have a problem,but the search time for me in this way is fast enough for me

Once you have started the program, it is kept in memory, so makes no sense why it should be slower when run off your HDD, unless you dont have enough memory.
I upgraded to 16GB of memory since I was using up all 8GB when doing RBT lookups and was spooling to the page file

 

[kebien]

Winston
Maybe that's the reason,but while it makes no sense,you can simply try and see if my account applies to everyone or is simply a one setup situation.
I kept asking myself why would be different,but it is,unequivocally.

 

[dale_para_bajo]

@Martin.Wigston, please do not see this as I am contradicting you. But instead it is because I want to learn how to track that memory used. How can I chem my memory been use you said :

I upgraded to 16GB of memory since I was using up all 8GB when doing RBT lookups and was spooling to the page file

I too have a very old Tower with only 8 Gigs of memory. Thanks.

 

[Martin.Wigston]

@Martin.Wigston, please do not see this as I am contradicting you. But instead it is because I want to learn how to track that memory used. How can I chem my memory been use you said :


I too have a very old Tower with only 8 Gigs of memory. Thanks.

Using task manager you can see memory usage on the performance tab, Windows 10 show it a lot better than XP, not sure about how its shown in other OS versions but XP shows what's available rather than used.

Since upgrading my memory my usage goes up from 3.5-4GB (normal usage) before doing a lookup to 8.5-9GB at the end of the SSD search