I am learning how to use the threading and the multiprocessing modules in Python to run certain operations in parallel and speed up my code. I am finding this hard (maybe because I don’t have any theoretical background about it) to understand what the difference is between a threading.Thread() object and a multiprocessing.Process() one. Also, … Read more
I was thrilled to see the new System.Collections.Concurrent namespace in .Net 4.0, quite nice! I’ve seen ConcurrentDictionary, ConcurrentQueue, ConcurrentStack, ConcurrentBag and BlockingCollection. One thing that seems to be mysteriously missing is a ConcurrentList<T>. Do I have to write that myself (or get it off the web 🙂 )? Am I missing something obvious here? 12 … Read more
I am porting some code to Parallel.ForEach and got an error with a continue I have in the code. Is there something equivalent I can use in a Parallel.ForEach functionally equivalent to continue in a foreach loop? Parallel.ForEach(items, parallelOptions, item => { if (!isTrue) continue; }); 2 Answers 2
Let’s say I have a 4-core CPU, and I want to run some process in the minimum amount of time. The process is ideally parallelizable, so I can run chunks of it on an infinite number of threads and each thread takes the same amount of time. Since I have 4 cores, I don’t expect … Read more
I am trying to write a .sh file that runs many programs simultaneously I tried this prog1 prog2 But that runs prog1 then waits until prog1 ends and then starts prog2… So how can I run them in parallel? 16 Answers 16
I thought that they were basically the same thing — writing programs that split tasks between processors (on machines that have 2+ processors). Then I’m reading this, which says: Async methods are intended to be non-blocking operations. An await expression in an async method doesn’t block the current thread while the awaited task is running. … Read more
What is the difference between concurrent programming and parallel programing? I asked google but didn’t find anything that helped me to understand that difference. Could you give me an example for both? For now I found this explanation: http://www.linux-mag.com/id/7411 – but “concurrency is a property of the program” vs “parallel execution is a property of … Read more
This is probably a trivial question, but how do I parallelize the following loop in python? # setup output lists output1 = list() output2 = list() output3 = list() for j in range(0, 10): # calc individual parameter value parameter = j * offset # call the calculation out1, out2, out3 = calc_stuff(parameter = parameter) … Read more
I need to execute some amount of tasks 4 at a time, something like this: ExecutorService taskExecutor = Executors.newFixedThreadPool(4); while(…) { taskExecutor.execute(new MyTask()); } //…wait for completion somehow How can I get notified once all of them are complete? For now I can’t think about anything better than setting some global task counter and decrease … Read more
Is it possible to specify a custom thread pool for Java 8 parallel stream? I can not find it anywhere. Imagine that I have a server application and I would like to use parallel streams. But the application is large and multi-threaded so I want to compartmentalize it. I do not want a slow running … Read more
