ThreadPoolExecutor中長度值增加的記憶體安全排列生成器

我參考了一些答案，最有意義的答案來自這里，所以我正在嘗試根據我的目的對其進行修改。

我正在嘗試創建一個生成器，它可以生成從長度為 1 到長度為MAX_RECURSION_LENGTH3 的所有排列，而不使用串列，因為這會立即占用 100% 的 RAM，同時使用多執行緒函式。

我在下面包含了一個最小的實驗代碼，它還不能作業，但它的評論很好，我想你會看到我想要做什么。帶有單詞串列：

list_of_words = ["I", "like", "to", "take", "my", "dogs", "for", "a",
                 "walk", "every", "day", "after", "work"]

...我希望它生成最大長度為 3 的所有排列，例如：

# Length of 1
("I",)
("like",) 
("to",)
("take",)
("my",)
...
("after",)
("work",)

# Length of 2
("I", "like",)
("I", "to",)
("I", "take",)
...
("I", "after",)
("I", "work",)

("like", "I",)
("like", "to",)
("like", "take",)
...
("like", "after",)
("like", "work",)
...

# Length of 3
("I", "like", "to",)
("I", "like", "take",)
("I", "like", "my",)
...
("I", "like", "after",)
("I", "like", "work",)

("like", "to", "take",)
("like", "to", "my",)
("like", "to", "dogs",)
...
("like", "to", "after",)
("like", "to", "work",)

# etc. etc., ending at
("work", "I", "like",)
("work", "I", "to",)
("work", "I", "take",)
...
("work", "after", "every",)
("work", "after", "day",)

請注意，我在這里沒有包含重復，但是我不介意是否有重復，("work", "work", "work",)因為如果可能的話，我的特定用例可以從中受益。請記住在運行此程式時注意您的 RAM，我不希望它達到 100%，這就是為什么我認為使用生成器并避免list(permutations())是最好的方法。另請注意，我不關心代碼的速度，只要它降低記憶體使用量即可。這是代碼：

import concurrent.futures
from itertools import permutations

# Variables
MAX_ENDPOINT_PERMUTATION_LENGTH = 3
MAX_WORKERS = 6

# List of words to permutate
list_of_words = ["I", "like", "to", "take", "my", "dogs", "for", "a",
                 "walk", "every", "day", "after", "work"]

# Generator from https://code.activestate.com/recipes/252178/
def all_perms(elements, length):
    # If the permutation length right now is 1, just return
    # the list_of_words as is
    if length == 1: 
        yield elements
    # Else, if the length is >= 2...
    else:
        # Iterate over the list, starting at "I" in the list_of_words
        for i in range(len(elements)):
            # Iterate over the current length of the permutation, which starts
            # at 1, so it should be ("I", "like",), then ("I", "to",), etc.
            for p in permutations(elements[:i]   elements[i 1:], length):
                # This yield throws an error because "p" is a tuple, even if it's
                # just one value at the beginning, and you can't add a tuple to
                # a string of "I". Might need to iterate over P, adding each
                # value to elements[i]  = p[k]? But how to do this with yield?
                yield elements[i]   p

# The multithreaded function we're going to use
def do_something_with_current_perm(current_perm):
    print(current_perm)

# Iterate through the range of MAX_ENDPOINT_PERMUTATION_LENGTH (3 by default)
for j in range(MAX_ENDPOINT_PERMUTATION_LENGTH):
    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        # Trying to pass in j 1 which is 0 1=1 at the beginning, meaning the current
        # permutation depth is 1 element, then it'll be 2, then 3
        try:
            future_results = {executor.submit(do_something_with_current_perm, perm): perm for perm in next(all_perms(list_of_words, j 1))}
        except KeyboardInterrupt:
            executor._threads.clear()
            concurrent.futures.thread._threads_queues.clear()
            break

更新

我能夠完成這項作業：

1. 我完全去掉了這個all_perms功能
2. 我改變permutations以product確保我得到我想要的重復

這是現在的作業代碼：

import concurrent.futures
from itertools import permutations, product

# Variables
MAX_ENDPOINT_PERMUTATION_LENGTH = 3
MAX_WORKERS = 6

# List of words to permutate
list_of_words = ["I", "like", "to", "take", "my", "dogs", "for", "a",
                 "walk", "every", "day", "after", "work"]

# The multithreaded function we're going to use
def do_something_with_current_perm(current_perm):
    print(current_perm)

# Iterate through the range of MAX_ENDPOINT_PERMUTATION_LENGTH (3 by default)
for j in range(1, MAX_ENDPOINT_PERMUTATION_LENGTH 1):
    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        try:
            future_results = {executor.submit(do_something_with_current_perm, perm): perm for perm in product(list_of_words, repeat=j)}
        except KeyboardInterrupt:
            executor._threads.clear()
            concurrent.futures.thread._threads_queues.clear()
            break

uj5u.com熱心網友回復：

要生成長度為 1 到 3 的所有排列，您可以使用下一個示例：

from itertools import permutations

list_of_words = ["I", "like", "to", "take", "my", "dogs", "for", "a",
                 "walk", "every", "day", "after", "work"]

for i in range(1, 4):
    for p in permutations(list_of_words, i):
        print(p)

印刷：

('I',)
('like',)
('to',)
('take',)
('my',)
('dogs',)
('for',)
('a',)
('walk',)
('every',)
('day',)
('after',)
('work',)
('I', 'like')
('I', 'to')
('I', 'take')

...

('work', 'after', 'walk')
('work', 'after', 'every')
('work', 'after', 'day')

標籤：python-3.x 多线程 排列

上一篇：我應該如何回復頻道訊息？

下一篇：了解Python中的多執行緒和鎖（概念和示例）