Top suggestions for Federated Learning Visualization PipelineExplore more searches like Federated Learning Visualization PipelinePeople interested in Federated Learning Visualization Pipeline also searched for |
Top suggestions for Federated Learning Visualization PipelineExplore more searches like Federated Learning Visualization PipelinePeople interested in Federated Learning Visualization Pipeline also searched for |
Feedback
.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.48c4cac955ce605745d1cb3034d3a904%3frik%3dhPL2F4SRyuBprQ%26pid%3dImgRaw%26r%3d0&exph=653&expw=1000&q=Federated+Learning+Visualization+Pipeline&FORM=IRPRST&ck=2EBE4116A49629F26CCEFE3BD989A9CC&selectedIndex=8&itb=0)
%252C+especially+large+language+models+(LLMs).+However%252C+for+many+downstream+tasks%252C+it+is+necessary+to+fine-tune+LLMs+using+private+data.+While+federated+learning+offers+a+promising+privacy-preserving+solution+to+LLM+fine-tuning%252C+the+substantial+size+of+an+LLM%252C+combined+with+high+computational+and+communication+demands%252C+makes+it+hard+to+apply+to+downstream+tasks.+More+importantly%252C+private+edge+servers+often+possess+varying+computing+and+network+resources+in+real-world+scenarios%252C+introducing+additional+complexities+to+LLM+fine-tuning.+To+tackle+these+problems%252C+we+design+and+implement+an+automated+federated+pipeline%252C+named+FedPipe%252C+to+fine-tune+LLMs+with+minimal+training+cost+but+without+adding+any+inference+latency.+FedPipe+firstly+identifies+the+weights+to+be+fine-tuned+based+on+their+contributions+to+the+LLM+training.+It+then+configures+a+low-rank+adapter+for+each+selected+weight+to+train+local+low-rank+adapters+on+an+edge+server%252C+and+aggregate+local+adapters+of+all+edge+servers+to+fine-tune+the+whole+LLM.+Finally%252C+it+appropriately+quantizes+the+parameters+of+LLM+to+reduce+memory+space+according+to+the+requirements+of+edge+servers.+Extensive+experiments+demonstrate+that+FedPipe+expedites+the+model+training+and+achieves+higher+accuracy+than+state-of-the-art+benchmarks.&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.54448a891ccfa6f116b1115372e3e6f7%3frik%3d863gQM9A7CUrHQ%26pid%3dImgRaw%26r%3d0&exph=630&expw=1200&q=Federated+Learning+Visualization+Pipeline&FORM=IRPRST&ck=CB189824DBFD3BDFC868A69C64A16E37&selectedIndex=19&itb=0)
