Understanding Federated Learning Through an Adversarial Lens · Understanding Federated Learning...

Understanding Federated Learning Through an Adversarial Lens

Supriyo Chakraborty IBM Research

1

Federated learning

2

Server

Federated learning

2

Server

Federated learning

wtG

<latexit sha1_base64="ceUL85/fRXJoUidO/o930Q5BiRc=">AAACLXicdVDLSgMxFM34rPXRVpdugkVwNcyIYJdFC7qsYB/QqSWTybShSWZIMpYyzE+41X/wa1wI4tbfMNNWsD4OhBzOuZd77/FjRpV2nFdrZXVtfWOzsFXc3tndK5Ur+20VJRKTFo5YJLs+UoRRQVqaaka6sSSI+4x0/PFl7nfuiVQ0Erd6GpM+R0NBQ4qRNlLX88PJ4OpOD8pVx645OeBv4tqz36mCBZqDilXygggnnAiNGVKq5zqx7qdIaooZyYpeokiM8BgNSc9QgThR/XS2cAaPjRLAMJLmCQ1n6veOFHGlptw3lRzpkfrp5eJfXi/RYa2fUhEnmgg8HxQmDOoI5tfDgEqCNZsagrCkZleIR0girE1GRU+QCY44RyJI82Sy1MtH+GE6ybJll+PGwsWIpQ1jmwi/coL/k/ap7Tq2e3NWrV8swiyAQ3AEToALzkEdXIMmaAEMGHgAj+DJerZerDfrfV66Yi16DsASrI9PmgqpIg==</latexit><latexit sha1_base64="ceUL85/fRXJoUidO/o930Q5BiRc=">AAACLXicdVDLSgMxFM34rPXRVpdugkVwNcyIYJdFC7qsYB/QqSWTybShSWZIMpYyzE+41X/wa1wI4tbfMNNWsD4OhBzOuZd77/FjRpV2nFdrZXVtfWOzsFXc3tndK5Ur+20VJRKTFo5YJLs+UoRRQVqaaka6sSSI+4x0/PFl7nfuiVQ0Erd6GpM+R0NBQ4qRNlLX88PJ4OpOD8pVx645OeBv4tqz36mCBZqDilXygggnnAiNGVKq5zqx7qdIaooZyYpeokiM8BgNSc9QgThR/XS2cAaPjRLAMJLmCQ1n6veOFHGlptw3lRzpkfrp5eJfXi/RYa2fUhEnmgg8HxQmDOoI5tfDgEqCNZsagrCkZleIR0girE1GRU+QCY44RyJI82Sy1MtH+GE6ybJll+PGwsWIpQ1jmwi/coL/k/ap7Tq2e3NWrV8swiyAQ3AEToALzkEdXIMmaAEMGHgAj+DJerZerDfrfV66Yi16DsASrI9PmgqpIg==</latexit><latexit sha1_base64="ceUL85/fRXJoUidO/o930Q5BiRc=">AAACLXicdVDLSgMxFM34rPXRVpdugkVwNcyIYJdFC7qsYB/QqSWTybShSWZIMpYyzE+41X/wa1wI4tbfMNNWsD4OhBzOuZd77/FjRpV2nFdrZXVtfWOzsFXc3tndK5Ur+20VJRKTFo5YJLs+UoRRQVqaaka6sSSI+4x0/PFl7nfuiVQ0Erd6GpM+R0NBQ4qRNlLX88PJ4OpOD8pVx645OeBv4tqz36mCBZqDilXygggnnAiNGVKq5zqx7qdIaooZyYpeokiM8BgNSc9QgThR/XS2cAaPjRLAMJLmCQ1n6veOFHGlptw3lRzpkfrp5eJfXi/RYa2fUhEnmgg8HxQmDOoI5tfDgEqCNZsagrCkZleIR0girE1GRU+QCY44RyJI82Sy1MtH+GE6ybJll+PGwsWIpQ1jmwi/coL/k/ap7Tq2e3NWrV8swiyAQ3AEToALzkEdXIMmaAEMGHgAj+DJerZerDfrfV66Yi16DsASrI9PmgqpIg==</latexit><latexit sha1_base64="ceUL85/fRXJoUidO/o930Q5BiRc=">AAACLXicdVDLSgMxFM34rPXRVpdugkVwNcyIYJdFC7qsYB/QqSWTybShSWZIMpYyzE+41X/wa1wI4tbfMNNWsD4OhBzOuZd77/FjRpV2nFdrZXVtfWOzsFXc3tndK5Ur+20VJRKTFo5YJLs+UoRRQVqaaka6sSSI+4x0/PFl7nfuiVQ0Erd6GpM+R0NBQ4qRNlLX88PJ4OpOD8pVx645OeBv4tqz36mCBZqDilXygggnnAiNGVKq5zqx7qdIaooZyYpeokiM8BgNSc9QgThR/XS2cAaPjRLAMJLmCQ1n6veOFHGlptw3lRzpkfrp5eJfXi/RYa2fUhEnmgg8HxQmDOoI5tfDgEqCNZsagrCkZleIR0girE1GRU+QCY44RyJI82Sy1MtH+GE6ybJll+PGwsWIpQ1jmwi/coL/k/ap7Tq2e3NWrV8swiyAQ3AEToALzkEdXIMmaAEMGHgAj+DJerZerDfrfV66Yi16DsASrI9PmgqpIg==</latexit>

2

Server

Federated learning

wtG


. . . . . . . . .

2

wtG

<latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit><latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit><latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit><latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit> wtG


<latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit><latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit><latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit><latexit sha1_base64="hCJsx0QcFUctrMMwYwyHXmYZf/8=">AAACLXicdZDNSgMxFIUz/tb601aXboJFcDVkSqt2JyroUsHaQqeWTJrR0CQzJBlLGeYl3Oo7+DQuBHHra5ipFazohcDhfPdyb04Qc6YNQq/O3PzC4tJyYaW4ura+USpXNq91lChCWyTikeoEWFPOJG0ZZjjtxIpiEXDaDoYnOW/fU6VZJK/MOKY9gW8lCxnBxlodPwhH/bMb0y9XkYtqh416DSK31kBNr2lFA3nN/Tr0XDSpKpjWRb/ilPxBRBJBpSEca931UGx6KVaGEU6zop9oGmMyxLe0a6XEgupeOjk4g7vWGcAwUvZJAyfuz4kUC63HIrCdAps7/Zvl5l+sm5jwsJcyGSeGSvK1KEw4NBHMfw8HTFFi+NgKTBSzt0JyhxUmxmZU9CUdkUgILAdpnkyW+vmKIExHWTZLBTmdUoJ5emqxjfA7J/i/uK65HnK9y3r16HgaZgFsgx2wBzxwAI7AObgALUAABw/gETw5z86L8+a8f7XOOdOZLTBTzscn6qGpUg==</latexit>

Server

Federated learning

wtG


. . . . . . . . .

2

wtG




Server

Federated learning

wtG


. . . . . . . . . Compute

�t+11

<latexit sha1_base64="D3aW/8S9R5G7eFMRvexSRdDWfV4=">AAACSnicdVBNSxxBEO3ZmETN1xqPOaTJEggElhkJxKMYDx4NZFXY2SzVPTXa2N0zdNe4LM38llyT/5I/kL/hTbykZ10haixo+vFeFa/qiVorT2n6J+k9Wnn85Onq2vqz5y9evupvvD70VeMkjmSlK3cswKNWFkekSONx7RCM0Hgkzr50+tE5Oq8q+43mNU4MnFhVKgkUqWl/Mxcm5AVqgnaafQ/0MWun/UE63E674vdBNlz86YAt62C6kbzNi0o2Bi1JDd6Ps7SmSQBHSmps1/PGYw3yDE5wHKEFg34SFtu3/H1kCl5WLj5LfMH+OxHAeD83InYaoFN/V+vI/2njhsrtSVC2bgitvDYqG82p4l0UvFAOJel5BCCdirtyeQoOJMXAbrkIE2+wOJOVMWCLkIty1oa8sxRlmLV3VCP3lqoEHfaiHCO9yY0/DA63hlk6zL5+GuzsLsNdZW/YO/aBZewz22H77ICNmGRz9oP9ZL+S38lFcplcXbf2kuXMJrtVvZW/VwizlA==</latexit><latexit sha1_base64="D3aW/8S9R5G7eFMRvexSRdDWfV4=">AAACSnicdVBNSxxBEO3ZmETN1xqPOaTJEggElhkJxKMYDx4NZFXY2SzVPTXa2N0zdNe4LM38llyT/5I/kL/hTbykZ10haixo+vFeFa/qiVorT2n6J+k9Wnn85Onq2vqz5y9evupvvD70VeMkjmSlK3cswKNWFkekSONx7RCM0Hgkzr50+tE5Oq8q+43mNU4MnFhVKgkUqWl/Mxcm5AVqgnaafQ/0MWun/UE63E674vdBNlz86YAt62C6kbzNi0o2Bi1JDd6Ps7SmSQBHSmps1/PGYw3yDE5wHKEFg34SFtu3/H1kCl5WLj5LfMH+OxHAeD83InYaoFN/V+vI/2njhsrtSVC2bgitvDYqG82p4l0UvFAOJel5BCCdirtyeQoOJMXAbrkIE2+wOJOVMWCLkIty1oa8sxRlmLV3VCP3lqoEHfaiHCO9yY0/DA63hlk6zL5+GuzsLsNdZW/YO/aBZewz22H77ICNmGRz9oP9ZL+S38lFcplcXbf2kuXMJrtVvZW/VwizlA==</latexit><latexit sha1_base64="D3aW/8S9R5G7eFMRvexSRdDWfV4=">AAACSnicdVBNSxxBEO3ZmETN1xqPOaTJEggElhkJxKMYDx4NZFXY2SzVPTXa2N0zdNe4LM38llyT/5I/kL/hTbykZ10haixo+vFeFa/qiVorT2n6J+k9Wnn85Onq2vqz5y9evupvvD70VeMkjmSlK3cswKNWFkekSONx7RCM0Hgkzr50+tE5Oq8q+43mNU4MnFhVKgkUqWl/Mxcm5AVqgnaafQ/0MWun/UE63E674vdBNlz86YAt62C6kbzNi0o2Bi1JDd6Ps7SmSQBHSmps1/PGYw3yDE5wHKEFg34SFtu3/H1kCl5WLj5LfMH+OxHAeD83InYaoFN/V+vI/2njhsrtSVC2bgitvDYqG82p4l0UvFAOJel5BCCdirtyeQoOJMXAbrkIE2+wOJOVMWCLkIty1oa8sxRlmLV3VCP3lqoEHfaiHCO9yY0/DA63hlk6zL5+GuzsLsNdZW/YO/aBZewz22H77ICNmGRz9oP9ZL+S38lFcplcXbf2kuXMJrtVvZW/VwizlA==</latexit><latexit sha1_base64="D3aW/8S9R5G7eFMRvexSRdDWfV4=">AAACSnicdVBNSxxBEO3ZmETN1xqPOaTJEggElhkJxKMYDx4NZFXY2SzVPTXa2N0zdNe4LM38llyT/5I/kL/hTbykZ10haixo+vFeFa/qiVorT2n6J+k9Wnn85Onq2vqz5y9evupvvD70VeMkjmSlK3cswKNWFkekSONx7RCM0Hgkzr50+tE5Oq8q+43mNU4MnFhVKgkUqWl/Mxcm5AVqgnaafQ/0MWun/UE63E674vdBNlz86YAt62C6kbzNi0o2Bi1JDd6Ps7SmSQBHSmps1/PGYw3yDE5wHKEFg34SFtu3/H1kCl5WLj5LfMH+OxHAeD83InYaoFN/V+vI/2njhsrtSVC2bgitvDYqG82p4l0UvFAOJel5BCCdirtyeQoOJMXAbrkIE2+wOJOVMWCLkIty1oa8sxRlmLV3VCP3lqoEHfaiHCO9yY0/DA63hlk6zL5+GuzsLsNdZW/YO/aBZewz22H77ICNmGRz9oP9ZL+S38lFcplcXbf2kuXMJrtVvZW/VwizlA==</latexit>

Compute

�t+12

<latexit sha1_base64="wDkksDghZJzr8BfzalpPBZhXvt8=">AAACSnicdVBNSxxBEO3ZmGjM1xqPOaTJEggElhkJ6FGMhxwVsirsbJbqnhpt7O4ZumtclmZ+S67Jf8kfyN/ITbzYs24gaixo+vFeFa/qiVorT2n6O+k9Wnn8ZHXt6fqz5y9evupvvD7yVeMkjmSlK3ciwKNWFkekSONJ7RCM0Hgszj93+vEFOq8q+5XmNU4MnFpVKgkUqWl/Mxcm5AVqgna69S3Qx6yd9gfpcCftit8H2XDxpwO2rIPpRvI2LyrZGLQkNXg/ztKaJgEcKamxXc8bjzXIczjFcYQWDPpJWGzf8veRKXhZufgs8QX770QA4/3ciNhpgM78Xa0j/6eNGyp3JkHZuiG08saobDSnindR8EI5lKTnEYB0Ku7K5Rk4kBQDu+UiTLzB4kxWxoAtQi7KWRvyzlKUYdbeUY3cX6oSdNiPcoz0b278YXC0NczSYXb4abC7twx3jb1h79gHlrFttsu+sAM2YpLN2Xf2g/1MfiV/ksvk6qa1lyxnNtmt6q1cA1jgs5U=</latexit><latexit sha1_base64="wDkksDghZJzr8BfzalpPBZhXvt8=">AAACSnicdVBNSxxBEO3ZmGjM1xqPOaTJEggElhkJ6FGMhxwVsirsbJbqnhpt7O4ZumtclmZ+S67Jf8kfyN/ITbzYs24gaixo+vFeFa/qiVorT2n6O+k9Wnn8ZHXt6fqz5y9evupvvD7yVeMkjmSlK3ciwKNWFkekSONJ7RCM0Hgszj93+vEFOq8q+5XmNU4MnFpVKgkUqWl/Mxcm5AVqgna69S3Qx6yd9gfpcCftit8H2XDxpwO2rIPpRvI2LyrZGLQkNXg/ztKaJgEcKamxXc8bjzXIczjFcYQWDPpJWGzf8veRKXhZufgs8QX770QA4/3ciNhpgM78Xa0j/6eNGyp3JkHZuiG08saobDSnindR8EI5lKTnEYB0Ku7K5Rk4kBQDu+UiTLzB4kxWxoAtQi7KWRvyzlKUYdbeUY3cX6oSdNiPcoz0b278YXC0NczSYXb4abC7twx3jb1h79gHlrFttsu+sAM2YpLN2Xf2g/1MfiV/ksvk6qa1lyxnNtmt6q1cA1jgs5U=</latexit><latexit sha1_base64="wDkksDghZJzr8BfzalpPBZhXvt8=">AAACSnicdVBNSxxBEO3ZmGjM1xqPOaTJEggElhkJ6FGMhxwVsirsbJbqnhpt7O4ZumtclmZ+S67Jf8kfyN/ITbzYs24gaixo+vFeFa/qiVorT2n6O+k9Wnn8ZHXt6fqz5y9evupvvD7yVeMkjmSlK3ciwKNWFkekSONJ7RCM0Hgszj93+vEFOq8q+5XmNU4MnFpVKgkUqWl/Mxcm5AVqgna69S3Qx6yd9gfpcCftit8H2XDxpwO2rIPpRvI2LyrZGLQkNXg/ztKaJgEcKamxXc8bjzXIczjFcYQWDPpJWGzf8veRKXhZufgs8QX770QA4/3ciNhpgM78Xa0j/6eNGyp3JkHZuiG08saobDSnindR8EI5lKTnEYB0Ku7K5Rk4kBQDu+UiTLzB4kxWxoAtQi7KWRvyzlKUYdbeUY3cX6oSdNiPcoz0b278YXC0NczSYXb4abC7twx3jb1h79gHlrFttsu+sAM2YpLN2Xf2g/1MfiV/ksvk6qa1lyxnNtmt6q1cA1jgs5U=</latexit><latexit sha1_base64="wDkksDghZJzr8BfzalpPBZhXvt8=">AAACSnicdVBNSxxBEO3ZmGjM1xqPOaTJEggElhkJ6FGMhxwVsirsbJbqnhpt7O4ZumtclmZ+S67Jf8kfyN/ITbzYs24gaixo+vFeFa/qiVorT2n6O+k9Wnn8ZHXt6fqz5y9evupvvD7yVeMkjmSlK3ciwKNWFkekSONJ7RCM0Hgszj93+vEFOq8q+5XmNU4MnFpVKgkUqWl/Mxcm5AVqgna69S3Qx6yd9gfpcCftit8H2XDxpwO2rIPpRvI2LyrZGLQkNXg/ztKaJgEcKamxXc8bjzXIczjFcYQWDPpJWGzf8veRKXhZufgs8QX770QA4/3ciNhpgM78Xa0j/6eNGyp3JkHZuiG08saobDSnindR8EI5lKTnEYB0Ku7K5Rk4kBQDu+UiTLzB4kxWxoAtQi7KWRvyzlKUYdbeUY3cX6oSdNiPcoz0b278YXC0NczSYXb4abC7twx3jb1h79gHlrFttsu+sAM2YpLN2Xf2g/1MfiV/ksvk6qa1lyxnNtmt6q1cA1jgs5U=</latexit>

�t+1k

<latexit sha1_base64="MOqhRpDxjLaZn7UrCxcs8bBuebE=">AAACSnicdVBNaxRBEO1ZoybxIxtz9JDGRRCEZSYEzDFoDh4juElgZ12qe2qSZrt7hu4al6WZ3+JV/4t/wL/hLeRiz2YDSdSCph/vVfGqnqi18pSmv5Leg7WHjx6vb2w+efrs+VZ/+8WJrxoncSQrXbkzAR61sjgiRRrPaodghMZTMfvQ6adf0XlV2c+0qHFi4NyqUkmgSE37O7kwIS9QE7TT2ZdAb7N22h+kw4O0K/43yIbLPx2wVR1Pt5PdvKhkY9CS1OD9OEtrmgRwpKTGdjNvPNYgZ3CO4wgtGPSTsNy+5a8jU/CycvFZ4kv29kQA4/3CiNhpgC78fa0j/6WNGyoPJkHZuiG08tqobDSnindR8EI5lKQXEYB0Ku7K5QU4kBQDu+MiTLzB4lxWxoAtQi7KeRvyzlKUYd7eU408WqkSdDiKcoz0Jjf+f3CyN8zSYfZpf3D4fhXuOnvJXrE3LGPv2CH7yI7ZiEm2YN/Yd/Yj+Zn8Ti6Tq+vWXrKa2WF3qrf2B8H4s84=</latexit><latexit sha1_base64="MOqhRpDxjLaZn7UrCxcs8bBuebE=">AAACSnicdVBNaxRBEO1ZoybxIxtz9JDGRRCEZSYEzDFoDh4juElgZ12qe2qSZrt7hu4al6WZ3+JV/4t/wL/hLeRiz2YDSdSCph/vVfGqnqi18pSmv5Leg7WHjx6vb2w+efrs+VZ/+8WJrxoncSQrXbkzAR61sjgiRRrPaodghMZTMfvQ6adf0XlV2c+0qHFi4NyqUkmgSE37O7kwIS9QE7TT2ZdAb7N22h+kw4O0K/43yIbLPx2wVR1Pt5PdvKhkY9CS1OD9OEtrmgRwpKTGdjNvPNYgZ3CO4wgtGPSTsNy+5a8jU/CycvFZ4kv29kQA4/3CiNhpgC78fa0j/6WNGyoPJkHZuiG08tqobDSnindR8EI5lKQXEYB0Ku7K5QU4kBQDu+MiTLzB4lxWxoAtQi7KeRvyzlKUYd7eU408WqkSdDiKcoz0Jjf+f3CyN8zSYfZpf3D4fhXuOnvJXrE3LGPv2CH7yI7ZiEm2YN/Yd/Yj+Zn8Ti6Tq+vWXrKa2WF3qrf2B8H4s84=</latexit><latexit sha1_base64="MOqhRpDxjLaZn7UrCxcs8bBuebE=">AAACSnicdVBNaxRBEO1ZoybxIxtz9JDGRRCEZSYEzDFoDh4juElgZ12qe2qSZrt7hu4al6WZ3+JV/4t/wL/hLeRiz2YDSdSCph/vVfGqnqi18pSmv5Leg7WHjx6vb2w+efrs+VZ/+8WJrxoncSQrXbkzAR61sjgiRRrPaodghMZTMfvQ6adf0XlV2c+0qHFi4NyqUkmgSE37O7kwIS9QE7TT2ZdAb7N22h+kw4O0K/43yIbLPx2wVR1Pt5PdvKhkY9CS1OD9OEtrmgRwpKTGdjNvPNYgZ3CO4wgtGPSTsNy+5a8jU/CycvFZ4kv29kQA4/3CiNhpgC78fa0j/6WNGyoPJkHZuiG08tqobDSnindR8EI5lKQXEYB0Ku7K5QU4kBQDu+MiTLzB4lxWxoAtQi7KeRvyzlKUYd7eU408WqkSdDiKcoz0Jjf+f3CyN8zSYfZpf3D4fhXuOnvJXrE3LGPv2CH7yI7ZiEm2YN/Yd/Yj+Zn8Ti6Tq+vWXrKa2WF3qrf2B8H4s84=</latexit><latexit sha1_base64="MOqhRpDxjLaZn7UrCxcs8bBuebE=">AAACSnicdVBNaxRBEO1ZoybxIxtz9JDGRRCEZSYEzDFoDh4juElgZ12qe2qSZrt7hu4al6WZ3+JV/4t/wL/hLeRiz2YDSdSCph/vVfGqnqi18pSmv5Leg7WHjx6vb2w+efrs+VZ/+8WJrxoncSQrXbkzAR61sjgiRRrPaodghMZTMfvQ6adf0XlV2c+0qHFi4NyqUkmgSE37O7kwIS9QE7TT2ZdAb7N22h+kw4O0K/43yIbLPx2wVR1Pt5PdvKhkY9CS1OD9OEtrmgRwpKTGdjNvPNYgZ3CO4wgtGPSTsNy+5a8jU/CycvFZ4kv29kQA4/3CiNhpgC78fa0j/6WNGyoPJkHZuiG08tqobDSnindR8EI5lKQXEYB0Ku7K5QU4kBQDu+MiTLzB4lxWxoAtQi7KeRvyzlKUYd7eU408WqkSdDiKcoz0Jjf+f3CyN8zSYfZpf3D4fhXuOnvJXrE3LGPv2CH7yI7ZiEm2YN/Yd/Yj+Zn8Ti6Tq+vWXrKa2WF3qrf2B8H4s84=</latexit>

Compute

For each agent j at t, �t+1j = argmin

�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �

�<latexit sha1_base64="ot0/khi+M/IHQaGQ9YFsx39xQPQ=">AAADTnicfVHdbtMwFHY6fkr5a+GSG4sKabCpSjZtXUGTyo+ACxBDotukuo0c12ndxUnkOLSV5SfgUbjlAXgEbnkR7hCctAVa/o4i+/P5zsmxvy9II5Fp1/3slDbOnb9wsXypcvnK1WvXq7Ubx1mSK8Y7LIkSdRrQjEci5h0tdMRPU8WpDCJ+Epw9LviTt1xlIonf6FnKe5IOYxEKRjWk/Op7ovlUm6eJwpyyEaZDHmts8RgvCEyLk97GBL5AGjLgkaa2b/SWZ/0xPsSEqqEUsW9WaPsCjvN+raiIrcUk4qHexASqwqk/7ovtWbES6xtx6MH/Yn9sHwA58Z/1Nd5aHYaJEsORvutX627D3d9r7brYbey5XrPVAuC6+we7O9gDUES93bj/4SNC6MivOe/IIGG5hDexiGZZ13NT3TNUacEibiskz3hK2Rm8ugswppJnPTNX1eI7kBngEJQJE9Bknl3tMFRm2UwGUCmpHmW/c0Xyb1w31+FBz4g4zTWP2WJQmEdYJ7iwCA+E4kxHMwCUKQF3xWxEFWUajKyQmE9YIiWNB4Yc267XM6SYEYSm7lm7zoOe9ic9+ZOd/mKnBfuEg06Kv4TUq5QrqhN1z8wdplC63P9XBmab5b6mRyAt2PfDI/xvcLzT8AC/Bh8foUWU0S10G20iDzVRGz1HR6iDmFNzmk7beVj6VPpS+lr6tigtOcuem2gtNsrfAYseGHk=</latexit><latexit sha1_base64="npLzmI/AO9sb0eZLeasrSgp90SQ=">AAADTnicfVHdbtMwFE4yfkr5a+GSG4sKabCpSjZtXUGTyo+ACxCbRLtJdRs5rtO6S5zIcWgry0/AC/AO3CLxGtzyItwhOGkLtPwdRfbn852TY39fkEY8U6772XY2zp2/cLF0qXz5ytVr1yvVG50sySVlbZpEiTwNSMYiLlhbcRWx01QyEgcROwnOHhf8yRsmM56I12qWsl5MhoKHnBIFKb/yHis2VfppIhEjdITIkAmFDBqjBYFIcVLbCMMXxBoPWKSI6Wu15Rl/jA4RJnIYc+HrFdq8gOO8X0nChTEIRyxUmwhDVTj1x32+PStWbHzNDz34n/DH5gGQE/9ZX6Gt1WEISz4cqbt+pebW3f295q6L3Pqe6zWaTQCuu3+wu4M8AEXUWvX7Hz5uH7878qv2WzxIaB7Dm2hEsqzruanqaSIVpxEzZZxnLCX0DF7dBShIzLKenqtq0B3IDFAIyoQJaDLPrnZoEmfZLA6gMiZqlP3OFcm/cd1chQc9zUWaKyboYlCYR0glqLAIDbhkVEUzAIRKDndFdEQkoQqMLGPBJjSJYyIGGndM1+tpXMwIQl3zjFnnQU/zk578yU5/sdOCfcJAJ8leQupVyiRRibyn5w4TKF3u/ysDs/VyX9MjiA3Y98Mj9G/Q2al7gI/Bx0fWIkrWLeu2tWl5VsNqWc+tI6ttUbtqN+yW/dD55HxxvjrfFqWOvey5aa3FRuk7s34ZhQ==</latexit><latexit sha1_base64="npLzmI/AO9sb0eZLeasrSgp90SQ=">AAADTnicfVHdbtMwFE4yfkr5a+GSG4sKabCpSjZtXUGTyo+ACxCbRLtJdRs5rtO6S5zIcWgry0/AC/AO3CLxGtzyItwhOGkLtPwdRfbn852TY39fkEY8U6772XY2zp2/cLF0qXz5ytVr1yvVG50sySVlbZpEiTwNSMYiLlhbcRWx01QyEgcROwnOHhf8yRsmM56I12qWsl5MhoKHnBIFKb/yHis2VfppIhEjdITIkAmFDBqjBYFIcVLbCMMXxBoPWKSI6Wu15Rl/jA4RJnIYc+HrFdq8gOO8X0nChTEIRyxUmwhDVTj1x32+PStWbHzNDz34n/DH5gGQE/9ZX6Gt1WEISz4cqbt+pebW3f295q6L3Pqe6zWaTQCuu3+wu4M8AEXUWvX7Hz5uH7878qv2WzxIaB7Dm2hEsqzruanqaSIVpxEzZZxnLCX0DF7dBShIzLKenqtq0B3IDFAIyoQJaDLPrnZoEmfZLA6gMiZqlP3OFcm/cd1chQc9zUWaKyboYlCYR0glqLAIDbhkVEUzAIRKDndFdEQkoQqMLGPBJjSJYyIGGndM1+tpXMwIQl3zjFnnQU/zk578yU5/sdOCfcJAJ8leQupVyiRRibyn5w4TKF3u/ysDs/VyX9MjiA3Y98Mj9G/Q2al7gI/Bx0fWIkrWLeu2tWl5VsNqWc+tI6ttUbtqN+yW/dD55HxxvjrfFqWOvey5aa3FRuk7s34ZhQ==</latexit><latexit sha1_base64="95jNTNhuLk8aocbZHDfBsjQS4/o=">AAADTnicfVFbb9MwFHY6LqVc1sIjLxYV0mBTlWzaugpNKhcBDyCGRLtJdRs5rtO6i5PIcWgry7+An8Kf4ZU/whuCk7ZAy+0osj+f75wc+/uCNBKZdt3PTmnr0uUrV8vXKtdv3Ly1Xa3d7mZJrhjvsCRK1HlAMx6JmHe00BE/TxWnMoj4WXDxtODP3nOViSR+p+cp70s6ikUoGNWQ8qsfieYzbZ4nCnPKxpiOeKyxxRO8JDAtTnoPE/gCaciQR5ragdG7nvUn+AQTqkZSxL5Zo+0rOC76taIithaTiId6BxOoCmf+ZCD25sVKrG/EiQf/i/2JfQTk1H8x0Hh3fRgmSozG+oFfrbsN9+iwdeBit3Hoes1WC4DrHh0f7GMPQBF1tIpTv+Z8IMOE5RLexCKaZT3PTXXfUKUFi7itkDzjKWUX8OoewJhKnvXNQlWL70NmiENQJkxAk0V2vcNQmWVzGUClpHqc/c4Vyb9xvVyHx30j4jTXPGbLQWEeYZ3gwiI8FIozHc0BUKYE3BWzMVWUaTCyQmI+ZYmUNB4a0rU9r29IMSMITd2zdpMHPe1PevonO/vFzgr2GQedFH8NqTcpV1Qn6qFZOEyhdLX/rwzMNqt9Q49AWrDvh0f436C73/AAv3Xr7ScrI8voLrqHdpCHmqiNXqJT1EHMqTlNp+08Ln0qfSl9LX1blpacVc8dtBFb5e8qOBZd</latexit>

2

Server

Federated learning

wtG


. . . . . . . . . Compute

�t+11


Compute

�t+12


�t+1k


Compute


�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �


2

Server

Federated learning

wtG


. . . . . . . . . Compute

�t+11


Compute

�t+12


�t+1k


Compute


�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �


�t+11

<latexit sha1_base64="THWwDZVG/mPRoQRJIW6OQtXKePg=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+uSZO60oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6ObIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrVcwUbWp/ZkUAYx2MEGNBs4aD8JKA6fy8u3KP/0EPujafcBlAzMrzp2utBKYpHl/v5A2FiUYFN2cf4z4knfz/oDl7NXogI8py0eMj/kokeGIMzakPGdrDMgGx/O97GlR1qq14FAZEcKUswZnUXjUykC3U7QBGqEuxTlME3XCQpjF9fUdfZ6Ukla1T88hXat/b0RhQ1hamSatwItw01uJ//KmLVbjWdSuaRGcugqqWkOxpqsqaKk9KDTLRITyOt1K1YXwQmEq7FqKtOkPDhaqtla4MhayWnSxWEXKKi66G65VRxtXCROPkp0q/dMb/T85Geac5fz9weDwzabcbfKEPCMvCCevySF5R47JhCiyJJ/JF/I1+5b9yH5mv65Ge9lmZ59cQ2/rN4vQs7I=</latexit><latexit sha1_base64="THWwDZVG/mPRoQRJIW6OQtXKePg=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+uSZO60oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6ObIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrVcwUbWp/ZkUAYx2MEGNBs4aD8JKA6fy8u3KP/0EPujafcBlAzMrzp2utBKYpHl/v5A2FiUYFN2cf4z4knfz/oDl7NXogI8py0eMj/kokeGIMzakPGdrDMgGx/O97GlR1qq14FAZEcKUswZnUXjUykC3U7QBGqEuxTlME3XCQpjF9fUdfZ6Ukla1T88hXat/b0RhQ1hamSatwItw01uJ//KmLVbjWdSuaRGcugqqWkOxpqsqaKk9KDTLRITyOt1K1YXwQmEq7FqKtOkPDhaqtla4MhayWnSxWEXKKi66G65VRxtXCROPkp0q/dMb/T85Geac5fz9weDwzabcbfKEPCMvCCevySF5R47JhCiyJJ/JF/I1+5b9yH5mv65Ge9lmZ59cQ2/rN4vQs7I=</latexit><latexit sha1_base64="THWwDZVG/mPRoQRJIW6OQtXKePg=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+uSZO60oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6ObIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrVcwUbWp/ZkUAYx2MEGNBs4aD8JKA6fy8u3KP/0EPujafcBlAzMrzp2utBKYpHl/v5A2FiUYFN2cf4z4knfz/oDl7NXogI8py0eMj/kokeGIMzakPGdrDMgGx/O97GlR1qq14FAZEcKUswZnUXjUykC3U7QBGqEuxTlME3XCQpjF9fUdfZ6Ukla1T88hXat/b0RhQ1hamSatwItw01uJ//KmLVbjWdSuaRGcugqqWkOxpqsqaKk9KDTLRITyOt1K1YXwQmEq7FqKtOkPDhaqtla4MhayWnSxWEXKKi66G65VRxtXCROPkp0q/dMb/T85Geac5fz9weDwzabcbfKEPCMvCCevySF5R47JhCiyJJ/JF/I1+5b9yH5mv65Ge9lmZ59cQ2/rN4vQs7I=</latexit><latexit sha1_base64="THWwDZVG/mPRoQRJIW6OQtXKePg=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+uSZO60oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6ObIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrVcwUbWp/ZkUAYx2MEGNBs4aD8JKA6fy8u3KP/0EPujafcBlAzMrzp2utBKYpHl/v5A2FiUYFN2cf4z4knfz/oDl7NXogI8py0eMj/kokeGIMzakPGdrDMgGx/O97GlR1qq14FAZEcKUswZnUXjUykC3U7QBGqEuxTlME3XCQpjF9fUdfZ6Ukla1T88hXat/b0RhQ1hamSatwItw01uJ//KmLVbjWdSuaRGcugqqWkOxpqsqaKk9KDTLRITyOt1K1YXwQmEq7FqKtOkPDhaqtla4MhayWnSxWEXKKi66G65VRxtXCROPkp0q/dMb/T85Geac5fz9weDwzabcbfKEPCMvCCevySF5R47JhCiyJJ/JF/I1+5b9yH5mv65Ge9lmZ59cQ2/rN4vQs7I=</latexit>

�t+12

<latexit sha1_base64="Dcc7Q2VKU1KHXAQxRXbwwL47fOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+tyk8m0oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6OaIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrZdqImtT+zMBQRnt1AQ1GnXWeAVWGHUqLt+u/NNPygdduw+4bNTMwrnTlZaASZr39wthY1Eqg9DNhx8jvuTdvD9gOXs1OuBjyvIR42M+SmQ44owNKc/ZGgOywfF8L3talLVsrXIoDYQw5azBWQSPWhrV7RRtUA3ISzhX00QdWBVmcX19R58npaRV7dNzSNfq3xsRbAhLK9KkBbwIN72V+C9v2mI1nkXtmhaVk1dBVWso1nRVBS21VxLNMhGQXqdbqbwADxJTYddShE1/cGoha2vBlbEQ1aKLxSpSVHHR3XCtPNq4Ekw8Snaq9E9v9P/kZJhzlvP3B4PDN5tyt8kT8oy8IJy8JofkHTkmEyLJknwmX8jX7Fv2I/uZ/boa7WWbnX1yDb2t342os7M=</latexit><latexit sha1_base64="Dcc7Q2VKU1KHXAQxRXbwwL47fOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+tyk8m0oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6OaIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrZdqImtT+zMBQRnt1AQ1GnXWeAVWGHUqLt+u/NNPygdduw+4bNTMwrnTlZaASZr39wthY1Eqg9DNhx8jvuTdvD9gOXs1OuBjyvIR42M+SmQ44owNKc/ZGgOywfF8L3talLVsrXIoDYQw5azBWQSPWhrV7RRtUA3ISzhX00QdWBVmcX19R58npaRV7dNzSNfq3xsRbAhLK9KkBbwIN72V+C9v2mI1nkXtmhaVk1dBVWso1nRVBS21VxLNMhGQXqdbqbwADxJTYddShE1/cGoha2vBlbEQ1aKLxSpSVHHR3XCtPNq4Ekw8Snaq9E9v9P/kZJhzlvP3B4PDN5tyt8kT8oy8IJy8JofkHTkmEyLJknwmX8jX7Fv2I/uZ/boa7WWbnX1yDb2t342os7M=</latexit><latexit sha1_base64="Dcc7Q2VKU1KHXAQxRXbwwL47fOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+tyk8m0oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6OaIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrZdqImtT+zMBQRnt1AQ1GnXWeAVWGHUqLt+u/NNPygdduw+4bNTMwrnTlZaASZr39wthY1Eqg9DNhx8jvuTdvD9gOXs1OuBjyvIR42M+SmQ44owNKc/ZGgOywfF8L3talLVsrXIoDYQw5azBWQSPWhrV7RRtUA3ISzhX00QdWBVmcX19R58npaRV7dNzSNfq3xsRbAhLK9KkBbwIN72V+C9v2mI1nkXtmhaVk1dBVWso1nRVBS21VxLNMhGQXqdbqbwADxJTYddShE1/cGoha2vBlbEQ1aKLxSpSVHHR3XCtPNq4Ekw8Snaq9E9v9P/kZJhzlvP3B4PDN5tyt8kT8oy8IJy8JofkHTkmEyLJknwmX8jX7Fv2I/uZ/boa7WWbnX1yDb2t342os7M=</latexit><latexit sha1_base64="Dcc7Q2VKU1KHXAQxRXbwwL47fOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQRhSJau7mPRPvhYwW0LO+tyk8m0oUlmSO64LGF+i6/6X/wD/g3fxBez2xVs0QOBwzn3cm6OaIwOyNj3rHdr6/adu9v3du4/ePhot7/3+CTUrZdqImtT+zMBQRnt1AQ1GnXWeAVWGHUqLt+u/NNPygdduw+4bNTMwrnTlZaASZr39wthY1Eqg9DNhx8jvuTdvD9gOXs1OuBjyvIR42M+SmQ44owNKc/ZGgOywfF8L3talLVsrXIoDYQw5azBWQSPWhrV7RRtUA3ISzhX00QdWBVmcX19R58npaRV7dNzSNfq3xsRbAhLK9KkBbwIN72V+C9v2mI1nkXtmhaVk1dBVWso1nRVBS21VxLNMhGQXqdbqbwADxJTYddShE1/cGoha2vBlbEQ1aKLxSpSVHHR3XCtPNq4Ekw8Snaq9E9v9P/kZJhzlvP3B4PDN5tyt8kT8oy8IJy8JofkHTkmEyLJknwmX8jX7Fv2I/uZ/boa7WWbnX1yDb2t342os7M=</latexit>

�t+1k

<latexit sha1_base64="QNnRc5vIMqBoGSzUBrCzadhzDUg=">AAACSnicdVDdShwxGM2stlr746qXvWjoUigUhmRxdS9Fveilha4KO9vlSyajYZPMkGRcljDP4m37Lr6Ar+Fd8cbsuoUq7YHA4Zzv43w5rFLSeUJuk9bK6ouXa+uvNl6/eftus721ferK2nIx4KUq7TkDJ5Q0YuClV+K8sgI0U+KMTY7m/tmVsE6W5rufVWKk4cLIQnLwURq3dzKmQ5YL5aEZT34E/4U243aHpGSvt0v7mKQ9Qvu0F0m3RwnpYpqSBTpoiZPxVvIhy0tea2E8V+DckJLKjwJYL7kSzUZWO1EBn8CFGEZqQAs3CovrG/wpKjkuShuf8Xih/r0RQDs30yxOavCX7rk3F//lDWtf9EdBmqr2wvDHoKJW2Jd4XgXOpRXcq1kkwK2Mt2J+CRa4j4U9SWE6/sGIKS+1BpOHjBXTJmTzSFaEafPM1fx46XJQ4TjasdI/veH/k9NuSklKv+12Dg6X5a6j9+gj+owo2kcH6Cs6QQPE0Qxdo5/oV3KT3CW/k/vH0Vay3NlBT9BafQD2wLPs</latexit><latexit sha1_base64="QNnRc5vIMqBoGSzUBrCzadhzDUg=">AAACSnicdVDdShwxGM2stlr746qXvWjoUigUhmRxdS9Fveilha4KO9vlSyajYZPMkGRcljDP4m37Lr6Ar+Fd8cbsuoUq7YHA4Zzv43w5rFLSeUJuk9bK6ouXa+uvNl6/eftus721ferK2nIx4KUq7TkDJ5Q0YuClV+K8sgI0U+KMTY7m/tmVsE6W5rufVWKk4cLIQnLwURq3dzKmQ5YL5aEZT34E/4U243aHpGSvt0v7mKQ9Qvu0F0m3RwnpYpqSBTpoiZPxVvIhy0tea2E8V+DckJLKjwJYL7kSzUZWO1EBn8CFGEZqQAs3CovrG/wpKjkuShuf8Xih/r0RQDs30yxOavCX7rk3F//lDWtf9EdBmqr2wvDHoKJW2Jd4XgXOpRXcq1kkwK2Mt2J+CRa4j4U9SWE6/sGIKS+1BpOHjBXTJmTzSFaEafPM1fx46XJQ4TjasdI/veH/k9NuSklKv+12Dg6X5a6j9+gj+owo2kcH6Cs6QQPE0Qxdo5/oV3KT3CW/k/vH0Vay3NlBT9BafQD2wLPs</latexit><latexit sha1_base64="QNnRc5vIMqBoGSzUBrCzadhzDUg=">AAACSnicdVDdShwxGM2stlr746qXvWjoUigUhmRxdS9Fveilha4KO9vlSyajYZPMkGRcljDP4m37Lr6Ar+Fd8cbsuoUq7YHA4Zzv43w5rFLSeUJuk9bK6ouXa+uvNl6/eftus721ferK2nIx4KUq7TkDJ5Q0YuClV+K8sgI0U+KMTY7m/tmVsE6W5rufVWKk4cLIQnLwURq3dzKmQ5YL5aEZT34E/4U243aHpGSvt0v7mKQ9Qvu0F0m3RwnpYpqSBTpoiZPxVvIhy0tea2E8V+DckJLKjwJYL7kSzUZWO1EBn8CFGEZqQAs3CovrG/wpKjkuShuf8Xih/r0RQDs30yxOavCX7rk3F//lDWtf9EdBmqr2wvDHoKJW2Jd4XgXOpRXcq1kkwK2Mt2J+CRa4j4U9SWE6/sGIKS+1BpOHjBXTJmTzSFaEafPM1fx46XJQ4TjasdI/veH/k9NuSklKv+12Dg6X5a6j9+gj+owo2kcH6Cs6QQPE0Qxdo5/oV3KT3CW/k/vH0Vay3NlBT9BafQD2wLPs</latexit><latexit sha1_base64="QNnRc5vIMqBoGSzUBrCzadhzDUg=">AAACSnicdVDdShwxGM2stlr746qXvWjoUigUhmRxdS9Fveilha4KO9vlSyajYZPMkGRcljDP4m37Lr6Ar+Fd8cbsuoUq7YHA4Zzv43w5rFLSeUJuk9bK6ouXa+uvNl6/eftus721ferK2nIx4KUq7TkDJ5Q0YuClV+K8sgI0U+KMTY7m/tmVsE6W5rufVWKk4cLIQnLwURq3dzKmQ5YL5aEZT34E/4U243aHpGSvt0v7mKQ9Qvu0F0m3RwnpYpqSBTpoiZPxVvIhy0tea2E8V+DckJLKjwJYL7kSzUZWO1EBn8CFGEZqQAs3CovrG/wpKjkuShuf8Xih/r0RQDs30yxOavCX7rk3F//lDWtf9EdBmqr2wvDHoKJW2Jd4XgXOpRXcq1kkwK2Mt2J+CRa4j4U9SWE6/sGIKS+1BpOHjBXTJmTzSFaEafPM1fx46XJQ4TjasdI/veH/k9NuSklKv+12Dg6X5a6j9+gj+owo2kcH6Cs6QQPE0Qxdo5/oV3KT3CW/k/vH0Vay3NlBT9BafQD2wLPs</latexit>

2

Server

Federated learning

. . . . . . . . . Compute

�t+11


Compute

�t+12


�t+1k


Compute


�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �


�t+11


�t+12


�t+1k


2

Server

Federated learning

. . . . . . . . . Compute

�t+11


Compute

�t+12


�t+1k


Compute


�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �


�t+11


�t+12


�t+1k


wt+1G = wt

G +kX

j=1

↵j�t+1j

<latexit sha1_base64="OGBemspsUbiFWogaMpCReQJCYoM=">AAACe3icdVBda9swFFW8bu26j6bb4x4qFgZjHcEeK+tLoWyB9rGFpi3EqbmW5UaJJBvpeiEI/6H9mr12/2UwOXGh7bYLQkfnnMvVPWkphcUwvOkEj9YeP1nfeLr57PmLl1vd7VfntqgM40NWyMJcpmC5FJoPUaDkl6XhoFLJL9LZt0a/+M6NFYU+w0XJxwqutcgFA/RU0h3EaT5Pjq4c7kb1AW1fSHdpbCuVuOlBVF+5WU1jkOUEkqm3KBdnXCLUyXTVl3R7YX8/bIr+DaL+8g57pK2TZLuzE2cFqxTXyCRYO4rCEscODAomeb0ZV5aXwGZwzUcealDcjt1y3Zq+80xG88L4o5Eu2bsdDpS1C5V6pwKc2IdaQ/5LG1WY74+d0GWFXLPVoLySFAvaZEczYThDufAAmBH+r5RNwABDn/C9KanyO2g+Z4VSoDPX5Fq7uBmZ5m5eP1AVG7QqA+kGXvaR3uZG/w/OP/WjsB+dfu4dfm3D3SBvyFvynkTkCzkkx+SEDAkjP8hPckN+dX4HveBD8HFlDTptz2tyr4K9P/QexAg=</latexit><latexit sha1_base64="OGBemspsUbiFWogaMpCReQJCYoM=">AAACe3icdVBda9swFFW8bu26j6bb4x4qFgZjHcEeK+tLoWyB9rGFpi3EqbmW5UaJJBvpeiEI/6H9mr12/2UwOXGh7bYLQkfnnMvVPWkphcUwvOkEj9YeP1nfeLr57PmLl1vd7VfntqgM40NWyMJcpmC5FJoPUaDkl6XhoFLJL9LZt0a/+M6NFYU+w0XJxwqutcgFA/RU0h3EaT5Pjq4c7kb1AW1fSHdpbCuVuOlBVF+5WU1jkOUEkqm3KBdnXCLUyXTVl3R7YX8/bIr+DaL+8g57pK2TZLuzE2cFqxTXyCRYO4rCEscODAomeb0ZV5aXwGZwzUcealDcjt1y3Zq+80xG88L4o5Eu2bsdDpS1C5V6pwKc2IdaQ/5LG1WY74+d0GWFXLPVoLySFAvaZEczYThDufAAmBH+r5RNwABDn/C9KanyO2g+Z4VSoDPX5Fq7uBmZ5m5eP1AVG7QqA+kGXvaR3uZG/w/OP/WjsB+dfu4dfm3D3SBvyFvynkTkCzkkx+SEDAkjP8hPckN+dX4HveBD8HFlDTptz2tyr4K9P/QexAg=</latexit><latexit sha1_base64="OGBemspsUbiFWogaMpCReQJCYoM=">AAACe3icdVBda9swFFW8bu26j6bb4x4qFgZjHcEeK+tLoWyB9rGFpi3EqbmW5UaJJBvpeiEI/6H9mr12/2UwOXGh7bYLQkfnnMvVPWkphcUwvOkEj9YeP1nfeLr57PmLl1vd7VfntqgM40NWyMJcpmC5FJoPUaDkl6XhoFLJL9LZt0a/+M6NFYU+w0XJxwqutcgFA/RU0h3EaT5Pjq4c7kb1AW1fSHdpbCuVuOlBVF+5WU1jkOUEkqm3KBdnXCLUyXTVl3R7YX8/bIr+DaL+8g57pK2TZLuzE2cFqxTXyCRYO4rCEscODAomeb0ZV5aXwGZwzUcealDcjt1y3Zq+80xG88L4o5Eu2bsdDpS1C5V6pwKc2IdaQ/5LG1WY74+d0GWFXLPVoLySFAvaZEczYThDufAAmBH+r5RNwABDn/C9KanyO2g+Z4VSoDPX5Fq7uBmZ5m5eP1AVG7QqA+kGXvaR3uZG/w/OP/WjsB+dfu4dfm3D3SBvyFvynkTkCzkkx+SEDAkjP8hPckN+dX4HveBD8HFlDTptz2tyr4K9P/QexAg=</latexit><latexit sha1_base64="OGBemspsUbiFWogaMpCReQJCYoM=">AAACe3icdVBda9swFFW8bu26j6bb4x4qFgZjHcEeK+tLoWyB9rGFpi3EqbmW5UaJJBvpeiEI/6H9mr12/2UwOXGh7bYLQkfnnMvVPWkphcUwvOkEj9YeP1nfeLr57PmLl1vd7VfntqgM40NWyMJcpmC5FJoPUaDkl6XhoFLJL9LZt0a/+M6NFYU+w0XJxwqutcgFA/RU0h3EaT5Pjq4c7kb1AW1fSHdpbCuVuOlBVF+5WU1jkOUEkqm3KBdnXCLUyXTVl3R7YX8/bIr+DaL+8g57pK2TZLuzE2cFqxTXyCRYO4rCEscODAomeb0ZV5aXwGZwzUcealDcjt1y3Zq+80xG88L4o5Eu2bsdDpS1C5V6pwKc2IdaQ/5LG1WY74+d0GWFXLPVoLySFAvaZEczYThDufAAmBH+r5RNwABDn/C9KanyO2g+Z4VSoDPX5Fq7uBmZ5m5eP1AVG7QqA+kGXvaR3uZG/w/OP/WjsB+dfu4dfm3D3SBvyFvynkTkCzkkx+SEDAkjP8hPckN+dX4HveBD8HFlDTptz2tyr4K9P/QexAg=</latexit>

2

Federated learning with a malicious agent

3


Server

3


Server

wtG


3


Server

wtG


. . . . . . . . .

3

Threat model - Single malicious agent - Data is i.i.d. across agents - Malicious agent has a fraction of

benign data


Server

wtG


. . . . . . . . .

3


benign data


wtG




Server

wtG


. . . . . . . . .

3


benign data


Server

wtG


. . . . . . . . .

3


benign data


Server

wtG


. . . . . . . . . Compute

�t+11


Compute

�t+12


8j 6= m, �t+1j = argmin

�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �

�<latexit sha1_base64="GHZBI5SNerlo3b36fzxvNemNrjg=">AAADOHicfVHLbtQwFHXCq5TXFJZsDBVSaatR0qovUKUKkGABokj0IY1nIsfjTD21neA4zIwsfwFr/oQVK36DHTvEli0bbqYFZnhdJfHJOde+1jlpIUVpo+hjEJ45e+78hZmLs5cuX7l6rTF3fb/MK8P4Hstlbg5TWnIpNN+zwkp+WBhOVSr5QXr8sNYPXnNTily/tKOCtxXtaZEJRi1QSeMtyXJDpcR9TDR/hdUyJvCkypEul5b6jrNLsU/6eBsTanpK6MRNyP4p/Fo+tM4aKrT3mEie2QVMoCsbJv2OWB7VX+ITJ7ZjOE8nfX8fxEHyuGPx0uQwTIzoHdm7SWM+akbra1urEY6aa1G8sbUFIIrWN1dXcAygrvmd5r13HxBCu8lc8IZ0c1Ypri2TtCxbcVTYtqPGCia5nyVVyQvKjmmPtwBqqnjZdmP/PL4DTBeDEfBqi8fs5A5HVVmOVAqditqj8netJv+mtSqbbbad0EVluWYng7JKYpvjOgzcFYYzK0cAKDMC7orZETWUWYhsFgIZsFwpqruO7PtW3HaknpFmbj72floHP/1PefCnOvylDmv1EQefDH8G1POCG2pzs+jGCVNoPV3/1wZhu9N1yo9UeYjvR0b432B/pRkDfgE5PkAnNYNuottoAcVoA+2gJ2gX7SGGvgW3gsVgKXwffgo/h19OWsPgdM8NNFXh1++CvhGi</latexit><latexit sha1_base64="dmUmBJIzwj0rRjPg+LrarRCdajg=">AAADOHicfVHLbhMxFPUMr1JeLSzZGCKk0lbRTKu+QJUqQIIFiFYibaU4GXkcT+rU9gwehySy/AWskfgQJCT+hB07xJYtG+6kAVJeVzPjM+dc+1rnpIUUpY2ij0F45uy58xdmLs5eunzl6rW5+ev7Zd43jDdYLnNzmNKSS6F5wwor+WFhOFWp5Afp8cNKP3jFTSly/cKOCt5StKtFJhi1QCVzb0iWGyol7mGi+UusljGBJ1WOdLi01LedXYp90sPbmFDTVUInbkr2T+HX8qF11lChvcdE8swuYAJd2TDptcXyqPoSnzixHcN5Oun5+yAOksdti5emh2FiRPfI3k3malE9Wl/bWo1wVF+L4o2tLQBRtL65uoJjAFXVdur33r1f3nu7m8wHr0knZ33FtWWSlmUzjgrbctRYwST3s6Rf8oKyY9rlTYCaKl623Ng/j+8A08FgBLza4jE7vcNRVZYjlUKnovao/F2ryL9pzb7NNltO6KJvuWYng7K+xDbHVRi4IwxnVo4AUGYE3BWzI2oosxDZLAQyYLlSVHcc2ffNuOVINSPNXC32/rQOfvqf8uBPdfhLHVbqIw4+Gf4MqOcFN9TmZtGNE6bQOln/1wZhu8l6yo9UeYjvR0b432B/pR4D3oMcH6CTmkE30W20gGK0gXbQE7SLGoihb8GtYDFYCj+En8LP4ZeT1jCY7LmBTlX49TurHhKu</latexit><latexit sha1_base64="dmUmBJIzwj0rRjPg+LrarRCdajg=">AAADOHicfVHLbhMxFPUMr1JeLSzZGCKk0lbRTKu+QJUqQIIFiFYibaU4GXkcT+rU9gwehySy/AWskfgQJCT+hB07xJYtG+6kAVJeVzPjM+dc+1rnpIUUpY2ij0F45uy58xdmLs5eunzl6rW5+ev7Zd43jDdYLnNzmNKSS6F5wwor+WFhOFWp5Afp8cNKP3jFTSly/cKOCt5StKtFJhi1QCVzb0iWGyol7mGi+UusljGBJ1WOdLi01LedXYp90sPbmFDTVUInbkr2T+HX8qF11lChvcdE8swuYAJd2TDptcXyqPoSnzixHcN5Oun5+yAOksdti5emh2FiRPfI3k3malE9Wl/bWo1wVF+L4o2tLQBRtL65uoJjAFXVdur33r1f3nu7m8wHr0knZ33FtWWSlmUzjgrbctRYwST3s6Rf8oKyY9rlTYCaKl623Ng/j+8A08FgBLza4jE7vcNRVZYjlUKnovao/F2ryL9pzb7NNltO6KJvuWYng7K+xDbHVRi4IwxnVo4AUGYE3BWzI2oosxDZLAQyYLlSVHcc2ffNuOVINSPNXC32/rQOfvqf8uBPdfhLHVbqIw4+Gf4MqOcFN9TmZtGNE6bQOln/1wZhu8l6yo9UeYjvR0b432B/pR4D3oMcH6CTmkE30W20gGK0gXbQE7SLGoihb8GtYDFYCj+En8LP4ZeT1jCY7LmBTlX49TurHhKu</latexit><latexit sha1_base64="S6l1zS0X+9dfzcsF1cE8gOMV+mA=">AAADOHicfVHLbhMxFHWGVwmPtrBkY4iQSlNVM636iFClCpBgAaJINK0UJyOP40mc2p7B45BElr+ANT/Dn7Bjh9iyZcOdNEDC62pmfOaca1/rnCSXorBh+LESXLh46fKVpavVa9dv3FxeWb3VLLKhYfyYZTIzpwktuBSaH1thJT/NDacqkfwkOXtc6idvuSlEpl/bSc7biva0SAWjFqh45T1JM0OlxANMNH+D1QYm8CTKkS6XlvqOs/XIxwN8gAk1PSV07OZk/xx+LR9bZw0V2ntMJE/tGibQlY7jQUdsTMov8bETBxGcp+OBfwjiKH7asbg+PwwTI3p9+yBeqYWb4e5OYzvE4eZOGO01GgDCcHd/ewtHAMqqoVkdxauVd6SbsaHi2jJJi6IVhbltO2qsYJL7KhkWPKfsjPZ4C6CmihdtN/XP4/vAdDEYAa+2eMrO73BUFcVEJdCpqO0Xv2sl+TetNbTpftsJnQ8t1+x8UDqU2Ga4DAN3heHMygkAyoyAu2LWp4YyC5FVIZARy5SiuutI07eitiPljCR1tcj7RR389D/l0Z/q+Jc6LtUnHHwy/AVQL3NuqM3MupsmTKF1tv6vDcJ2s3XBj0R5iO9HRvjfoLm1GQF+FdYOH82CXEJ30D20hiK0hw7RM3SEjhFD3yp3K+uVevAh+BR8Dr6ctwaV2Z7baKGCr98BIdgPhg==</latexit>

3


benign data


Server

wtG


. . . . . . . . . Compute

�t+11


Compute

�t+12



�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �


Compute

�t+1m

<latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit><latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit><latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit><latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit>

�t+1m = A

�{xi

m, yim}nmi=1, {x

l, T l}nmall=1 ;wt

G + ��

<latexit sha1_base64="6ANt6WdN7mGWkLZTWins3aluZbI=">AAADL3icfZFdaxNBFIZn168aP5rqjeDN2CC0NpSsNy1IoX6A3ogVmrSQSdbZyWwydGZ2mT1rEof5BQr+Fn+NeCPeeusvcDYJ2jTigd15Oc97mOG8SS5FAa3WtyC8dPnK1Wtr12s3bt66vV7fuNMpstIw3maZzMxpQgsuheZtECD5aW44VYnkJ8nZ84qfvOemEJk+hmnOe4oOtUgFo+Bbcf0TSZQlAy6Bur6FncjFCh9goiiMGJX2qcNE8hS2MLEkSSex6ovmtPoTF1txEPkpHSvXnOO+bB73ZYXkAlkCfAJWUemce+I94/hlH/AOPncxJkYMR7Ad1xut3das8KqIFqJxuP35wzuE0FG8EXwkg4yVimtgkhZFN2rl0LPUgGCSuxopC55TdkaHvOulpooXPTtbm8MPfWeA08z4TwOedc9PWKqKYqoS76zWUVxkVfNfrFtCut+zQuclcM3mF6WlxJDhKgM8EIYzkFMvKDPCvxWzETWUgU+qRjQfs0wpqgeWdFw36tlZHElqG5Fzy9zv0/3B41U6+UsnFX3B/Z4Mf+1bb3JuKGTmkSXUDBX11sX5P5vQc5s/l/aRKOfjiy6GtSo6j3cjr9/6HJ+hea2h+2gTbaEI7aFD9AodoTZi6FdwL3gQbIZfwq/h9/DH3BoGi5m7aKnCn78BkD8OIA==</latexit><latexit sha1_base64="XVCJ1cGRruwHIcZtSVJOhzSPkJs=">AAADL3icfZFdixMxFIYz41etX129EbyJW4RdtywdbxRkYf0AvRFX2HYXmnbIpJk2bJIZMmdsa8gvUPDWX+C9v0a8EW+99ReYaYtut+KBmbyc5z0knDfJpSig3f4WhOfOX7h4qXa5fuXqtes3Ghs3u0VWGsY7LJOZOU5owaXQvAMCJD/ODacqkfwoOXlW8aO33BQi04cwy3lf0ZEWqWAUfCtufCCJsmTIJVA3sLATuVjhPUwUhTGj0j5xmEiewhYmliTpNFYD0ZpVf+JiK/YiP6Vj5VoLPJCtw4GskFwiS4BPwSoqnXOPvWcSvxgA3sGnLsbEiNEYtuNGs73bnhdeF9FSNPe3P77La58/HcQbwXsyzFipuAYmaVH0onYOfUsNCCa5q5Oy4DllJ3TEe15qqnjRt/O1OXzPd4Y4zYz/NOB59/SEpaooZirxzmodxVlWNf/FeiWkj/pW6LwErtniorSUGDJcZYCHwnAGcuYFZUb4t2I2poYy8EnVieYTlilF9dCSrutFfTuPI0ltM3Julft9uj94sk6nf+m0os+535Phr3zrdc4Nhczct4SakaLeujz/ZxN6YfPnyj4S5Xx80dmw1kX3wW7k9Ruf41O0qBq6gzbRForQQ7SPXqID1EEM/QpuB3eDzfBL+DX8Hv5YWMNgOXMLrVT48zdRXA92</latexit><latexit sha1_base64="XVCJ1cGRruwHIcZtSVJOhzSPkJs=">AAADL3icfZFdixMxFIYz41etX129EbyJW4RdtywdbxRkYf0AvRFX2HYXmnbIpJk2bJIZMmdsa8gvUPDWX+C9v0a8EW+99ReYaYtut+KBmbyc5z0knDfJpSig3f4WhOfOX7h4qXa5fuXqtes3Ghs3u0VWGsY7LJOZOU5owaXQvAMCJD/ODacqkfwoOXlW8aO33BQi04cwy3lf0ZEWqWAUfCtufCCJsmTIJVA3sLATuVjhPUwUhTGj0j5xmEiewhYmliTpNFYD0ZpVf+JiK/YiP6Vj5VoLPJCtw4GskFwiS4BPwSoqnXOPvWcSvxgA3sGnLsbEiNEYtuNGs73bnhdeF9FSNPe3P77La58/HcQbwXsyzFipuAYmaVH0onYOfUsNCCa5q5Oy4DllJ3TEe15qqnjRt/O1OXzPd4Y4zYz/NOB59/SEpaooZirxzmodxVlWNf/FeiWkj/pW6LwErtniorSUGDJcZYCHwnAGcuYFZUb4t2I2poYy8EnVieYTlilF9dCSrutFfTuPI0ltM3Julft9uj94sk6nf+m0os+535Phr3zrdc4Nhczct4SakaLeujz/ZxN6YfPnyj4S5Xx80dmw1kX3wW7k9Ruf41O0qBq6gzbRForQQ7SPXqID1EEM/QpuB3eDzfBL+DX8Hv5YWMNgOXMLrVT48zdRXA92</latexit><latexit sha1_base64="ZD/s+6XiWYtYQcLgw5gtKz9Q6P4=">AAADL3icfZFdaxNBFIZn168aP5rqjeDN2CBUG8quNwpSqB+gN2KFpi1kkmV2MpsMnZldZs+ahGF+gf4Zf414I9566y9wNlm0acQDu/Nynvcww3nTQooSouhbEF66fOXqtY3rrRs3b93ebG/dOS7zyjDeY7nMzWlKSy6F5j0QIPlpYThVqeQn6dmrmp985KYUuT6CecEHio61yASj4FtJ+zNJlSUjLoG6oYXd2CUK72OiKEwYlfaFw0TyDHYwsSTNZokaiu68/hOXWLEf+ymdKNdd4qHsHg1ljWSDLAE+A6uodM49955p8mYIeBefuxgTI8YTeJS0O9FetCi8LuJGdFBTh8lW8ImMclYproFJWpb9OCpgYKkBwSR3LVKVvKDsjI5530tNFS8HdrE2hx/6zghnufGfBrzonp+wVJXlXKXeWa+jvMjq5r9Yv4Ls2cAKXVTANVtelFUSQ47rDPBIGM5Azr2gzAj/Vswm1FAGPqkW0XzKcqWoHlly7PrxwC7iSDPbiZ1b5X6f7g+ertPZXzqr6Wvu92T4O996X3BDITePLaFmrKi3Nuf/bEIvbf5c2UeqnI8vvhjWujh+shd7/SHqHLxsgtxA99E22kExeooO0Ft0iHqIoV/BveBBsB1+Cb+G38MfS2sYNDN30UqFP38DyCwMTg==</latexit>

Attacker Objective Cause targeted misclassification of

an auxiliary set of examples for the global model

and ensure global model has good

performance

3


benign data


Server

wtG


. . . . . . . . . Compute

�t+11


Compute

�t+12



�Ltrain

�{xi

j , yij}

nj

i=1;wtG + �


�t+11


�t+12


�t+1m

<latexit sha1_base64="2tKnzEzLHiNILVXHjVMWUEK/uOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQrCkmjV+lZqH3ys4LaFne2SZO60oUlmSO64LGF+S1/b/+If8G/4Jr6Y3Y5gix4IHM65l3NzZG10QMa+Z707K3fv3V99sPbw0eMn6/2Np4eharyCkapM5Y+lCGC0gxFqNHBcexBWGjiS5x8X/tFX8EFX7gvOa5hYcep0qZXAJE37m7m0MS/AoGin9iTiK95O+wM2fMv4h3eMsiFbYkl2+BtOeacMSIeD6Ub2PC8q1VhwqIwIYcxZjZMoPGploF3LmwC1UOfiFMaJOmEhTOLy+pa+TEpBy8qn55Au1b83orAhzK1Mk1bgWbjtLcR/eeMGy51J1K5uEJy6DiobQ7GiiypooT0oNPNEhPI63UrVmfBCYSrsRoq06Q8OZqqyVrgi5rKctTFfRMoyztpbrlX7nauEifvJTpX+6Y3+nxy+HnI25J+3B7t7Xbmr5Bl5QbYIJ+/JLvlEDsiIKDInF+SSXGXfsh/Zz+zX9Wgv63Y2yQ30Vn4D1zWz2g==</latexit><latexit sha1_base64="2tKnzEzLHiNILVXHjVMWUEK/uOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQrCkmjV+lZqH3ys4LaFne2SZO60oUlmSO64LGF+S1/b/+If8G/4Jr6Y3Y5gix4IHM65l3NzZG10QMa+Z707K3fv3V99sPbw0eMn6/2Np4eharyCkapM5Y+lCGC0gxFqNHBcexBWGjiS5x8X/tFX8EFX7gvOa5hYcep0qZXAJE37m7m0MS/AoGin9iTiK95O+wM2fMv4h3eMsiFbYkl2+BtOeacMSIeD6Ub2PC8q1VhwqIwIYcxZjZMoPGploF3LmwC1UOfiFMaJOmEhTOLy+pa+TEpBy8qn55Au1b83orAhzK1Mk1bgWbjtLcR/eeMGy51J1K5uEJy6DiobQ7GiiypooT0oNPNEhPI63UrVmfBCYSrsRoq06Q8OZqqyVrgi5rKctTFfRMoyztpbrlX7nauEifvJTpX+6Y3+nxy+HnI25J+3B7t7Xbmr5Bl5QbYIJ+/JLvlEDsiIKDInF+SSXGXfsh/Zz+zX9Wgv63Y2yQ30Vn4D1zWz2g==</latexit><latexit sha1_base64="2tKnzEzLHiNILVXHjVMWUEK/uOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQrCkmjV+lZqH3ys4LaFne2SZO60oUlmSO64LGF+S1/b/+If8G/4Jr6Y3Y5gix4IHM65l3NzZG10QMa+Z707K3fv3V99sPbw0eMn6/2Np4eharyCkapM5Y+lCGC0gxFqNHBcexBWGjiS5x8X/tFX8EFX7gvOa5hYcep0qZXAJE37m7m0MS/AoGin9iTiK95O+wM2fMv4h3eMsiFbYkl2+BtOeacMSIeD6Ub2PC8q1VhwqIwIYcxZjZMoPGploF3LmwC1UOfiFMaJOmEhTOLy+pa+TEpBy8qn55Au1b83orAhzK1Mk1bgWbjtLcR/eeMGy51J1K5uEJy6DiobQ7GiiypooT0oNPNEhPI63UrVmfBCYSrsRoq06Q8OZqqyVrgi5rKctTFfRMoyztpbrlX7nauEifvJTpX+6Y3+nxy+HnI25J+3B7t7Xbmr5Bl5QbYIJ+/JLvlEDsiIKDInF+SSXGXfsh/Zz+zX9Wgv63Y2yQ30Vn4D1zWz2g==</latexit><latexit sha1_base64="2tKnzEzLHiNILVXHjVMWUEK/uOA=">AAACSnicdVBdaxQxFM2sVWv96NY++mBwEQrCkmjV+lZqH3ys4LaFne2SZO60oUlmSO64LGF+S1/b/+If8G/4Jr6Y3Y5gix4IHM65l3NzZG10QMa+Z707K3fv3V99sPbw0eMn6/2Np4eharyCkapM5Y+lCGC0gxFqNHBcexBWGjiS5x8X/tFX8EFX7gvOa5hYcep0qZXAJE37m7m0MS/AoGin9iTiK95O+wM2fMv4h3eMsiFbYkl2+BtOeacMSIeD6Ub2PC8q1VhwqIwIYcxZjZMoPGploF3LmwC1UOfiFMaJOmEhTOLy+pa+TEpBy8qn55Au1b83orAhzK1Mk1bgWbjtLcR/eeMGy51J1K5uEJy6DiobQ7GiiypooT0oNPNEhPI63UrVmfBCYSrsRoq06Q8OZqqyVrgi5rKctTFfRMoyztpbrlX7nauEifvJTpX+6Y3+nxy+HnI25J+3B7t7Xbmr5Bl5QbYIJ+/JLvlEDsiIKDInF+SSXGXfsh/Zz+zX9Wgv63Y2yQ30Vn4D1zWz2g==</latexit>

Compute

�t+1m

<latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit><latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit><latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit><latexit sha1_base64="Gzxzi/3mpo1xAgKqA2ZEeTIBX9s=">AAACSnicbVDLShxBFK2emPjIw1GXLmwcBCEwdIuQLCVxkaWBjArTk+FW9W0trKpuqm47DEV/S7bxX/ID+Q134saasRc6eqDgcM69nFuHV0o6SpL/UefN0tt3yyura+8/fPy03t3YPHVlbQUORKlKe87BoZIGByRJ4XllETRXeMavvs/8s2u0TpbmF00rHGm4MLKQAihI4+5WxrXPclQEzVj/9vQ5bcbdXtJP5ohfkrQlPdbiZLwR7WR5KWqNhoQC54ZpUtHIgyUpFDZrWe2wAnEFFzgM1IBGN/Lz65t4Lyh5XJQ2PEPxXH264UE7N9U8TGqgS7fozcTXvGFNxdeRl6aqCY14DCpqFVMZz6qIc2lRkJoGAsLKcGssLsGCoFDYsxSuwx8MTkSpNZjcZ7yYND6bRfLCT5oFV4vj1hWg/HGwQ6XpYoEvyelBP0366c/D3tG3ttwVts122T5L2Rd2xH6wEzZggk3ZH/aX3UT/otvoLrp/HO1E7c4We4bO0gObYLO4</latexit>

�t+1m = A

�{xi

m, yim}nmi=1, {x

l, T l}nmall=1 ;wt

G + ��

<latexit sha1_base64="6ANt6WdN7mGWkLZTWins3aluZbI=">AAADL3icfZFdaxNBFIZn168aP5rqjeDN2CC0NpSsNy1IoX6A3ogVmrSQSdbZyWwydGZ2mT1rEof5BQr+Fn+NeCPeeusvcDYJ2jTigd15Oc97mOG8SS5FAa3WtyC8dPnK1Wtr12s3bt66vV7fuNMpstIw3maZzMxpQgsuheZtECD5aW44VYnkJ8nZ84qfvOemEJk+hmnOe4oOtUgFo+Bbcf0TSZQlAy6Bur6FncjFCh9goiiMGJX2qcNE8hS2MLEkSSex6ovmtPoTF1txEPkpHSvXnOO+bB73ZYXkAlkCfAJWUemce+I94/hlH/AOPncxJkYMR7Ad1xut3das8KqIFqJxuP35wzuE0FG8EXwkg4yVimtgkhZFN2rl0LPUgGCSuxopC55TdkaHvOulpooXPTtbm8MPfWeA08z4TwOedc9PWKqKYqoS76zWUVxkVfNfrFtCut+zQuclcM3mF6WlxJDhKgM8EIYzkFMvKDPCvxWzETWUgU+qRjQfs0wpqgeWdFw36tlZHElqG5Fzy9zv0/3B41U6+UsnFX3B/Z4Mf+1bb3JuKGTmkSXUDBX11sX5P5vQc5s/l/aRKOfjiy6GtSo6j3cjr9/6HJ+hea2h+2gTbaEI7aFD9AodoTZi6FdwL3gQbIZfwq/h9/DH3BoGi5m7aKnCn78BkD8OIA==</latexit><latexit sha1_base64="XVCJ1cGRruwHIcZtSVJOhzSPkJs=">AAADL3icfZFdixMxFIYz41etX129EbyJW4RdtywdbxRkYf0AvRFX2HYXmnbIpJk2bJIZMmdsa8gvUPDWX+C9v0a8EW+99ReYaYtut+KBmbyc5z0knDfJpSig3f4WhOfOX7h4qXa5fuXqtes3Ghs3u0VWGsY7LJOZOU5owaXQvAMCJD/ODacqkfwoOXlW8aO33BQi04cwy3lf0ZEWqWAUfCtufCCJsmTIJVA3sLATuVjhPUwUhTGj0j5xmEiewhYmliTpNFYD0ZpVf+JiK/YiP6Vj5VoLPJCtw4GskFwiS4BPwSoqnXOPvWcSvxgA3sGnLsbEiNEYtuNGs73bnhdeF9FSNPe3P77La58/HcQbwXsyzFipuAYmaVH0onYOfUsNCCa5q5Oy4DllJ3TEe15qqnjRt/O1OXzPd4Y4zYz/NOB59/SEpaooZirxzmodxVlWNf/FeiWkj/pW6LwErtniorSUGDJcZYCHwnAGcuYFZUb4t2I2poYy8EnVieYTlilF9dCSrutFfTuPI0ltM3Julft9uj94sk6nf+m0os+535Phr3zrdc4Nhczct4SakaLeujz/ZxN6YfPnyj4S5Xx80dmw1kX3wW7k9Ruf41O0qBq6gzbRForQQ7SPXqID1EEM/QpuB3eDzfBL+DX8Hv5YWMNgOXMLrVT48zdRXA92</latexit><latexit sha1_base64="XVCJ1cGRruwHIcZtSVJOhzSPkJs=">AAADL3icfZFdixMxFIYz41etX129EbyJW4RdtywdbxRkYf0AvRFX2HYXmnbIpJk2bJIZMmdsa8gvUPDWX+C9v0a8EW+99ReYaYtut+KBmbyc5z0knDfJpSig3f4WhOfOX7h4qXa5fuXqtes3Ghs3u0VWGsY7LJOZOU5owaXQvAMCJD/ODacqkfwoOXlW8aO33BQi04cwy3lf0ZEWqWAUfCtufCCJsmTIJVA3sLATuVjhPUwUhTGj0j5xmEiewhYmliTpNFYD0ZpVf+JiK/YiP6Vj5VoLPJCtw4GskFwiS4BPwSoqnXOPvWcSvxgA3sGnLsbEiNEYtuNGs73bnhdeF9FSNPe3P77La58/HcQbwXsyzFipuAYmaVH0onYOfUsNCCa5q5Oy4DllJ3TEe15qqnjRt/O1OXzPd4Y4zYz/NOB59/SEpaooZirxzmodxVlWNf/FeiWkj/pW6LwErtniorSUGDJcZYCHwnAGcuYFZUb4t2I2poYy8EnVieYTlilF9dCSrutFfTuPI0ltM3Julft9uj94sk6nf+m0os+535Phr3zrdc4Nhczct4SakaLeujz/ZxN6YfPnyj4S5Xx80dmw1kX3wW7k9Ruf41O0qBq6gzbRForQQ7SPXqID1EEM/QpuB3eDzfBL+DX8Hv5YWMNgOXMLrVT48zdRXA92</latexit><latexit sha1_base64="ZD/s+6XiWYtYQcLgw5gtKz9Q6P4=">AAADL3icfZFdaxNBFIZn168aP5rqjeDN2CBUG8quNwpSqB+gN2KFpi1kkmV2MpsMnZldZs+ahGF+gf4Zf414I9566y9wNlm0acQDu/Nynvcww3nTQooSouhbEF66fOXqtY3rrRs3b93ebG/dOS7zyjDeY7nMzWlKSy6F5j0QIPlpYThVqeQn6dmrmp985KYUuT6CecEHio61yASj4FtJ+zNJlSUjLoG6oYXd2CUK72OiKEwYlfaFw0TyDHYwsSTNZokaiu68/hOXWLEf+ymdKNdd4qHsHg1ljWSDLAE+A6uodM49955p8mYIeBefuxgTI8YTeJS0O9FetCi8LuJGdFBTh8lW8ImMclYproFJWpb9OCpgYKkBwSR3LVKVvKDsjI5530tNFS8HdrE2hx/6zghnufGfBrzonp+wVJXlXKXeWa+jvMjq5r9Yv4Ls2cAKXVTANVtelFUSQ47rDPBIGM5Azr2gzAj/Vswm1FAGPqkW0XzKcqWoHlly7PrxwC7iSDPbiZ1b5X6f7g+ertPZXzqr6Wvu92T4O996X3BDITePLaFmrKi3Nuf/bEIvbf5c2UeqnI8vvhjWujh+shd7/SHqHLxsgtxA99E22kExeooO0Ft0iHqIoV/BveBBsB1+Cb+G38MfS2sYNDN30UqFP38DyCwMTg==</latexit>

Attacker Objective Cause targeted misclassification of

an auxiliary set of examples for the global model

and ensure global model has good

performance

3

Adversarial challenges

4


1. No access to other agents’ updates at time t: Adversary has no access to current updates from the other agents when attempting model poisoning

4



Approach: Generate malicious update with respect to wtG

<latexit sha1_base64="TOvSKS0/P7tUcN4qF29Oy7PQw1Y=">AAACq3icfVHbbhMxEHWWWwm3Fh55sYiQEBKRN0qgeasACV4QRZC0anaJZp3Z1KrtXdle0sjaL+CVV/gv/gZvEi7hNpI1R+ec0YxnslIK6xj72oouXLx0+crO1fa16zdu3trduz22RWU4jnghC3OcgUUpNI6ccBKPS4OgMolH2dmzRj/6gMaKQr9zyxJTBXMtcsHBBeokyfLF9MV77+rpbod1WW9/0O9R1u0N2DAeBjBg8fBxn8ZdtooO2cThdK/1MZkVvFKoHZdg7SRmpUs9GCe4xLqdVBZL4Gcwx0mAGhTa1K9Grun9wMxoXpjwtKMr9tcKD8rapcqCU4E7tb9rDfk3bVK5fD/1QpeVQ83XjfJKUlfQ5v90JgxyJ5cBADcizEr5KRjgLmypnWhc8EIp0DOfjOtJnPqk6ZHlvhPX9bYedlf/kBd/quc/1fNGfY5hTwZfBep1iQZcYR76BMxcQbBu8v9sQq9tIW/tI1PN+b7fiP4bjHvdOOA3/c7B080hd8hdco88IDF5Qg7IS3JIRoQTTT6Rz+RL9Ch6G51EydoatTY1d8hWRPgNLEfZcw==</latexit><latexit sha1_base64="TOvSKS0/P7tUcN4qF29Oy7PQw1Y=">AAACq3icfVHbbhMxEHWWWwm3Fh55sYiQEBKRN0qgeasACV4QRZC0anaJZp3Z1KrtXdle0sjaL+CVV/gv/gZvEi7hNpI1R+ec0YxnslIK6xj72oouXLx0+crO1fa16zdu3trduz22RWU4jnghC3OcgUUpNI6ccBKPS4OgMolH2dmzRj/6gMaKQr9zyxJTBXMtcsHBBeokyfLF9MV77+rpbod1WW9/0O9R1u0N2DAeBjBg8fBxn8ZdtooO2cThdK/1MZkVvFKoHZdg7SRmpUs9GCe4xLqdVBZL4Gcwx0mAGhTa1K9Grun9wMxoXpjwtKMr9tcKD8rapcqCU4E7tb9rDfk3bVK5fD/1QpeVQ83XjfJKUlfQ5v90JgxyJ5cBADcizEr5KRjgLmypnWhc8EIp0DOfjOtJnPqk6ZHlvhPX9bYedlf/kBd/quc/1fNGfY5hTwZfBep1iQZcYR76BMxcQbBu8v9sQq9tIW/tI1PN+b7fiP4bjHvdOOA3/c7B080hd8hdco88IDF5Qg7IS3JIRoQTTT6Rz+RL9Ch6G51EydoatTY1d8hWRPgNLEfZcw==</latexit><latexit sha1_base64="TOvSKS0/P7tUcN4qF29Oy7PQw1Y=">AAACq3icfVHbbhMxEHWWWwm3Fh55sYiQEBKRN0qgeasACV4QRZC0anaJZp3Z1KrtXdle0sjaL+CVV/gv/gZvEi7hNpI1R+ec0YxnslIK6xj72oouXLx0+crO1fa16zdu3trduz22RWU4jnghC3OcgUUpNI6ccBKPS4OgMolH2dmzRj/6gMaKQr9zyxJTBXMtcsHBBeokyfLF9MV77+rpbod1WW9/0O9R1u0N2DAeBjBg8fBxn8ZdtooO2cThdK/1MZkVvFKoHZdg7SRmpUs9GCe4xLqdVBZL4Gcwx0mAGhTa1K9Grun9wMxoXpjwtKMr9tcKD8rapcqCU4E7tb9rDfk3bVK5fD/1QpeVQ83XjfJKUlfQ5v90JgxyJ5cBADcizEr5KRjgLmypnWhc8EIp0DOfjOtJnPqk6ZHlvhPX9bYedlf/kBd/quc/1fNGfY5hTwZfBep1iQZcYR76BMxcQbBu8v9sQq9tIW/tI1PN+b7fiP4bjHvdOOA3/c7B080hd8hdco88IDF5Qg7IS3JIRoQTTT6Rz+RL9Ch6G51EydoatTY1d8hWRPgNLEfZcw==</latexit><latexit sha1_base64="TOvSKS0/P7tUcN4qF29Oy7PQw1Y=">AAACq3icfVHbbhMxEHWWWwm3Fh55sYiQEBKRN0qgeasACV4QRZC0anaJZp3Z1KrtXdle0sjaL+CVV/gv/gZvEi7hNpI1R+ec0YxnslIK6xj72oouXLx0+crO1fa16zdu3trduz22RWU4jnghC3OcgUUpNI6ccBKPS4OgMolH2dmzRj/6gMaKQr9zyxJTBXMtcsHBBeokyfLF9MV77+rpbod1WW9/0O9R1u0N2DAeBjBg8fBxn8ZdtooO2cThdK/1MZkVvFKoHZdg7SRmpUs9GCe4xLqdVBZL4Gcwx0mAGhTa1K9Grun9wMxoXpjwtKMr9tcKD8rapcqCU4E7tb9rDfk3bVK5fD/1QpeVQ83XjfJKUlfQ5v90JgxyJ5cBADcizEr5KRjgLmypnWhc8EIp0DOfjOtJnPqk6ZHlvhPX9bYedlf/kBd/quc/1fNGfY5hTwZfBep1iQZcYR76BMxcQbBu8v9sQq9tIW/tI1PN+b7fiP4bjHvdOOA3/c7B080hd8hdco88IDF5Qg7IS3JIRoQTTT6Rz+RL9Ch6G51EydoatTY1d8hWRPgNLEfZcw==</latexit>

4



2. Averaging with other agents: Updates from other agents could render malicious agent’s update ineffective



4






Approach: Boost malicious update to overcome effect of scaling

4




3. Randomness in choice of agents: Malicious agent is not chosen in every iteration if large number of agents




4








Approach: Boost malicious update to overcome effect of random selection

4





4. Avoid detection: Server may detect attack based on effect on accuracy on validation data or weight update statistics





4





4. Avoid detection: Server may detect attack based on effect on accuracy on validation data or weight update statistics




Approach: Improve on baseline by adding benign training and distance constraints


4

Related Work

✦ Byzantine Attacks and Defenses

✦ Attacks focus on convergence to ineffective models (poor performance on test data)

✦ Defenses such as Krum (Blanchard et al., 2017) and coordinate-wise median (Yin et al.) rely on robust aggregation mechanisms to avoid including arbitrary updates

✦ Bagdasaryan et al. (2018) perform model replacement to insert backdoors without affecting convergence, but are mostly effective when training has converged

5

Experimental setup

✦ Fashion MNIST data [2]

✦ CNN achieving 91.5% accuracy on test data

✦ Total of 10 agents, all called every time step

✦ Training is stopped when global model achieves above 91% validation accuracy

✦ Adversarial objective: Classify (‘sandal’, class 5) as a ‘sneaker’, class 7

6

Strategy Malicious agent’s update computationBoosting malicious update,

no local training

Targeted Model Poisoning

�mal = argmin�Cross-entropy({xlm, T l

m}nmall=1 ;wG + �)

<latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit><latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit><latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit><latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit>

�mal ! ��mal<latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit><latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit><latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit><latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit>

7

Strategy Malicious agent’s update computationBoosting malicious update,

no local training

Targeted Model Poisoning

�mal = argmin�Cross-entropy({xlm, T l

m}nmall=1 ;wG + �)

<latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit><latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit><latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit><latexit sha1_base64="J/M0Ent6MdJ4NKeZ70D9owiBcn4=">AAACyHicdVFtb9MwEHYyXsZ46+AjQlhUwBAQOVU7VqFJkzYJxKchrdukposcx+ms2U5kO3SR5S/8A34ev4SvOG2BdYKTbD2+585391xWcaYNQj+CcO3GzVu31+9s3L13/8HDzuajY13WitARKXmpTjOsKWeSjgwznJ5WimKRcXqSXey3/MlXqjQr5ZFpKjoReCpZwQg23pV2vieZsElOucEutYmhl8YKzJ2Du3DxwmoqmGzJv5Fuye2rUut3VBpVVo3bSmwisDnPCnvpUnHG3x61d+Jz+W7szqxcqeA+JFkxSz/CN/DK16/TThdFqLcz6PcginoDNIyHHgxQPNzuwzhCc+uCpR2mm8GzJC9JLXwfhGOtxzGqzMQ3bhjh1G0ktaYVJhd4SsceSiyonti5dg6+8J4cFqXyRxo4917NsFho3YjMR7az6etc6/wXN65NsTOxTFa1oZIsChU1h6aE7SJgzhQlhjceYKKY7xWSc6wwMX5dK1Uy4WeQdEZKIbDMbSub+yP1zF1jBTlYsgRze+BpL+lv3eD/wXEvilEUf+l399BS3HXwBDwHWyAG78Ee+AQOwQgQ8DN4GrwMXoWfwyqchc0iNAyWOY/BioXffgE4TuQI</latexit>

�mal ! ��mal<latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit><latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit><latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit><latexit sha1_base64="SVLAfWw+ZsQa7W5GOoVDFFtpN2Y=">AAACenicdVBNaxsxFJS3X2n65bTHHipiCi2FRWvsJr4FkkOPKdRJwGvMk1Zri0jaRXpbx4j9Qf01vTb/pYfKjgtNaAYEw8x7jN7wWiuPjF13kgcPHz1+svN099nzFy9fdfden/mqcUKORaUrd8HBS62sHKNCLS9qJ8FwLc/55fHaP/8unVeV/YarWk4NzK0qlQCM0qx7nHMT8kJqhHYWcpRXGAzots2dmi8QnKuWNOcSgd43SWfdHktZ/3A46FOW9odslI0iGbJs9HlAs5Rt0CNbnM72Ou/yohKNkRaFBu8nGatxGsChElq2u3njZQ3iEuZyEqkFI/00bK5t6fuoFLSsXHwW6Ub9dyOA8X5leJw0gAt/11uL//MmDZaH06Bs3aC04iaobDTFiq6ro4VyUqBeRQLCqfhXKhbgQGAs+FYKN/EGK5eiMgZsEXJeLtuQryN5GZbtHdeIk60rQIeTaMdK//ZG7ydn/TRjafZ10Dti23J3yFuyTz6QjByQI/KFnJIxEeQH+Ul+kevO72Q/+Zh8uhlNOtudN+QWksEfQjjGuQ==</latexit>

- Adam for 5 epochs - Boosting by 10

7

Targeted Model Poisoning: Results

Takeaways 1. Targeted backdoor inserted with high confidence 2. Accuracy on validation data does not suffer for global model

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Validation Accuracy GlobalMalicious objective confidence (5!7) Global

(a) Confidence on malicious objective (poisoning) and the accuracyon validation data (targeted) on the global model

(b) Comparison of weight update distributions for be-nign and malicious agents

Figure 1: Targeted model poisoning attack for CNN on Fashion MNIST data. The total number of agentsis K = 10, including the malicious agents. All agents train their local models for 5 epochs with the appro-priate objective.

3.1 Adversarial optimization setupFrom Eq. 1, two challenges for the adversary are immediately clear. First, the objective represents a dif-ficult combinatorial optimization problem so we relax Eq. 1 in terms of the cross-entropy loss for whichautomatic di↵erentiation can be used. Second, the adversary does not have access to the global parametervector wt

G for the current iteration and can only influence it though the weight update �tm it provides tothe server S . So, it performs the optimization over wt

G, which is an estimate of the value of wtG based on all

the information I tm available to the adversary. The objective function for the adversary to achieve targetedmodel poisoning on the tth iteration is

argmin�tm

L({xi ,⌧i }ri=1,wtG),

s.t. wtG = g(I tm),

(4)

where g(·) is an estimator. For the rest of this section, we use the estimate wtG =wt�1

G +↵m�tm, implying thatthe malicious agent ignores the updates from the other agents but accounts for scaling at aggregation. Thisassumption is enough to ensure the attack works in practice.

3.2 Targeted model poisoning for standard federated learning

The adversary can directly optimize the adversarial objective L({xi ,⌧i }ri=1,wtG) with wt

G =wt�1G +↵m�tm. How-

ever, this setup implies that the optimizer has to account for the scaling factor ↵m implicitly. In practice, wefind that when using a gradient-based optimizer such as SGD, explicit boosting is much more e↵ective. Therest of the section focuses on explicit boosting and an analysis of implicit boosting is deferred to Section Aof the Appendix.Explicit Boosting: Mimicking a benign agent, the malicious agent can run Em steps of a gradient-based op-timizer starting from wt�1

G to obtain wtm which minimizes the loss over {xi ,⌧i }ri=1. The malicious agent then

obtains an initial update �tm = wtm �wt�1

G . However, since the malicious agent’s update tries to ensure thatthe model learns labels di↵erent from the true labels for the data of its choice (Daux), it has to overcome thee↵ect of scaling, which would otherwise mostly nullify the desired classification outcomes. This happensbecause the learning objective for all the other agents is very di↵erent from that of the malicious agent, es-pecially in the i.i.d. case. The final weight update sent back by the malicious agent is then �tm = ��tm, where� is the factor by which the malicious agent boosts the initial update. Note that with wt

G = wt�1G + ↵m�tm

and � = 1↵m

, then wtG =wt

m, implying that if the estimation was exact, the global weight vector should nowsatisfy the malicious agent’s objective.Results: In the attack with explicit boosting, the malicious agent runs Em = 5 steps of the Adam optimizer[12] to obtain �tm, and then boosts it by 1

↵m= K . The results for the case with K = 10 for the Fashion MNIST

5

8

Targeted Model Poisoning: Weight update distribution

Takeaways 1. Fewer weights modified for malicious update 2. Malicious update could be ‘hidden’ inside a benign one

9

Detectability/stealth of the attack?

✦ Can the weight update distribution of the malicious agent be used to discriminate it from the benign agents?

✦ Can the validation accuracy of the malicious model in isolation be used to detect the attack?

10

Strategy Malicious agent’s update computationJoint minimization of

benign and malicious objectives, with distance

constraints

Stealthy Model Poisoning

Benign Objective Malicious Objective Distance Constraint

�mal = argmin�

L�{xi

m, yim}nmi=1;wG + �

�+ �L

�{xl, T l}nmal

l=1 ;wG + ��+ ⇢k� � �consk22

<latexit sha1_base64="2NkVr+zB9hbKe1ZjUdHZqjGafFg=">AAADpXicfVJdb9MwFE1WPkb52uCRF4sKMWCrktDCKjRpAiR42MSQ2m5S3UaO67TWbCeyHdbK8y9A4hX+Gv8Gpy2shY0rJb4659xj+14nOaNKB8FPf61y7fqNm+u3qrfv3L13f2PzQVdlhcSkgzOWyZMEKcKoIB1NNSMnuSSIJ4wcJ6fvSv74C5GKZqKtpznpczQSNKUYaQfFm74PE27gkDCNbGygJhNtOGLWgj0AkRxxKhx8obHgADKS6i0AHZxOYj6g29PyD1093QvtwIiY2zeOPIs/gBdgqRhASUdj/WyGEo3AAZibzb0GbLs9YKUPW/gsH+gKy2VPOc4APF+hd8AlF8SZUO4m8DyOBlG8UQvqQbTbbEQgqEfNoBW2XNIMwtarBgjrwSxq3iKOXNO+wmGGC06Exgwp1QuDXPcNkppiRmwVForkCJ+iEem5VCBOVN/MhmXBE4cMQZpJ9wkNZuhyhUFcqSlPnJIjPVZ/cyV4GdcrdLrbN1TkhSYCzzdKCwZ0BsrJgyGVBGs2dQnCkrqzAjxGEmHt3kcVCnKGM86RGBrYtb2wb2C5R5KaWmjtKu/mYP/QZ/+ykwt2UrLvieuTJIcO+pQTiXQmn5vZ60JOulj/J6NiLnPrSj8Sbt34fs8IXJ10o3r4sh59btT23y4Gue498h57W17ovfb2vY/ekdfxsD/2v/nf/R+Vp5XDSrvSnUvX/EXNQ28lKvEvJMg05g==</latexit>

11

Strategy Malicious agent’s update computationJoint minimization of


constraints

Stealthy Model Poisoning

Benign Objective Malicious Objective Distance Constraint

Experiment settings - Boosting by 10 ( ) - - Adam for 10 epochs - Cross-entropy loss - Constrain w.r.t. previous

cumulative update from other agents

� = 10<latexit sha1_base64="h1yJ0xDNnnoGJzAMiU5spdEXuvg=">AAACq3icfVHbbhMxEHWWWwm3lj7yYhEhISSidYuAPiBVhQdeEEWQtGp2VY2d2dSqLyvbSxqt9gt47Wv5L/4GbxIuocBI1hydc0YznuGlkj6k6bdOcuXqtes31m52b92+c/fe+sb9obeVEzgQVll3yMGjkgYHQQaFh6VD0FzhAT993eoHn9F5ac2nMCsx1zAxspACQqSOMo4B6CvK0uP1XtrfSdnOc0YvA9ZP59Ejy9g/3uh8ycZWVBpNEAq8H7G0DHkNLkihsOlmlccSxClMcBShAY0+r+cjN/RRZMa0sC4+E+ic/b2iBu39TPPo1BBO/J9aS/5NG1WheJnX0pRVQCMWjYpK0WBp+386lg5FULMIQDgZZ6XiBByIELfUzQxOhdUazLjOhs2I5XXW9uBF3WNNs6rzYtr8lKeX1bNf6lmrvsG4J4fvIvW+RAfBuid1Bm6iIVqX+X82aRa2mFf2wXUTz/fjRvTfYLjVZ9v9rQ/Pert7y0OukQfkIXlMGHlBdslbsk8GRBBDzskF+Zo8TT4mR0m2sCadZc0mWYkEvwOD7thI</latexit>

⇢ = 1e� 4<latexit sha1_base64="lpLSxcn6fKqS/RfueLmRpfSZtnM=">AAACrHicfVHLahRBFK1pH4njK9Glm8JBEMGmK4ZoFkKILtyIETKTQHcTbtfcnilSj6aq2snQ9Bdk7Va/y7+xemZ8jFEvFHU451zu5dyiksL5JPnWi65dv3FzY/NW//adu/fub20/GDlTW45DbqSxpwU4lELj0Asv8bSyCKqQeFKcv+n0k09onTD62M8rzBVMtCgFBx+oNLNTQ19Ths93z7YGSbyfsP09Rq8CFieLGpBVHZ1t9y6zseG1Qu25BOdSllQ+b8B6wSW2/ax2WAE/hwmmAWpQ6PJmsXNLnwRmTEtjw9OeLtjfOxpQzs1VEZwK/NT9qXXk37S09uWrvBG6qj1qvhxU1pJ6Q7sA6FhY5F7OAwBuRdiV8ilY4D7E1M80zrhRCvS4yUZtyvIm62YUZTNgbbuuF+Ws/SnPrqoXv9SLTn2LISeL7wP1oUIL3thnTQZ2oiBYV///bEIvbeFfy6NQbTjfjxvRf4PRTsxexDsfdwcHh6tDbpJH5DF5Shh5SQ7IO3JEhoQTQz6TL+RrFEfHURrlS2vUW/U8JGsVld8BQnHYlQ==</latexit>

�mal = argmin�

L�{xi

m, yim}nmi=1;wG + �

�+ �L

�{xl, T l}nmal

l=1 ;wG + ��+ ⇢k� � �consk22

<latexit sha1_base64="2NkVr+zB9hbKe1ZjUdHZqjGafFg=">AAADpXicfVJdb9MwFE1WPkb52uCRF4sKMWCrktDCKjRpAiR42MSQ2m5S3UaO67TWbCeyHdbK8y9A4hX+Gv8Gpy2shY0rJb4659xj+14nOaNKB8FPf61y7fqNm+u3qrfv3L13f2PzQVdlhcSkgzOWyZMEKcKoIB1NNSMnuSSIJ4wcJ6fvSv74C5GKZqKtpznpczQSNKUYaQfFm74PE27gkDCNbGygJhNtOGLWgj0AkRxxKhx8obHgADKS6i0AHZxOYj6g29PyD1093QvtwIiY2zeOPIs/gBdgqRhASUdj/WyGEo3AAZibzb0GbLs9YKUPW/gsH+gKy2VPOc4APF+hd8AlF8SZUO4m8DyOBlG8UQvqQbTbbEQgqEfNoBW2XNIMwtarBgjrwSxq3iKOXNO+wmGGC06Exgwp1QuDXPcNkppiRmwVForkCJ+iEem5VCBOVN/MhmXBE4cMQZpJ9wkNZuhyhUFcqSlPnJIjPVZ/cyV4GdcrdLrbN1TkhSYCzzdKCwZ0BsrJgyGVBGs2dQnCkrqzAjxGEmHt3kcVCnKGM86RGBrYtb2wb2C5R5KaWmjtKu/mYP/QZ/+ykwt2UrLvieuTJIcO+pQTiXQmn5vZ60JOulj/J6NiLnPrSj8Sbt34fs8IXJ10o3r4sh59btT23y4Gue498h57W17ovfb2vY/ekdfxsD/2v/nf/R+Vp5XDSrvSnUvX/EXNQ28lKvEvJMg05g==</latexit>

11

Stealthy Model Poisoning: Results and Weight update

12


0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12 14 16

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Val. Acc. GlobalConf. (5!7) Global

Val. Acc. Mal. (stealthy poison)Val. Acc. Mal. (targeted poison)

(a) Confidence on malicious objective and accuracy on valida-tion data for wt

G . Stealth with respect to accuracy checking isalso shown for both the stealthy and targeted model poisoningattacks.

(b) Comparison of weight update distributionsfor benign and malicious agents

Figure 2: Stealthy model poisoning for CNN on Fashion MNIST. We use � = 10 and ⇢ = 1e�4 for themalicious agent’s objective.

data are shown in Figure 3.1. The attack is clearly successful at causing the global model to classify thechosen example in the target class. In fact, after t = 3, the global model is highly confident in its (incorrect)prediction. Further, the global model converges with good performance on the validation set in spite of thetargeted poisoning for 1 example. Results for the Adult Census dataset (Section B.1) demonstrate targetedmodel poisoning is possible across datasets and models. Thus, the explicit boosting attack is able to achievetargeted poisoning in the federated learning setting.Performance on stealth metrics: While the targeted model poisoning attack using explicit boosting doesnot take stealth metrics into account, it is instructive to study properties of the model update it generates.Compared to the weight update from a benign agent, the update from the malicious agent is much sparserand has a smaller range (Figure 1b). In Figure 4, the spread of L2 distances between all benign updatesand between the malicious update and the benign updates is plotted. For targeted model poisoning, boththe minimum and maximum distance away from any of the benign updates keeps decreasing over timesteps, while it remains relatively constant for the other agents. In Figure 2a the accuracy of the maliciousmodel on the validation data (Val. Acc. Mal (targeted poison)) is shown, which is much lower than the globalmodel’s accuracy. Thus, both accuracy checking and weight update statistics based detection is possible forthe targeted model poisoning attack.

3.3 Stealthy model poisoningAs discussed in Section 2.3, there are two properties which the server can use to detect anomalous updates:accuracy on validation data and weight update statistics. In order to maintain stealth with respect to bothof these properties, the adversary can add loss terms corresponding to both of those metrics to the modelpoisoning objective function from Eq. 4 and improve targeted model poisoning. First, in order to improvethe accuracy on validation data, the adversary adds the training loss over the malicious agent’s local datashard Dm (L(Dm,wt

G)) to the objective. Since the training data is i.i.d. with the validation data, this willensure that the malicious agent’s update is similar to that of a benign agent in terms of validation loss andwill make it challenging for the server to flag the malicious update as anomalous.

Second, the adversary needs to ensure that its update is as close as possible to the benign agents’ updatesin the appropriate distance metric. For our experiments, we use the `p normwith p = 2. Since the adversarydoes not have access to the updates for the current time step t that are generated by the other agents, itconstrains �tm with respect to �t�1ben =

Pi2[k]\m↵i�t�1i , which is the average update from all the other agents

for the previous iteration, which the malicious agent has access to. Thus, the adversary adds ⇢k�tm � �t�1benk2to its objective as well. We note that the addition of the training loss term is not su�cient to ensure thatthe malicious weight update is close to that of the benign agents since there could be multiple local minima

6

Takeaways 1. Malicious objective is met 2. Improved validation accuracy compared

to Targeted Model Poisoning

12


0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12 14 16

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time


Val. Acc. Mal. (stealthy poison)Val. Acc. Mal. (targeted poison)


G . Stealth with respect to accuracy checking isalso shown for both the stealthy and targeted model poisoningattacks.


Figure 2: Stealthy model poisoning for CNN on Fashion MNIST. We use � = 10 and ⇢ = 1e�4 for themalicious agent’s objective.

data are shown in Figure 3.1. The attack is clearly successful at causing the global model to classify thechosen example in the target class. In fact, after t = 3, the global model is highly confident in its (incorrect)prediction. Further, the global model converges with good performance on the validation set in spite of thetargeted poisoning for 1 example. Results for the Adult Census dataset (Section B.1) demonstrate targetedmodel poisoning is possible across datasets and models. Thus, the explicit boosting attack is able to achievetargeted poisoning in the federated learning setting.Performance on stealth metrics: While the targeted model poisoning attack using explicit boosting doesnot take stealth metrics into account, it is instructive to study properties of the model update it generates.Compared to the weight update from a benign agent, the update from the malicious agent is much sparserand has a smaller range (Figure 1b). In Figure 4, the spread of L2 distances between all benign updatesand between the malicious update and the benign updates is plotted. For targeted model poisoning, boththe minimum and maximum distance away from any of the benign updates keeps decreasing over timesteps, while it remains relatively constant for the other agents. In Figure 2a the accuracy of the maliciousmodel on the validation data (Val. Acc. Mal (targeted poison)) is shown, which is much lower than the globalmodel’s accuracy. Thus, both accuracy checking and weight update statistics based detection is possible forthe targeted model poisoning attack.

3.3 Stealthy model poisoningAs discussed in Section 2.3, there are two properties which the server can use to detect anomalous updates:accuracy on validation data and weight update statistics. In order to maintain stealth with respect to bothof these properties, the adversary can add loss terms corresponding to both of those metrics to the modelpoisoning objective function from Eq. 4 and improve targeted model poisoning. First, in order to improvethe accuracy on validation data, the adversary adds the training loss over the malicious agent’s local datashard Dm (L(Dm,wt

G)) to the objective. Since the training data is i.i.d. with the validation data, this willensure that the malicious agent’s update is similar to that of a benign agent in terms of validation loss andwill make it challenging for the server to flag the malicious update as anomalous.

Second, the adversary needs to ensure that its update is as close as possible to the benign agents’ updatesin the appropriate distance metric. For our experiments, we use the `p normwith p = 2. Since the adversarydoes not have access to the updates for the current time step t that are generated by the other agents, itconstrains �tm with respect to �t�1ben =

Pi2[k]\m↵i�t�1i , which is the average update from all the other agents

for the previous iteration, which the malicious agent has access to. Thus, the adversary adds ⇢k�tm � �t�1benk2to its objective as well. We note that the addition of the training loss term is not su�cient to ensure thatthe malicious weight update is close to that of the benign agents since there could be multiple local minima

6

Takeaways 1. Malicious objective is met 2. Improved validation accuracy compared

to Targeted Model Poisoning

Takeaway Closer match between weight updates for benign and malicious agents

12

Strategy Malicious agent’s update computation Alternating minimization of


constraints

Alternating minimization

Repeat:�0mal = argmin�Cross-entropy({xl

m, T lm}nmal

l=1 ;wG + �)<latexit sha1_base64="+1V7VtyagHHXve5ilOYiCbnVH9U=">AAACyXicdVFtb9MwEHbC2xhvHXxEAosKrQiInKqFVWjSpE0CiS9DWrdJTRc5rtNZs51gO7TF8if+Af+Of8JHnDbAOsFJth7fc+e7ey4rOdMGoR9BeO36jZu3Nm5v3rl77/6D1tbDY11UitAhKXihTjOsKWeSDg0znJ6WimKRcXqSXezX/MkXqjQr5JFZlHQs8FSynBFsvCttfU8yYZMJ5Qa77dQmhs6NFZg7B3fh6oXVVDDpPPk31DXcviq0fk2lUUW5cJ3EJgKb8yy3c5eKM/7qqL4Tn8t3Y3dm5VoF9y7J8ln6Hr6El75+kbbaKELdnX6vC1HU7aNBPPCgj+LBmx6MI7S0NmjsMN0KniaTglTC90E41noUo9KMfeOGEU7dZlJpWmJygad05KHEguqxXYrn4HPvmcC8UP5IA5feyxkWC60XIvOR9Wz6Klc7/8WNKpPvjC2TZWWoJKtCecWhKWC9CThhihLDFx5gopjvFZJzrDAxfl9rVTLhZ5B0RgohsJzYWjb3R+qZu8IKctCwBHN74Gkv6W/d4P/BcTeKURR/6rX3UCPuBngMnoEOiMFbsAc+gEMwBAT8DJ4E20En/Bh+Dufh11VoGDQ5j8Cahd9+Ab6w5Dk=</latexit><latexit sha1_base64="+1V7VtyagHHXve5ilOYiCbnVH9U=">AAACyXicdVFtb9MwEHbC2xhvHXxEAosKrQiInKqFVWjSpE0CiS9DWrdJTRc5rtNZs51gO7TF8if+Af+Of8JHnDbAOsFJth7fc+e7ey4rOdMGoR9BeO36jZu3Nm5v3rl77/6D1tbDY11UitAhKXihTjOsKWeSDg0znJ6WimKRcXqSXezX/MkXqjQr5JFZlHQs8FSynBFsvCttfU8yYZMJ5Qa77dQmhs6NFZg7B3fh6oXVVDDpPPk31DXcviq0fk2lUUW5cJ3EJgKb8yy3c5eKM/7qqL4Tn8t3Y3dm5VoF9y7J8ln6Hr6El75+kbbaKELdnX6vC1HU7aNBPPCgj+LBmx6MI7S0NmjsMN0KniaTglTC90E41noUo9KMfeOGEU7dZlJpWmJygad05KHEguqxXYrn4HPvmcC8UP5IA5feyxkWC60XIvOR9Wz6Klc7/8WNKpPvjC2TZWWoJKtCecWhKWC9CThhihLDFx5gopjvFZJzrDAxfl9rVTLhZ5B0RgohsJzYWjb3R+qZu8IKctCwBHN74Gkv6W/d4P/BcTeKURR/6rX3UCPuBngMnoEOiMFbsAc+gEMwBAT8DJ4E20En/Bh+Dufh11VoGDQ5j8Cahd9+Ab6w5Dk=</latexit><latexit sha1_base64="+1V7VtyagHHXve5ilOYiCbnVH9U=">AAACyXicdVFtb9MwEHbC2xhvHXxEAosKrQiInKqFVWjSpE0CiS9DWrdJTRc5rtNZs51gO7TF8if+Af+Of8JHnDbAOsFJth7fc+e7ey4rOdMGoR9BeO36jZu3Nm5v3rl77/6D1tbDY11UitAhKXihTjOsKWeSDg0znJ6WimKRcXqSXezX/MkXqjQr5JFZlHQs8FSynBFsvCttfU8yYZMJ5Qa77dQmhs6NFZg7B3fh6oXVVDDpPPk31DXcviq0fk2lUUW5cJ3EJgKb8yy3c5eKM/7qqL4Tn8t3Y3dm5VoF9y7J8ln6Hr6El75+kbbaKELdnX6vC1HU7aNBPPCgj+LBmx6MI7S0NmjsMN0KniaTglTC90E41noUo9KMfeOGEU7dZlJpWmJygad05KHEguqxXYrn4HPvmcC8UP5IA5feyxkWC60XIvOR9Wz6Klc7/8WNKpPvjC2TZWWoJKtCecWhKWC9CThhihLDFx5gopjvFZJzrDAxfl9rVTLhZ5B0RgohsJzYWjb3R+qZu8IKctCwBHN74Gkv6W/d4P/BcTeKURR/6rX3UCPuBngMnoEOiMFbsAc+gEMwBAT8DJ4E20En/Bh+Dufh11VoGDQ5j8Cahd9+Ab6w5Dk=</latexit><latexit sha1_base64="+1V7VtyagHHXve5ilOYiCbnVH9U=">AAACyXicdVFtb9MwEHbC2xhvHXxEAosKrQiInKqFVWjSpE0CiS9DWrdJTRc5rtNZs51gO7TF8if+Af+Of8JHnDbAOsFJth7fc+e7ey4rOdMGoR9BeO36jZu3Nm5v3rl77/6D1tbDY11UitAhKXihTjOsKWeSDg0znJ6WimKRcXqSXezX/MkXqjQr5JFZlHQs8FSynBFsvCttfU8yYZMJ5Qa77dQmhs6NFZg7B3fh6oXVVDDpPPk31DXcviq0fk2lUUW5cJ3EJgKb8yy3c5eKM/7qqL4Tn8t3Y3dm5VoF9y7J8ln6Hr6El75+kbbaKELdnX6vC1HU7aNBPPCgj+LBmx6MI7S0NmjsMN0KniaTglTC90E41noUo9KMfeOGEU7dZlJpWmJygad05KHEguqxXYrn4HPvmcC8UP5IA5feyxkWC60XIvOR9Wz6Klc7/8WNKpPvjC2TZWWoJKtCecWhKWC9CThhihLDFx5gopjvFZJzrDAxfl9rVTLhZ5B0RgohsJzYWjb3R+qZu8IKctCwBHN74Gkv6W/d4P/BcTeKURR/6rX3UCPuBngMnoEOiMFbsAc+gEMwBAT8DJ4E20En/Bh+Dufh11VoGDQ5j8Cahd9+Ab6w5Dk=</latexit>

�0mal ! ��0mal<latexit sha1_base64="zVk37AnjomAR2N02F+4MmPeucUs=">AAACfHicdVBdixMxFE3Hr3X96uqjDwarKIhDprTr9m3BFXxcwe4udEq5yWTasElmSO5YS5hf5K/xUf0vYtqt4C7ugcDhnHs5uYfXWnlk7EcnuXHz1u07O3d3791/8PBRd+/xia8aJ+RYVLpyZxy81MrKMSrU8qx2EgzX8pSfv1/7p1+k86qyn3FVy6mBuVWlEoBRmnU/5NyEvJAaoX01CznKrxgM6LaluVPzBYJz1ZLmXCLQa2dn3R5LWf9gOOhTlvaHbJSNIhmybLQ/oFnKNuiRLY5ne51neVGJxkiLQoP3k4zVOA3gUAkt29288bIGcQ5zOYnUgpF+Gjb3tvRlVApaVi4+i3Sj/rsRwHi/MjxOGsCFv+qtxf95kwbLg2lQtm5QWnERVDaaYkXX5dFCOSlQryIB4VT8KxULcCAwVnwphZt4g5VLURkDtgg5L5dtyNeRvAzL9oprxNHWFaDDUbRjpX97o9eTk36asTT7NOgdsm25O+QpeU5ek4y8I4fkIzkmYyLIN/Kd/CS/Or+TF8mb5O3FaNLZ7jwhl5Ds/wE1C8cb</latexit><latexit sha1_base64="zVk37AnjomAR2N02F+4MmPeucUs=">AAACfHicdVBdixMxFE3Hr3X96uqjDwarKIhDprTr9m3BFXxcwe4udEq5yWTasElmSO5YS5hf5K/xUf0vYtqt4C7ugcDhnHs5uYfXWnlk7EcnuXHz1u07O3d3791/8PBRd+/xia8aJ+RYVLpyZxy81MrKMSrU8qx2EgzX8pSfv1/7p1+k86qyn3FVy6mBuVWlEoBRmnU/5NyEvJAaoX01CznKrxgM6LaluVPzBYJz1ZLmXCLQa2dn3R5LWf9gOOhTlvaHbJSNIhmybLQ/oFnKNuiRLY5ne51neVGJxkiLQoP3k4zVOA3gUAkt29288bIGcQ5zOYnUgpF+Gjb3tvRlVApaVi4+i3Sj/rsRwHi/MjxOGsCFv+qtxf95kwbLg2lQtm5QWnERVDaaYkXX5dFCOSlQryIB4VT8KxULcCAwVnwphZt4g5VLURkDtgg5L5dtyNeRvAzL9oprxNHWFaDDUbRjpX97o9eTk36asTT7NOgdsm25O+QpeU5ek4y8I4fkIzkmYyLIN/Kd/CS/Or+TF8mb5O3FaNLZ7jwhl5Ds/wE1C8cb</latexit><latexit sha1_base64="zVk37AnjomAR2N02F+4MmPeucUs=">AAACfHicdVBdixMxFE3Hr3X96uqjDwarKIhDprTr9m3BFXxcwe4udEq5yWTasElmSO5YS5hf5K/xUf0vYtqt4C7ugcDhnHs5uYfXWnlk7EcnuXHz1u07O3d3791/8PBRd+/xia8aJ+RYVLpyZxy81MrKMSrU8qx2EgzX8pSfv1/7p1+k86qyn3FVy6mBuVWlEoBRmnU/5NyEvJAaoX01CznKrxgM6LaluVPzBYJz1ZLmXCLQa2dn3R5LWf9gOOhTlvaHbJSNIhmybLQ/oFnKNuiRLY5ne51neVGJxkiLQoP3k4zVOA3gUAkt29288bIGcQ5zOYnUgpF+Gjb3tvRlVApaVi4+i3Sj/rsRwHi/MjxOGsCFv+qtxf95kwbLg2lQtm5QWnERVDaaYkXX5dFCOSlQryIB4VT8KxULcCAwVnwphZt4g5VLURkDtgg5L5dtyNeRvAzL9oprxNHWFaDDUbRjpX97o9eTk36asTT7NOgdsm25O+QpeU5ek4y8I4fkIzkmYyLIN/Kd/CS/Or+TF8mb5O3FaNLZ7jwhl5Ds/wE1C8cb</latexit><latexit sha1_base64="zVk37AnjomAR2N02F+4MmPeucUs=">AAACfHicdVBdixMxFE3Hr3X96uqjDwarKIhDprTr9m3BFXxcwe4udEq5yWTasElmSO5YS5hf5K/xUf0vYtqt4C7ugcDhnHs5uYfXWnlk7EcnuXHz1u07O3d3791/8PBRd+/xia8aJ+RYVLpyZxy81MrKMSrU8qx2EgzX8pSfv1/7p1+k86qyn3FVy6mBuVWlEoBRmnU/5NyEvJAaoX01CznKrxgM6LaluVPzBYJz1ZLmXCLQa2dn3R5LWf9gOOhTlvaHbJSNIhmybLQ/oFnKNuiRLY5ne51neVGJxkiLQoP3k4zVOA3gUAkt29288bIGcQ5zOYnUgpF+Gjb3tvRlVApaVi4+i3Sj/rsRwHi/MjxOGsCFv+qtxf95kwbLg2lQtm5QWnERVDaaYkXX5dFCOSlQryIB4VT8KxULcCAwVnwphZt4g5VLURkDtgg5L5dtyNeRvAzL9oprxNHWFaDDUbRjpX97o9eTk36asTT7NOgdsm25O+QpeU5ek4y8I4fkIzkmYyLIN/Kd/CS/Or+TF8mb5O3FaNLZ7jwhl5Ds/wE1C8cb</latexit>

�00mal = argmin� Cross-entropy�{xi

m, yim}ni=1;wG + ��0mal + ��+⇢k��consk22

<latexit sha1_base64="bpGqatx5D7OXRuiowl4CsCBvZi8=">AAADfnicfVJbb9MwGHVWLqPcNnjkxaKCbbCVpNpgFZpUGBK8IIZEu0l1GxzXaa3FdrBd2uL5F/DA70P8GZx2Ky23T3J8dM6xP8fHSZ4xbcLwe7BSunT5ytXVa+XrN27eur22fqel5VAR2iQyk+okwZpmTNCmYSajJ7mimCcZPU5ODwv9+DNVmknxwUxy2uG4L1jKCDaeitd+oIRb1KOZwW5jI0aGjo3lOHPwACKs+pyJ2C543MxxqKTWO1QYJfOJgyijqdlE3piOY95l25Pii1xs2UHkula4514axa/hY4gSajBcbOsbzNu6qWOuQaRYf2C2ClYNJERnC+LOAr7Yg0ih/SHP4lq3Fq9VwmpY29/brcGwWtsL61Hdg70wqj/dhVE1nFalsfXty0cAwFG8HnxFPUmG3P8YybDW7SjMTcdiZRjJqCujoaY5Jqe4T9seCsyp7thpCg4+8EwPplL5IQycsosrLOZaT3jinRybgf5dK8i/ae2hSfc7lol8aKggs0bpMINGwiJS2GOKEpNNPMBEMX9WSAZYYWJ88GUk6IhIzrHoWdRy7ahjUdEjSW0lcm5Z9yG5uTz6Ux3/UseF+or6e1L0rafe5VRhI9UjO3032FvP5//ZmJjZ/Lx0Hwl3Pr6LjOC/QatWjTx+H1UaL8GsVsE9cB9sggg8Aw3wBhyBJiDBi6Af5MGnEig9LO2UnsysK8H5mrtgqUr7PwG5wCqW</latexit><latexit sha1_base64="o3NR/rwYsUwyoLX4pYQ6qUzsbFs=">AAADfnicfVLdbtMwGHUWfsb42+CSG4sKtgErSbXBJjRpMCS4QQyJdpPqNnJcp7UW28F2WIvnJ+CCW56Ad0K8DE67lZa/T3J8dM6xP8fHaZEzbaLoe7AQXrh46fLilaWr167fuLm8cqulZakIbRKZS3WUYk1zJmjTMJPTo0JRzNOcHqbH+5V++JEqzaR4b0YF7XDcFyxjBBtPJcs/UMot6tHcYLe6miBDh8ZynDu4CxFWfc5EYmc8buLYV1LrDSqMksXIQZTTzKwhb8yGCe+yR6Pqi1xi2W7sula4Z146SV7BhxCl1GA429Y3mLZ1Y8dUg0ix/sCsV6waSIhOZ8SNGXy+B5FC+0OeJo1uI1muRfWosb212YBRvbEV7cQ7HmxF8c6TTRjXo3HV9ta/fCoWv309SFaCz6gnScn9j5Eca92Oo8J0LFaGkZy6JVRqWmByjPu07aHAnOqOHafg4D3P9GAmlR/CwDE7u8JirvWIp97JsRno37WK/JvWLk223bFMFKWhgkwaZWUOjYRVpLDHFCUmH3mAiWL+rJAMsMLE+OCXkKAnRHKORc+ilmvHHYuqHmlma7Fz87oPyU3lkz/V4S91WKkvqb8nRd946m1BFTZSPbDjd4O99Wz+n42Jic3Pc/eRcufjO88I/hu0GvXY43dxbe8FmNQiuAPugjUQg6dgD7wGB6AJSPA86AdF8CEE4f1wI3w8sS4EZ2tug7kKt38Cet0r7A==</latexit><latexit sha1_base64="o3NR/rwYsUwyoLX4pYQ6qUzsbFs=">AAADfnicfVLdbtMwGHUWfsb42+CSG4sKtgErSbXBJjRpMCS4QQyJdpPqNnJcp7UW28F2WIvnJ+CCW56Ad0K8DE67lZa/T3J8dM6xP8fHaZEzbaLoe7AQXrh46fLilaWr167fuLm8cqulZakIbRKZS3WUYk1zJmjTMJPTo0JRzNOcHqbH+5V++JEqzaR4b0YF7XDcFyxjBBtPJcs/UMot6tHcYLe6miBDh8ZynDu4CxFWfc5EYmc8buLYV1LrDSqMksXIQZTTzKwhb8yGCe+yR6Pqi1xi2W7sula4Z146SV7BhxCl1GA429Y3mLZ1Y8dUg0ix/sCsV6waSIhOZ8SNGXy+B5FC+0OeJo1uI1muRfWosb212YBRvbEV7cQ7HmxF8c6TTRjXo3HV9ta/fCoWv309SFaCz6gnScn9j5Eca92Oo8J0LFaGkZy6JVRqWmByjPu07aHAnOqOHafg4D3P9GAmlR/CwDE7u8JirvWIp97JsRno37WK/JvWLk223bFMFKWhgkwaZWUOjYRVpLDHFCUmH3mAiWL+rJAMsMLE+OCXkKAnRHKORc+ilmvHHYuqHmlma7Fz87oPyU3lkz/V4S91WKkvqb8nRd946m1BFTZSPbDjd4O99Wz+n42Jic3Pc/eRcufjO88I/hu0GvXY43dxbe8FmNQiuAPugjUQg6dgD7wGB6AJSPA86AdF8CEE4f1wI3w8sS4EZ2tug7kKt38Cet0r7A==</latexit><latexit sha1_base64="xa440SpmTZi3T5MqG/LGuh2EwYQ=">AAADfnicfVJbb9MwGHUWLmPcOnjkxaKCDdhKEq2wCk0aDAleEENiF6nuIsd1WmuxHWyHtfL8C/iFiD+D03aj5fZJjo/OOfbn+DgrC6ZNFH0PlsIrV69dX76xcvPW7Tt3G6v3DrWsFKEHRBZSHWdY04IJemCYKehxqSjmWUGPstO9Wj/6SpVmUnw245L2OB4IljOCjafSxg+UcYv6tDDYra2lyNCRsRwXDu5AhNWAM5HaOY+bOvaU1HqTCqNkOXYQFTQ368gb81HKT9jGuP4il1q2E7sTK9wrL52l7+AziDJqMJxv6xtctnUTx6UGkWKDoXlSs2ooITqfEzfn8MUeRArtD3meJidJ2mhGrSjZbm8lMGol7agTdzxoR3HnxRaMW9GkmmBW++lq8A31Jam4/zFSYK27cVSansXKMFJQt4IqTUtMTvGAdj0UmFPds5MUHHzkmT7MpfJDGDhh51dYzLUe88w7OTZD/btWk3/TupXJt3uWibIyVJBpo7wqoJGwjhT2maLEFGMPMFHMnxWSIVaYGB/8ChL0jEjOsehbdOi6cc+iukeW22bs3KLuQ3KX8tmf6uiXOqrVt9Tfk6IfPPWxpAobqZ7aybvB3jqb/2djYmrz88J9ZNz5+C4ygv8Gh0kr9vhT3Nx9MwtyGTwAD8E6iMFLsAveg31wAEjwOhgEZfAlBOHjcDN8PrUuBbM198FChds/AfGtKMQ=</latexit>

Experimental choice For every step w.r.t. to the malicious loss, take 10 steps for the benign loss

13

Alternating minimization: Results and Weight update

14


Takeaway Malicious objective is met while maintaining high validation accuracy for malicious model

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time


Val. Acc. Mal. (stealth)


G . Stealth with respect to accuracy checking isalso shown.


Figure 3: Alternating minimization attack with distance constraints for CNN on Fashion MNIST data.We use � = 10 and ⇢ = 1e�4. The number of epochs used by the malicious agent is Em = 10 and it runs 10steps of the stealth objective for every step of the malicious objective.

with similar loss values. Overall, the adversarial objective then becomes:

argmin�tm

�L({xi ,⌧i }ri=1,wtG) +L(Dm,wt

m) + ⇢k�tm � �t�1benk2 (5)

Note that for the training loss, the optimization is just performed with respect to wtm = wt�1

G + �tm, as abenign agent would do. Using explicit boosting, wt

G is replaced by wtm as well so that only the portion of

the loss corresponding to the malicious objective gets boosted by a factor �.Results and e↵ect on stealth: From Figure 2a, it is clear that the stealthy model poisoning attack is ableto cause targeted poisoning of the global model. We set the accuracy threshold �t to be 10% which impliesthat the malicious model is chosen for 10 iterations out of 15. This is in contrast to the targeted modelpoisoning attack which never has validation accuracy within 10% of the global model. Further, the weightupdate distribution for the stealthy poisoning attack (Figure 2b) is similar to that of a benign agent, owingto the additional terms in the loss function. Finally, in Figure 4, we see that the range of `2 distances for themalicious agent Rm is close, according to Eq. 3, to that between benign agents.

Concurrent work onmodel poisoning boosts the entire update (instead of just the malicious loss compo-nent as we do) when the global model is close to convergence in an attempt to perform model replacement[2] but this strategy is ine↵ective when the model has not converged.

3.4 Alternating minimization for improved model poisoning

30

40

50

60

70

80

90

100

110

2 4 6 8 10 12 14 16

Distanc

e

Time

Targeted Model Poisoning (Benign)Targeted Model Poisoning (Malicious)

Stealthy Model Poisoning (Benign)Stealthy Model Poisoning (Malicious)

Alternating Minimization (Benign)Alternating Minimization (Malicious)

Figure 4: Range of `2 distances between all benignagents and between the malicious agent and the be-nign agents.

While the stealthy model poisoning attack ensurestargeted poisoning of the global model while main-taining stealth according to the two conditions re-quired, it does not ensure that the malicious agent’supdate is chosen in every iteration. To achieve this,we propose an alternating minimization attack strat-egywhich decouples the targeted objective from thestealth objectives, providing finer control over therelative e↵ect of the two objectives. It works as fol-lows for iteration t. For each epoch i, the adversar-ial objective is first minimized starting from wi�1,t

m ,giving an update vector �i,tm . This is then boosted bya factor � and added to wi�1,t

m . Finally, the stealthobjective for that epoch is minimized starting fromwi,t

m =wi�1,tm +��i,tm , providing the malicious weight

7

14


Takeaway Shape and range match closely due to distance constraint

Takeaway Malicious objective is met while maintaining high validation accuracy for malicious model

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time








argmin�tm


m) + ⇢k�tm � �t�1benk2 (5)







30

40

50

60

70

80

90

100

110

2 4 6 8 10 12 14 16

Distanc

e

Time









7

14

Weight update distance spread (attack stealth measure)

Spread of distances between all the benign agents and between the malicious agent and the benign agents

L2<latexit sha1_base64="MfZBb7miRWUhZrN4uqzFU4g5LAw=">AAACpHicfVFLbxMxEJ5seZTwaovEhYtFBEIcovWqgeYWAQcORbQqSStll+B1ZlOra3tle0mj1f4CxBV+Gxd+C96kPMJrJGs+fd83mvFMWuTCujD80go2Ll2+cnXzWvv6jZu3bm9t74ysLg3HIde5Nicps5gLhUMnXI4nhUEm0xyP07PnjX78Ho0VWr1xiwITyWZKZIIz56mj/Uk02eqE3TDa6+1GJOxGvbBP+x70Qtp/sktoN1xGZ3D38Os7ADiYbLc+xFPNS4nK8ZxZO6Zh4ZKKGSd4jnU7Li0WjJ+xGY49VEyiTarlrDV54JkpybTxTzmyZH+tqJi0diFT75TMndrftYb8mzYuXbaXVEIVpUPFV42yMidOk+bjZCoMcpcvPGDcCD8r4afMMO78etqxwjnXUjI1reJRPaZJFTc90qzq0Lpe19NsXv+Q53+q5z/V80Z9gX5PBl956nWBhjltHlcxMzPJvPUi/88m1Mrm89o+Uln7832/Efk3GEVd6vEh7QyewSo24R7ch0dA4SkM4CUcwBA4zOAjfILPwcNgPzgKhitr0LqouQNrEbz9BuDI1+E=</latexit><latexit sha1_base64="lGyHA8gNsQMyszT+qjwwOHyEoy8=">AAACpHicfVHLbhMxFHWGVwmvFiQ2bCwiEGIR2aMGml1UWLAoolVJWikzRB7nTmrVj5HtIY1G8wWILXwFf8FPsOFb8CTlEV5Xsu7ROefqXt+bFVI4T8iXVnTh4qXLVzautq9dv3Hz1ubW7ZEzpeUw5EYae5wxB1JoGHrhJRwXFpjKJBxlp88a/egtWCeMfu0XBaSKzbTIBWc+UId7k3iy2SFdEu/0tmNMunGP9Gk/gB6h/SfbmHbJMjqDuwdfxafdz/uTrda7ZGp4qUB7LplzY0oKn1bMesEl1O2kdFAwfspmMA5QMwUurZaz1vhBYKY4NzY87fGS/bWiYsq5hcqCUzF/4n7XGvJv2rj0+U5aCV2UHjRfNcpLib3BzcfxVFjgXi4CYNyKMCvmJ8wy7sN62omGOTdKMT2tklE9pmmVND2yvOrQul7Xs3xe/5Dnf6pnP9WzRn0OYU8WXgbqVQGWeWMfVwmzM8WC9Tz/zyb0yhby2j4yVYfzfb8R/jcYxV0a8AHtDHbRKjbQPXQfPUIUPUUD9ALtoyHiaIbeow/oY/Qw2osOo+HKGrXOa+6gtYjefAN0v9md</latexit><latexit sha1_base64="lGyHA8gNsQMyszT+qjwwOHyEoy8=">AAACpHicfVHLbhMxFHWGVwmvFiQ2bCwiEGIR2aMGml1UWLAoolVJWikzRB7nTmrVj5HtIY1G8wWILXwFf8FPsOFb8CTlEV5Xsu7ROefqXt+bFVI4T8iXVnTh4qXLVzautq9dv3Hz1ubW7ZEzpeUw5EYae5wxB1JoGHrhJRwXFpjKJBxlp88a/egtWCeMfu0XBaSKzbTIBWc+UId7k3iy2SFdEu/0tmNMunGP9Gk/gB6h/SfbmHbJMjqDuwdfxafdz/uTrda7ZGp4qUB7LplzY0oKn1bMesEl1O2kdFAwfspmMA5QMwUurZaz1vhBYKY4NzY87fGS/bWiYsq5hcqCUzF/4n7XGvJv2rj0+U5aCV2UHjRfNcpLib3BzcfxVFjgXi4CYNyKMCvmJ8wy7sN62omGOTdKMT2tklE9pmmVND2yvOrQul7Xs3xe/5Dnf6pnP9WzRn0OYU8WXgbqVQGWeWMfVwmzM8WC9Tz/zyb0yhby2j4yVYfzfb8R/jcYxV0a8AHtDHbRKjbQPXQfPUIUPUUD9ALtoyHiaIbeow/oY/Qw2osOo+HKGrXOa+6gtYjefAN0v9md</latexit><latexit sha1_base64="vU3NtfEqp3HVOWF2pZdDd4z34iQ=">AAACpHicfVHbbhMxEHWWWwm3Xh55sYhAiIfIXjXQvFWUBx6KKCpJK2WXyOvMplZ9WdneptFqvwD1tXwbf4M3CZdwG8mao3POaMYzWSGF84R8bUU3bt66fWfjbvve/QcPH21ubQ+dKS2HATfS2NOMOZBCw8ALL+G0sMBUJuEkOz9o9JMLsE4Y/dHPC0gVm2qRC858oI4Px/F4s0O6JN7r7caYdOMe6dN+AD1C+y93Me2SRXTQKo7GW63PycTwUoH2XDLnRpQUPq2Y9YJLqNtJ6aBg/JxNYRSgZgpcWi1mrfHTwExwbmx42uMF+2tFxZRzc5UFp2L+zP2uNeTftFHp8720ErooPWi+bJSXEnuDm4/jibDAvZwHwLgVYVbMz5hl3If1tBMNM26UYnpSJcN6RNMqaXpkedWhdb2uZ/ms/iHP/lQvf6qXjfoGwp4svAvU+wIs88a+qBJmp4oF6yr/zyb00hby2j4yVYfzfb8R/jcYxl0a8Afa2X+9OuQGeoyeoOeIoldoH71FR2iAOJqiK3SNvkTPosPoOBosrVFrVbOD1iL69A3xsNX8</latexit>

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time








argmin�tm


m) + ⇢k�tm � �t�1benk2 (5)







30

40

50

60

70

80

90

100

110

2 4 6 8 10 12 14 16

Distanc

e

Time









7

15




Benign for all 3 attacks

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time








argmin�tm


m) + ⇢k�tm � �t�1benk2 (5)







30

40

50

60

70

80

90

100

110

2 4 6 8 10 12 14 16

Distanc

e

Time









7

15




Stealthy poison

Alt.min.


Targeted poison

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time








argmin�tm


m) + ⇢k�tm � �t�1benk2 (5)







30

40

50

60

70

80

90

100

110

2 4 6 8 10 12 14 16

Distanc

e

Time









7

15




Stealthy poison

Alt.min.


Adding distance constraints reduces distinguishability of malicious update

Takeaway

Targeted poison

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time








argmin�tm


m) + ⇢k�tm � �t�1benk2 (5)







30

40

50

60

70

80

90

100

110

2 4 6 8 10 12 14 16

Distanc

e

Time









7

15

Fragility of interpretability (Attack Stealth)

Using a suite of interpretability techniques [3] to compare global model decisions

16


Global model trained using only benign agents


16



Global model trained with one malicious model and the rest benign


16



Global model trained with one malicious model and the rest benign

Only two which appear to be

visually different


16

“Analyzing Federated Learning Through an Adversarial Lens”, joint work with Arjun Bhagoji, Prateek Mittal (Princeton University), Seraphin Calo (IBM Research), ICML, 2019.

Attacks on Byzantine-resilient aggregation

vector wi,tm for the next epoch. The malicious agent

can run this alternating minimization until boththe adversarial and stealth objectives have su�-ciently low values. Further, the independent mini-mization allows for each objective to be optimized for a di↵erent number of steps, depending on which ismore di�cult in achieve. In particular, we find that optimizing the stealth objective for a larger number ofsteps each epoch compared to the malicious objective leads to better stealth performance while maintainingtargeted poisoning.Results and e↵ect on stealth: The adversarial objective is achieved at the global model with high confi-dence starting from time step t = 2 and the global model converges to a point with good performance on thevalidation set. This attack can bypass the accuracy checking method as the accuracy on validation data ofthe malicious model is close to that of the global model.In Figure 4, we can see that the distance spread forthis attack closely follows and even overlaps that of benign updates throughout, thus achieving completestealth with respect to both properties.

4 Attacking Byzantine-resilient aggregation

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Val. Acc. Global (Krum)Mal. Conf. Global (Krum)Val. Acc. Global (Coomed)

Mal. Conf. Global (Coomed)

Figure 5: Model poisoning attacks with Byzantineresilient aggregation mechanisms. We use targetedmodel poisoning for coomed and alternating mini-mization for Krum.

There has been considerable recent work that hasproposed gradient aggregationmechanisms for dis-tributed learning that ensure convergence of theglobal model [4, 8, 18, 6, 29]. However, the aim ofthe Byzantine adversaries considered in this line ofwork is to ensure convergence to ine↵ective mod-els, i.e. models with poor classification perfor-mance. The goal of the adversary we consider is tar-geted model poisoning, which implies convergenceto an e↵ective model on the test data. This di↵er-ence in objectives leads to the lack of robustness ofthese Byzantine-resilient aggregation mechanismsagainst our attacks. We consider the aggregationmechanisms Krum [4] and coordinate-wise median[29] for our evaluation, both of which are provablyByzantine-resilient and converge under appropri-ate conditions on the loss function. Both aggregation mechanisms are also e�cient. Note that in general,these conditions do not hold for neural networks so the guarantees are only empirical.

4.1 KrumGiven n agents of which f are Byzantine, Krum requires that n � 2f + 3. At any time step t, updates⇣�t1, . . . ,�

tn

⌘are received at the server. For each �ti , the n � f � 2 closest (in terms of Lp norm) other updates

are chosen to form a set Ci and their distances added up to give a score S(�ti ) =P

�2Cik�ti � �k. Krum then

chooses �krum = �ti with the lowest score to add towti to givewt+1

i =wti +�krum. In Figure 5, we see the e↵ect

of the alternating minimization attack on Krum with a boosting factor of � = 2 for a federated learningsetup with 10 agents. Since there is no need to overcome the constant scaling factor ↵m, the attack canuse a much smaller boosting factor � than the number of agents to ensure model poisoning. The maliciousagent’s update is chosen by Krum for 26 of 40 time steps which leads to the malicious objective being met.Further, the global model converges to a point with good performance as the malicious agent has added thetraining loss to its stealth objective. We note that with the use of targeted model poisoning, we can causeKrum to converge to a model with poor performance as well (see Appendix B.4).

8

Takeaways 1. Adding resilience against attackers aiming to prevent convergence is ineffective

against model poisoning attacks 2. Krum chooses update closest to all others distance-constrained attacks are

effective⇒

17

Effect of Colluding Agents (parameter subspaces)

18

Agent 1 modified ~67K weights, agent 2 modified ~65 weights, overlap ~49K weights.

Agent1 modified ~93K weights, agent 2 modified ~65 weights, overlap ~29K weights.

100 agents, 2 colluding malicious agents10 agents, 2 colluding malicious agents

1. Collusion represents a stronger threat. Attacks require low boosting increasing attack stealth. 2. We observed that the decrease in boosting is not linear in the number of colluding agents.

Takeaways

Distributed Backdoor Attack

19

Distributed Backdoor Attack

19

“DBA: Distributed Backdoor Attacks Against Federated Learning”, Chulin Xie, Keli Huang, Pin-Yu Chen, Bo Li, ICLR, 2020.

Targeted Poisoning of P2P Learning Systems

20

A single malicious peer attempts to trick the benign peers to perform targeted misclassification

• Each peer i updates parameters with SGD on one random mini batch

• Each peer decides to share the parameters with probability

• If sharing, the peer randomly selects a peer j to share with

• If it has received parameters, the peer merges them with its own

pi<latexit sha1_base64="hq2aIKwkSQBQbN6vWzmW9iSTH/Q=">AAAB6nicdVDLSgNBEOyNrxhfUY9eBoPgaZnVLGZvQS8eI5oHJEuYncwmQ2YfzMwKIeQTvHhQxKtf5M2/cTaJoKIFDUVVN91dQSq40hh/WIWV1bX1jeJmaWt7Z3evvH/QUkkmKWvSRCSyExDFBI9ZU3MtWCeVjESBYO1gfJX77XsmFU/iOz1JmR+RYcxDTok20m3a5/1yBdvY9VwHI2y72PHOc+J5tarrIsfGc1RgiUa//N4bJDSLWKypIEp1HZxqf0qk5lSwWamXKZYSOiZD1jU0JhFT/nR+6gydGGWAwkSaijWaq98npiRSahIFpjMieqR+e7n4l9fNdFjzpzxOM81iulgUZgLpBOV/owGXjGoxMYRQyc2tiI6IJFSbdEomhK9P0f+kdWY72HZuqpX65TKOIhzBMZyCAxdQh2toQBMoDOEBnuDZEtaj9WK9LloL1nLmEH7AevsExK2OHg==</latexit><latexit sha1_base64="hq2aIKwkSQBQbN6vWzmW9iSTH/Q=">AAAB6nicdVDLSgNBEOyNrxhfUY9eBoPgaZnVLGZvQS8eI5oHJEuYncwmQ2YfzMwKIeQTvHhQxKtf5M2/cTaJoKIFDUVVN91dQSq40hh/WIWV1bX1jeJmaWt7Z3evvH/QUkkmKWvSRCSyExDFBI9ZU3MtWCeVjESBYO1gfJX77XsmFU/iOz1JmR+RYcxDTok20m3a5/1yBdvY9VwHI2y72PHOc+J5tarrIsfGc1RgiUa//N4bJDSLWKypIEp1HZxqf0qk5lSwWamXKZYSOiZD1jU0JhFT/nR+6gydGGWAwkSaijWaq98npiRSahIFpjMieqR+e7n4l9fNdFjzpzxOM81iulgUZgLpBOV/owGXjGoxMYRQyc2tiI6IJFSbdEomhK9P0f+kdWY72HZuqpX65TKOIhzBMZyCAxdQh2toQBMoDOEBnuDZEtaj9WK9LloL1nLmEH7AevsExK2OHg==</latexit><latexit sha1_base64="hq2aIKwkSQBQbN6vWzmW9iSTH/Q=">AAAB6nicdVDLSgNBEOyNrxhfUY9eBoPgaZnVLGZvQS8eI5oHJEuYncwmQ2YfzMwKIeQTvHhQxKtf5M2/cTaJoKIFDUVVN91dQSq40hh/WIWV1bX1jeJmaWt7Z3evvH/QUkkmKWvSRCSyExDFBI9ZU3MtWCeVjESBYO1gfJX77XsmFU/iOz1JmR+RYcxDTok20m3a5/1yBdvY9VwHI2y72PHOc+J5tarrIsfGc1RgiUa//N4bJDSLWKypIEp1HZxqf0qk5lSwWamXKZYSOiZD1jU0JhFT/nR+6gydGGWAwkSaijWaq98npiRSahIFpjMieqR+e7n4l9fNdFjzpzxOM81iulgUZgLpBOV/owGXjGoxMYRQyc2tiI6IJFSbdEomhK9P0f+kdWY72HZuqpX65TKOIhzBMZyCAxdQh2toQBMoDOEBnuDZEtaj9WK9LloL1nLmEH7AevsExK2OHg==</latexit><latexit sha1_base64="hq2aIKwkSQBQbN6vWzmW9iSTH/Q=">AAAB6nicdVDLSgNBEOyNrxhfUY9eBoPgaZnVLGZvQS8eI5oHJEuYncwmQ2YfzMwKIeQTvHhQxKtf5M2/cTaJoKIFDUVVN91dQSq40hh/WIWV1bX1jeJmaWt7Z3evvH/QUkkmKWvSRCSyExDFBI9ZU3MtWCeVjESBYO1gfJX77XsmFU/iOz1JmR+RYcxDTok20m3a5/1yBdvY9VwHI2y72PHOc+J5tarrIsfGc1RgiUa//N4bJDSLWKypIEp1HZxqf0qk5lSwWamXKZYSOiZD1jU0JhFT/nR+6gydGGWAwkSaijWaq98npiRSahIFpjMieqR+e7n4l9fNdFjzpzxOM81iulgUZgLpBOV/owGXjGoxMYRQyc2tiI6IJFSbdEomhK9P0f+kdWY72HZuqpX65TKOIhzBMZyCAxdQh2toQBMoDOEBnuDZEtaj9WK9LloL1nLmEH7AevsExK2OHg==</latexit>

Gossip SGD

Validation Accuracy

21

pi = 0.02 pi = 0.05 pi = 1

10 peers with one single attacker; fmnist dataset used

Attack Confidence

22

pi = 0.02 pi = 0.05 pi = 1

Distribution of the Weight Updates

23

“Model Poisoning Attacks Against Distributed Machine Learning Systems”, joint work with Richard Tomsett (IBM Research), Kevin Chen (ARL), SPIE, 2019.

Conclusion

✦ Scalability: implementing attacks at scale

✦ Attack Robustness: behavior of the poisoned models in parameter space (effect of noise, parameter truncation, defenses on attack accuracy)

✦ Attack Generalizability: behavior in input space around poisoned points. Attacking samples that are not in the adversary’s auxiliary dataset.

Our attacks demonstrate the feasibility of using model poisoning as an attack vector towards introducing targeted backdoor into a global model in a federated learning setting.

Next Steps:

24

Thank you for listening! Questions?

[1] McMahan et al., Communication-Efficient Learning of Deep Networks from Decentralized Data, AISTATS 2017

[2] Xiao et al., Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, arXiv preprint arXiv:1708.07747, 2017 [3] Alber et al., iNNvestigate neural networks!, arXiv preprint arXiv:1808.04260, 2018 [4] Sun et al. Can You Really Backdoor Federated Learning? arXiv preprint arXiv:1911.07963

References

25

Backup slides

26

Outlier Sparsification/Scaling and Noise Based Defense

27

Attack: Alternating Minimization with projected gradient and boosting

Related Work (detailed)

✦ Byzantine defenses are all designed to tackle attacks which cause convergence to the global model to a point which is ineffective, i.e. has bad performance on test data (from Mhamdi et al. - ‘‘sub-optimal to utterly ineffective models’’)

✦ Blanchard et al. - proposed Krum, which looks for the update closest to k-2 of the others

✦ Yin et al. - coordinate-wise median, provably robust against Byzantine adversaries

✦ Bagdasaryan et al. - proposed attacks at convergence time to insert backdoors, but this severely restricts the adversary

28

Estimation to improve attacks

wtG = wt�1

G + �[k]\m + ↵m�tm<latexit sha1_base64="NbJliDCLJ9JzA9UVs7qZE3wfzbg=">AAADB3icfVHLjtMwFHXDayiP6cCSjUWFhEBUSdXCdIE0AiTYIAaJdkZqQnTjOK1V24lsh05l5QMQH8MOsWXLH/AXbGGF03ag5XUly0fnnOtrnZsUnGnj+18a3pmz585f2LnYvHT5ytXd1t61kc5LReiQ5DxXxwloypmkQ8MMp8eFoiASTo+S2eNaP3pDlWa5fGUWBY0ETCTLGAHjqLiVhlMwNkyyefXaxE/xQ7xBWHMvqBx595QUNkwpN1BVsR3PIhxqagSTpcaiql3AiynEAm84Y+HebbX9jt/d7/e62O90+/4gGDjQ94PB/R4OOv6y2mhdh/Fe412Y5qQUVBrCQetx4BcmsqAMI5xWzbDUtAAygwkdOyhBUB3ZZRwVvuWYFGe5ckcavGQ3OywIrRcicU4BZqp/12ryb9q4NNl+ZJksSkMlWQ3KSo5NjutsccoUJYYvHACimPsrJlNQQIzbQDOUdE5yIUCmNhxV4yCyYT0jyWw7qKptvc7/pzz/Uz35pZ7U6hPqclL0uaNeFFSBydUdG4KaCHDW9f0/G5Mrm7u38khE5dZ3uiP8bzDqdgKHX/baB4/Wi9xBN9BNdBsF6AE6QM/QIRoigj6jr+gb+u699d57H7yPK6vXWPdcR1vlffoBROL+mA==</latexit><latexit sha1_base64="NbJliDCLJ9JzA9UVs7qZE3wfzbg=">AAADB3icfVHLjtMwFHXDayiP6cCSjUWFhEBUSdXCdIE0AiTYIAaJdkZqQnTjOK1V24lsh05l5QMQH8MOsWXLH/AXbGGF03ag5XUly0fnnOtrnZsUnGnj+18a3pmz585f2LnYvHT5ytXd1t61kc5LReiQ5DxXxwloypmkQ8MMp8eFoiASTo+S2eNaP3pDlWa5fGUWBY0ETCTLGAHjqLiVhlMwNkyyefXaxE/xQ7xBWHMvqBx595QUNkwpN1BVsR3PIhxqagSTpcaiql3AiynEAm84Y+HebbX9jt/d7/e62O90+/4gGDjQ94PB/R4OOv6y2mhdh/Fe412Y5qQUVBrCQetx4BcmsqAMI5xWzbDUtAAygwkdOyhBUB3ZZRwVvuWYFGe5ckcavGQ3OywIrRcicU4BZqp/12ryb9q4NNl+ZJksSkMlWQ3KSo5NjutsccoUJYYvHACimPsrJlNQQIzbQDOUdE5yIUCmNhxV4yCyYT0jyWw7qKptvc7/pzz/Uz35pZ7U6hPqclL0uaNeFFSBydUdG4KaCHDW9f0/G5Mrm7u38khE5dZ3uiP8bzDqdgKHX/baB4/Wi9xBN9BNdBsF6AE6QM/QIRoigj6jr+gb+u699d57H7yPK6vXWPdcR1vlffoBROL+mA==</latexit><latexit sha1_base64="NbJliDCLJ9JzA9UVs7qZE3wfzbg=">AAADB3icfVHLjtMwFHXDayiP6cCSjUWFhEBUSdXCdIE0AiTYIAaJdkZqQnTjOK1V24lsh05l5QMQH8MOsWXLH/AXbGGF03ag5XUly0fnnOtrnZsUnGnj+18a3pmz585f2LnYvHT5ytXd1t61kc5LReiQ5DxXxwloypmkQ8MMp8eFoiASTo+S2eNaP3pDlWa5fGUWBY0ETCTLGAHjqLiVhlMwNkyyefXaxE/xQ7xBWHMvqBx595QUNkwpN1BVsR3PIhxqagSTpcaiql3AiynEAm84Y+HebbX9jt/d7/e62O90+/4gGDjQ94PB/R4OOv6y2mhdh/Fe412Y5qQUVBrCQetx4BcmsqAMI5xWzbDUtAAygwkdOyhBUB3ZZRwVvuWYFGe5ckcavGQ3OywIrRcicU4BZqp/12ryb9q4NNl+ZJksSkMlWQ3KSo5NjutsccoUJYYvHACimPsrJlNQQIzbQDOUdE5yIUCmNhxV4yCyYT0jyWw7qKptvc7/pzz/Uz35pZ7U6hPqclL0uaNeFFSBydUdG4KaCHDW9f0/G5Mrm7u38khE5dZ3uiP8bzDqdgKHX/baB4/Wi9xBN9BNdBsF6AE6QM/QIRoigj6jr+gb+u699d57H7yPK6vXWPdcR1vlffoBROL+mA==</latexit><latexit sha1_base64="NbJliDCLJ9JzA9UVs7qZE3wfzbg=">AAADB3icfVHLjtMwFHXDayiP6cCSjUWFhEBUSdXCdIE0AiTYIAaJdkZqQnTjOK1V24lsh05l5QMQH8MOsWXLH/AXbGGF03ag5XUly0fnnOtrnZsUnGnj+18a3pmz585f2LnYvHT5ytXd1t61kc5LReiQ5DxXxwloypmkQ8MMp8eFoiASTo+S2eNaP3pDlWa5fGUWBY0ETCTLGAHjqLiVhlMwNkyyefXaxE/xQ7xBWHMvqBx595QUNkwpN1BVsR3PIhxqagSTpcaiql3AiynEAm84Y+HebbX9jt/d7/e62O90+/4gGDjQ94PB/R4OOv6y2mhdh/Fe412Y5qQUVBrCQetx4BcmsqAMI5xWzbDUtAAygwkdOyhBUB3ZZRwVvuWYFGe5ckcavGQ3OywIrRcicU4BZqp/12ryb9q4NNl+ZJksSkMlWQ3KSo5NjutsccoUJYYvHACimPsrJlNQQIzbQDOUdE5yIUCmNhxV4yCyYT0jyWw7qKptvc7/pzz/Uz35pZ7U6hPqclL0uaNeFFSBydUdG4KaCHDW9f0/G5Mrm7u38khE5dZ3uiP8bzDqdgKHX/baB4/Wi9xBN9BNdBsF6AE6QM/QIRoigj6jr+gb+u699d57H7yPK6vXWPdcR1vlffoBROL+mA==</latexit>

�[k]\m = �t�1[k]\m

<latexit sha1_base64="D1nVs6RHQbU8adP5DTkzy5zubXQ=">AAAC7nicfVFdixMxFE3Hr7V+bFcffQkWQQTLTGl1+yAs6oMg4gq2uzAZSya904YmmSHJ2C1hfoHvvomvvvprxDf9J2baqltdvRByOOdcbnJuWghubBh+bQTnzl+4eGnncvPK1WvXd1t7N0YmLzWDIctFro9TakBwBUPLrYDjQgOVqYCjdP6k1o/egjY8V6/tsoBE0qniGWfUemrceo7JjFpHUunIBISlVTV28TzBxICVXJUGywo/wviU442z96MzbONWO+yE3f1+r4vDTrcfDqKBB/0wGjzo4agTrqqNNnU43mu8I5OclRKUZYIaE0dhYRNHteVMQNUkpYGCsjmdQuyhohJM4la/rvAdz0xwlmt/lMUr9nSHo9KYpUy9U1I7M39qNXmWFpc2208cV0VpQbH1oKwU2Oa4jhBPuAZmxdIDyjT3b8VsRjVl1gfdJAoWLJeSqokjoyqOEkfqGWnm2lFVbetptqh+yYu/1ZPf6kmtPgWfk4YXnnpZgKY21/ccoXoqqbdu7v/ZuFrb/L2VR7pa388d4X+DUbcTefyq1z54vFnkDrqFbqO7KEIP0QF6hg7REDH0GX1B39D3oAjeBx+Cj2tr0Nj03ERbFXz6AUim9KE=</latexit><latexit sha1_base64="D1nVs6RHQbU8adP5DTkzy5zubXQ=">AAAC7nicfVFdixMxFE3Hr7V+bFcffQkWQQTLTGl1+yAs6oMg4gq2uzAZSya904YmmSHJ2C1hfoHvvomvvvprxDf9J2baqltdvRByOOdcbnJuWghubBh+bQTnzl+4eGnncvPK1WvXd1t7N0YmLzWDIctFro9TakBwBUPLrYDjQgOVqYCjdP6k1o/egjY8V6/tsoBE0qniGWfUemrceo7JjFpHUunIBISlVTV28TzBxICVXJUGywo/wviU442z96MzbONWO+yE3f1+r4vDTrcfDqKBB/0wGjzo4agTrqqNNnU43mu8I5OclRKUZYIaE0dhYRNHteVMQNUkpYGCsjmdQuyhohJM4la/rvAdz0xwlmt/lMUr9nSHo9KYpUy9U1I7M39qNXmWFpc2208cV0VpQbH1oKwU2Oa4jhBPuAZmxdIDyjT3b8VsRjVl1gfdJAoWLJeSqokjoyqOEkfqGWnm2lFVbetptqh+yYu/1ZPf6kmtPgWfk4YXnnpZgKY21/ccoXoqqbdu7v/ZuFrb/L2VR7pa388d4X+DUbcTefyq1z54vFnkDrqFbqO7KEIP0QF6hg7REDH0GX1B39D3oAjeBx+Cj2tr0Nj03ERbFXz6AUim9KE=</latexit><latexit sha1_base64="D1nVs6RHQbU8adP5DTkzy5zubXQ=">AAAC7nicfVFdixMxFE3Hr7V+bFcffQkWQQTLTGl1+yAs6oMg4gq2uzAZSya904YmmSHJ2C1hfoHvvomvvvprxDf9J2baqltdvRByOOdcbnJuWghubBh+bQTnzl+4eGnncvPK1WvXd1t7N0YmLzWDIctFro9TakBwBUPLrYDjQgOVqYCjdP6k1o/egjY8V6/tsoBE0qniGWfUemrceo7JjFpHUunIBISlVTV28TzBxICVXJUGywo/wviU442z96MzbONWO+yE3f1+r4vDTrcfDqKBB/0wGjzo4agTrqqNNnU43mu8I5OclRKUZYIaE0dhYRNHteVMQNUkpYGCsjmdQuyhohJM4la/rvAdz0xwlmt/lMUr9nSHo9KYpUy9U1I7M39qNXmWFpc2208cV0VpQbH1oKwU2Oa4jhBPuAZmxdIDyjT3b8VsRjVl1gfdJAoWLJeSqokjoyqOEkfqGWnm2lFVbetptqh+yYu/1ZPf6kmtPgWfk4YXnnpZgKY21/ccoXoqqbdu7v/ZuFrb/L2VR7pa388d4X+DUbcTefyq1z54vFnkDrqFbqO7KEIP0QF6hg7REDH0GX1B39D3oAjeBx+Cj2tr0Nj03ERbFXz6AUim9KE=</latexit><latexit sha1_base64="D1nVs6RHQbU8adP5DTkzy5zubXQ=">AAAC7nicfVFdixMxFE3Hr7V+bFcffQkWQQTLTGl1+yAs6oMg4gq2uzAZSya904YmmSHJ2C1hfoHvvomvvvprxDf9J2baqltdvRByOOdcbnJuWghubBh+bQTnzl+4eGnncvPK1WvXd1t7N0YmLzWDIctFro9TakBwBUPLrYDjQgOVqYCjdP6k1o/egjY8V6/tsoBE0qniGWfUemrceo7JjFpHUunIBISlVTV28TzBxICVXJUGywo/wviU442z96MzbONWO+yE3f1+r4vDTrcfDqKBB/0wGjzo4agTrqqNNnU43mu8I5OclRKUZYIaE0dhYRNHteVMQNUkpYGCsjmdQuyhohJM4la/rvAdz0xwlmt/lMUr9nSHo9KYpUy9U1I7M39qNXmWFpc2208cV0VpQbH1oKwU2Oa4jhBPuAZmxdIDyjT3b8VsRjVl1gfdJAoWLJeSqokjoyqOEkfqGWnm2lFVbetptqh+yYu/1ZPf6kmtPgWfk4YXnnpZgKY21/ccoXoqqbdu7v/ZuFrb/L2VR7pa388d4X+DUbcTefyq1z54vFnkDrqFbqO7KEIP0QF6hg7REDH0GX1B39D3oAjeBx+Cj2tr0Nj03ERbFXz6AUim9KE=</latexit>

Attack TargetedModel Poisoning

AlternatingMinimization

Estimation None Previous step None Previous stept = 2 0.63 0.82 0.17 0.47t = 3 0.93 0.98 0.34 0.89t = 4 0.99 1.0 0.88 1.0

Table 1: Comparison of confidence of targeted misclassification with and without the use of previous stepestimation for the targeted model poisoning and alternating minimization attacks.

5.2.2 Results

Attacks using previous step estimation with the pre-optimization correction are more e↵ective at achievingthe adversarial objective for both the targeted model poisoning and alternating minimization attacks. InTable 1, we can see that the global model misclassifies the desired sample with a higher confidence whenusing previous step estimation in the first few iterations. We found that using post-optimization correctionwas not e↵ective, leading to both lower attack success and a↵ecting global model convergence.

6 Discussion

6.1 Model poisoning vs. data poisoningIn this section, we elucidate the di↵erences between model poisoning and data poisoning both qualitativelyand quantitatively. Data poisoning attacks largely fall in two categories: clean-label [20, 13] and dirty-label[7, 10, 15]. Clean-label attacks assume that the adversary cannot change the label of any training data asthere is a process by which data is certified as belonging to the correct class and the poisoning of datasamples has to be imperceptible. On the other hand, to carry out dirty-label poisoning, the adversary justhas to introduce a number of copies of the data sample it wishes to mis-classify with the desired targetlabel into the training set since there is no requirement that a data sample belong to the correct class.Dirty-label data poisoning has been shown to achieve high-confidence targeted misclassification for deepneural networks with the addition of around 50 poisoned samples to the training data [7].

6.1.1 Dirty-label data poisoning in federated learning

In our comparison with data poisoning, we use the dirty-label data poisoning framework for two reasons.First, federated learning operates under the assumption that data is never shared, only learned models.Thus, the adversary is not concerned with notions of imperceptibility for data certification. Second, clean-label data poisoning assumes access at train time to the global parameter vector, which is absent in thefederated learning setting. Using the same experimental setup as before (CNN on Fashion MNIST data,10 agents chosen every time step), we add copies of the sample that is to be misclassified to the trainingset of the malicious agent with the appropriate target label. We experiment with two settings. In the first,we add multiple copies of the same sample to the training set. In the second, we add a small amount ofrandom uniform noise to each pixel [7] when generating copies. We observe that even when we add 1000copies of the sample to the training set, the data poisoning attack is completely ine↵ective at causing targetedpoisoning in the global model. This occurs due to the fact that malicious agent’s update is scaled, which againunderlies the importance of boosting while performing model poisoning. We note also that if the updategenerated using data poisoning is boosted, it a↵ects the performance of the global model as the entireupdate is boosted, not just the malicious part. Thus, model poisoning attacks are much more e↵ective thandata poisoning in the federated learning setting.

6.2 Interpreting poisoned modelsNeural networks are often treated as black boxes with little transparency into their internal representationor understanding of the underlying basis for their decisions. Interpretability techniques are designed to

10

Estimating update from other agents

Previous step estimation:

Improvement in attack confidence (CNN on Fashion MNIST, 10 agents) 29

Results on Adult Census dataset

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time



(a) Targeted model poisoning (b) Comparison of weight update distributionsfor targeted model poisoning

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time



(c) Stealthy model poisoning with � = 20 and ⇢ = 1e�4 (d) Comparison of weight update distributionsfor stealthy model poisoning

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time



(e) Alternating minimization with � = 20 and ⇢ = 1e�4 and 10epochs for the malicious agent

(f) Comparison of weight update distributions foralternating minimization

Figure 8: Attacks on a fully connected neural network on the Census dataset.

14

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time




0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time




0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time






14

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time




0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time




0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time






14

30

Results on 100 agents

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40 45 50

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Val. Acc. GlobalConf. Global (5!7)

(a) Targeted model poisoning with � = 100.

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40 45 50

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Val. Acc. GlobalConf. Global (5!7)

Val. Acc. Mal. (Stealth)

(b) Alternating minimization with � = 100, 100 epochs for themalicious agent and 10 steps for the stealth objective for everystep of the benign objective.

Figure 10: Attacks on federated learning in a setting with K = 100 and a single malicious agent for aCNN on the Fashion MNIST data.

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time



(a) Targeted model poisoning with � = 2 against Krum.

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time



(b) Alternating minimization attack with � = 2 againstcoomed.

Figure 11: Additional results for attacks on Byzantine-resilient aggregation mechanisms.

16

31

Attack with 10 targets

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Val. Acc. GlobalMal. Obj. (Fraction of targets)


(a) Targeted model poisoning.

0

0.2

0.4

0.6

0.8

1

5 10 15 20 25 30 35 40

0

20

40

60

80

100

Con

fide

nce

Classification

accu

racy

Time

Val. Acc. GlobalMal. Obj. (Fraction of targets)


(b) Alternating minimization with 10 epochs for the maliciousagent and 10 steps for the stealth objective for every step of thebenign objective.

Figure 9: Attacks with multiple targets (r = 10) for a CNN on the Fashion MNIST data.

B.3 Randomized agent selectionWhen the number of agents increases to k = 100, the malicious agent is not selected in every step. Further,the size of |Dm| decreases, which makes the benign training step in the alternating minimization attackmore challenging. The challenges posed in this setting are reflected in Figure 10a, where although targetedmodel poisoning is able to introduce a targeted backdoor, it is not present for every step as there are stepswhere only benign agents provide updates. Nevertheless, targetedmodel poisoning is e↵ective overall, withthe malicious objective achieved along with convergence of the global model at the end of training. Thealternating minimization attack strategy with stealth (Figure 10b) is also able to introduce the backdoor, aswell as increase the classification accuracy of the malicious model on test data. However, the improvementin performance is limited by the paucity of data for the malicious agent. It is an open question if dataaugmentation could help improve this accuracy.

B.4 Bypassing Byzantine-resilient aggregation mechanismsIn Section 4, we presented the results of successful attacks on two di↵erent Byzantine resilient aggregationmechanisms: Krum [4] and coordinate-wise median (coomed) [29]. In this section, we present the resultsfor targeted model poisoning when Krum is used (Figure 11a). The attack uses a boosting factor of � = 2with k = 10. Since there is no need to overcome the constant scaling factor ↵m, the attacks can use a muchsmaller boosting factor � to ensure the global model has the targeted backdoor. With the targeted modelpoisoning attack, the malicious agent’s update is the one chosen by Krum for 34 of 40 time steps but thiscauses the validation accuracy on the global model to be extremely low. Thus, our attack causes Krum toconverge to an ine↵ective model, in contrast to its stated claims of being Byzantine-resilient. However,our attack does not achieve its goal of ensuring that the global model converges to a point with goodperformance on the test set due to Krum selecting just a single agent at each time step.

We also consider the e↵ectiveness of the alternating minimization attack strategy when coomed is usedfor aggregation. While we have shown targeted model poisoning to be e↵ective even when coomed is used,Figure 11b demonstrates that alternating minimization, which ensures that the local model learned at themalicious agent also has high validation accuracy, is not e↵ective.

C Visualization of weight update distributionsFigure C shows the evolution of weight update distributions for the 4 di↵erent attack strategies on theCNN trained on the Faishon MNIST dataset. Time slices of this evolution were shown in the main text ofthe paper. The baseline and concatenated training attacks lead to weight update distributions that di↵erwidely for benign and malicious agents. The alternating minimization attack without distance constraintsreduces this qualitative di↵erence somewhat but the closest weight update distributions are obtained withthe alternating minimization attack with distance constraints.

15

32

Understanding Federated Learning Through an Adversarial Lens · Understanding Federated Learning...

Documents

Transcript of Understanding Federated Learning Through an Adversarial Lens · Understanding Federated Learning...