settings: rework tracking settings connections and settings plugins

Completely rework how settings plugin handle connections and how
NMSettings tracks the list of connections.

Previously, settings plugins would return objects of (a subtype of) type
NMSettingsConnection. The NMSettingsConnection was tightly coupled with
the settings plugin. That has a lot of downsides.

Change that. When changing this basic relation how settings connections
are tracked, everything falls appart. That's why this is a huge change.
Also, since I have to largely rewrite the settings plugins, I also
added support for multiple keyfile directories, handle in-memory
connections only by keyfile plugin and (partly) use copy-on-write NMConnection
instances. I don't want to spend effort rewriting large parts while
preserving the old way, that anyway should change. E.g. while rewriting ifcfg-rh,
I don't want to let it handle in-memory connections because that's not right
long-term.

--

If the settings plugins themself create subtypes of NMSettingsConnection
instances, then a lot of knowledge about tracking connections moves
to the plugins.
Just try to follow the code what happend during nm_settings_add_connection().
Note how the logic is spread out:
 - nm_settings_add_connection() calls plugin's add_connection()
 - add_connection() creates a NMSettingsConnection subtype
 - the plugin has to know that it's called during add-connection and
   not emit NM_SETTINGS_PLUGIN_CONNECTION_ADDED signal
 - NMSettings calls claim_connection() which hocks up the new
   NMSettingsConnection instance and configures the instance
   (like calling nm_settings_connection_added()).
This summary does not sound like a lot, but try to follow that code. The logic
is all over the place.

Instead, settings plugins should have a very simple API for adding, modifying,
deleting, loading and reloading connections. All the plugin does is to return a
NMSettingsStorage handle. The storage instance is a handle to identify a profile
in storage (e.g. a particular file). The settings plugin is free to subtype
NMSettingsStorage, but it's not necessary.
There are no more events raised, and the settings plugin implements the small
API in a straightforward manner.
NMSettings now drives all of this. Even NMSettingsConnection has now
very little concern about how it's tracked and delegates only to NMSettings.

This should make settings plugins simpler. Currently settings plugins
are so cumbersome to implement, that we avoid having them. It should not be
like that and it should be easy, beneficial and lightweight to create a new
settings plugin.

Note also how the settings plugins no longer care about duplicate UUIDs.
Duplicated UUIDs are a fact of life and NMSettings must handle them. No
need to overly concern settings plugins with that.

--

NMSettingsConnection is exposed directly on D-Bus (being a subtype of
NMDBusObject) but it was also a GObject type provided by the settings
plugin. Hence, it was not possible to migrate a profile from one plugin to
another.
However that would be useful when one profile does not support a
connection type (like ifcfg-rh not supporting VPN). Currently such
migration is not implemented except for migrating them to/from keyfile's
run directory. The problem is that migrating profiles in general is
complicated but in some cases it is important to do.

For example checkpoint rollback should recreate the profile in the right
settings plugin, not just add it to persistent storage. This is not yet
properly implemented.

--

Previously, both keyfile and ifcfg-rh plugin implemented in-memory (unsaved)
profiles, while ifupdown plugin cannot handle them. That meant duplication of code
and a ifupdown profile could not be modified or made unsaved.
This is now unified and only keyfile plugin handles in-memory profiles (bgo #744711).
Also, NMSettings is aware of such profiles and treats them specially.
In particular, NMSettings drives the migration between persistent and non-persistent
storage.

Note that a settings plugins may create truly generated, in-memory profiles.
The settings plugin is free to generate and persist the profiles in any way it
wishes. But the concept of "unsaved" profiles is now something explicitly handled
by keyfile plugin. Also, these "unsaved" keyfile profiles are persisted to file system
too, to the /run directory. This is great for two reasons: first of all, all
profiles from keyfile storage in fact have a backing file -- even the
unsaved ones. It also means you can create "unsaved" profiles in /run
and load them with `nmcli connection load`, meaning there is a file
based API for creating unsaved profiles.
The other advantage is that these profiles now survive restarting
NetworkManager. It's paramount that restarting the daemon is as
non-disruptive as possible. Persisting unsaved files to /run improves
here significantly.

--

In the past, NMSettingsConnection also implemented NMConnection interface.
That was already changed a while ago and instead users call now
nm_settings_connection_get_connection() to delegate to a
NMSimpleConnection. What however still happened was that the NMConnection
instance gets never swapped but instead the instance was modified with
nm_connection_replace_settings_from_connection(), clear-secrets, etc.
Change that and treat the NMConnection instance immutable. Instead of modifying
it, reference/clone a new instance. This changes that previously when somebody
wanted to keep a reference to an NMConnection, then the profile would be cloned.
Now, it is supposed to be safe to reference the instance directly and everybody
must ensure not to modify the instance. nmtst_connection_assert_unchanging()
should help with that.
The point is that the settings plugins may keep references to the
NMConnection instance, and so does the NMSettingsConnection. We want
to avoid cloning the instances as long as they are the same.
Likewise, the device's applied connection can now also be referenced
instead of cloning it. This is not yet done, and possibly there are
further improvements possible.

--

Also implement multiple keyfile directores /usr/lib, /etc, /run (rh #1674545,
bgo #772414).

It was always the case that multiple files could provide the same UUID
(both in case of keyfile and ifcfg-rh). For keyfile plugin, if a profile in
read-only storage in /usr/lib gets modified, then it gets actually stored in
/etc (or /run, if the profile is unsaved).

--

While at it, make /etc/network/interfaces profiles for ifupdown plugin reloadable.

--

https://bugzilla.gnome.org/show_bug.cgi?id=772414
https://bugzilla.gnome.org/show_bug.cgi?id=744711
https://bugzilla.redhat.com/show_bug.cgi?id=1674545

58 files changed, 6358 insertions, 3250 deletions

diff --git a/Makefile.am b/Makefile.am
index 47ccace714..3c190f0e73 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -2094,13 +2094,17 @@ src_libNetworkManager_la_SOURCES = \
 	src/settings/nm-secret-agent.h \
 	src/settings/nm-settings-connection.c \
 	src/settings/nm-settings-connection.h \
+	src/settings/nm-settings-storage.c \
+	src/settings/nm-settings-storage.h \
 	src/settings/nm-settings-plugin.c \
 	src/settings/nm-settings-plugin.h \
 	src/settings/nm-settings.c \
 	src/settings/nm-settings.h \
+	src/settings/nm-settings-utils.c \
+	src/settings/nm-settings-utils.h \
 	\
-	src/settings/plugins/keyfile/nms-keyfile-connection.c \
-	src/settings/plugins/keyfile/nms-keyfile-connection.h \
+	src/settings/plugins/keyfile/nms-keyfile-storage.c \
+	src/settings/plugins/keyfile/nms-keyfile-storage.h \
 	src/settings/plugins/keyfile/nms-keyfile-plugin.c \
 	src/settings/plugins/keyfile/nms-keyfile-plugin.h \
 	src/settings/plugins/keyfile/nms-keyfile-reader.c \
@@ -2819,10 +2823,11 @@ $(src_settings_plugins_ifcfg_rh_libnms_ifcfg_rh_core_la_OBJECTS): $(libnm_core_l
 ###############################################################################
 
 src_settings_plugins_ifcfg_rh_libnm_settings_plugin_ifcfg_rh_la_SOURCES = \
-	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.c \
-	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.h \
+	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.c \
+	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.h \
 	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.c \
-	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.h
+	src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.h \
+	$(NULL)
 
 src_settings_plugins_ifcfg_rh_libnm_settings_plugin_ifcfg_rh_la_CPPFLAGS = $(src_cppflags_base)
 
@@ -3135,8 +3140,6 @@ src_settings_plugins_ifupdown_libnms_ifupdown_core_la_LIBADD = \
 	$(NULL)
 
 src_settings_plugins_ifupdown_libnm_settings_plugin_ifupdown_la_SOURCES = \
-	src/settings/plugins/ifupdown/nms-ifupdown-connection.c \
-	src/settings/plugins/ifupdown/nms-ifupdown-connection.h \
 	src/settings/plugins/ifupdown/nms-ifupdown-plugin.c \
 	src/settings/plugins/ifupdown/nms-ifupdown-plugin.h \
 	$(NULL)
@@ -4834,6 +4837,7 @@ install-data-hook: $(install_data_hook)
 	$(mkinstalldirs) -m 0755 $(DESTDIR)$(nmconfdir)/dnsmasq-shared.d
 	$(mkinstalldirs) -m 0755 $(DESTDIR)$(nmlibdir)/conf.d
 	$(mkinstalldirs) -m 0755 $(DESTDIR)$(nmlibdir)/VPN
+	$(mkinstalldirs) -m 0755 $(DESTDIR)$(nmlibdir)/system-connections
 	$(mkinstalldirs) -m 0700 $(DESTDIR)$(nmstatedir)
 	$(mkinstalldirs) -m 0755 $(DESTDIR)$(plugindir)
 
diff --git a/contrib/fedora/rpm/NetworkManager.spec b/contrib/fedora/rpm/NetworkManager.spec
index 6f57d57468..255a84027c 100644
--- a/contrib/fedora/rpm/NetworkManager.spec
+++ b/contrib/fedora/rpm/NetworkManager.spec
@@ -847,6 +847,7 @@ fi
 %dir %{nmlibdir}
 %dir %{nmlibdir}/conf.d
 %dir %{nmlibdir}/VPN
+%dir %{nmlibdir}/system-connections
 %{_mandir}/man1/*
 %{_mandir}/man5/*
 %{_mandir}/man7/nmcli-examples.7*
diff --git a/introspection/org.freedesktop.NetworkManager.Settings.xml b/introspection/org.freedesktop.NetworkManager.Settings.xml
index 61940f912a..73da345ceb 100644
--- a/introspection/org.freedesktop.NetworkManager.Settings.xml
+++ b/introspection/org.freedesktop.NetworkManager.Settings.xml
@@ -56,9 +56,7 @@
         does not start the network connection unless (1) device is idle and able
         to connect to the network described by the new connection, and (2) the
         connection is allowed to be started automatically. Use the 'Save' method
-        on the connection to save these changes to disk. Note that unsaved changes
-        will be lost if the connection is reloaded from disk (either automatically
-        on file change or due to an explicit ReloadConnections call).
+        on the connection to save these changes to disk.
     -->
     <method name="AddConnectionUnsaved">
       <arg name="connection" type="a{sa{sv}}" direction="in"/>
diff --git a/libnm-core/nm-dbus-interface.h b/libnm-core/nm-dbus-interface.h
index 9950d67bfc..c68c7757b2 100644
--- a/libnm-core/nm-dbus-interface.h
+++ b/libnm-core/nm-dbus-interface.h
@@ -1014,6 +1014,9 @@ typedef enum { /*< flags >*/
  *   is not deleted but merely the connection is decoupled from the file
  *   on disk. If you later delete an in-memory connection, the connection
  *   on disk will be deleted as well.
+ *   Note: with 1.20, this flag is no longer implemented because in-memory connections
+ *   are also persisted under /run. For the moment, this behaves the same as
+ *   @NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED.
  * @NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED: this is like @NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY,
  *   but if the connection has a corresponding file on disk, the association between
  *   the connection and the file is forgotten but the file is not modified.
diff --git a/libnm-core/nm-keyfile-internal.h b/libnm-core/nm-keyfile-internal.h
index 47838bb639..e1fb10c2b4 100644
--- a/libnm-core/nm-keyfile-internal.h
+++ b/libnm-core/nm-keyfile-internal.h
@@ -171,8 +171,13 @@ gboolean _nm_keyfile_has_values (GKeyFile *keyfile);
 
 /*****************************************************************************/
 
+#define NM_KEYFILE_GROUP_NMMETA                 ".nmmeta"
+#define NM_KEYFILE_KEY_NMMETA_NM_GENERATED      "nm-generated"
+#define NM_KEYFILE_KEY_NMMETA_VOLATILE          "volatile"
+
+#define NM_KEYFILE_PATH_NAME_LIB                 NMLIBDIR  "/system-connections"
 #define NM_KEYFILE_PATH_NAME_ETC_DEFAULT         NMCONFDIR "/system-connections"
-#define NM_KEYFILE_PATH_NAME_RUN                 NMRUNDIR "/system-connections"
+#define NM_KEYFILE_PATH_NAME_RUN                 NMRUNDIR  "/system-connections"
 
 #define NM_KEYFILE_PATH_SUFFIX_NMCONNECTION      ".nmconnection"
 
diff --git a/libnm-core/nm-keyfile.c b/libnm-core/nm-keyfile.c
index 1d3f24cfd6..85a9ea11cb 100644
--- a/libnm-core/nm-keyfile.c
+++ b/libnm-core/nm-keyfile.c
@@ -3537,7 +3537,9 @@ nm_keyfile_read (GKeyFile *keyfile,
 			vpn_secrets = TRUE;
 		} else if (NM_STR_HAS_PREFIX (groups[i], NM_KEYFILE_GROUPPREFIX_WIREGUARD_PEER))
 			_read_setting_wireguard_peer (&info);
-		else
+		else if (nm_streq (groups[i], NM_KEYFILE_GROUP_NMMETA)) {
+			/* pass */
+		} else
 			_read_setting (&info);
 
 		info.group = NULL;
diff --git a/src/devices/bluetooth/nm-bluez-device.c b/src/devices/bluetooth/nm-bluez-device.c
index 5360992282..e8ca6dd159 100644
--- a/src/devices/bluetooth/nm-bluez-device.c
+++ b/src/devices/bluetooth/nm-bluez-device.c
@@ -235,15 +235,17 @@ pan_connection_check_create (NMBluezDevice *self)
 	 * which then already finds the suitable connection in priv->connections. This is confusing,
 	 * so block the signal. check_emit_usable will succeed after this function call returns. */
 	g_signal_handlers_block_by_func (priv->settings, cp_connection_added, self);
-	added = nm_settings_add_connection (priv->settings, connection, FALSE, &error);
+	nm_settings_add_connection (priv->settings,
+	                            connection,
+	                            NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
+	                            NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED,
+	                            &added,
+	                            &error);
 	g_signal_handlers_unblock_by_func (priv->settings, cp_connection_added, self);
 
 	if (added) {
 		nm_assert (!g_slist_find (priv->connections, added));
 		nm_assert (connection_compatible (self, added));
-
-		nm_settings_connection_set_flags (added, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED, TRUE);
-
 		priv->connections = g_slist_prepend (priv->connections, g_object_ref (added));
 		priv->pan_connection = added;
 		nm_log_dbg (LOGD_BT, "bluez[%s] added new Bluetooth connection for NAP device: '%s' (%s)", priv->path, id, uuid);
@@ -392,7 +394,7 @@ cp_connection_removed (NMSettings *settings,
 static void
 cp_connection_updated (NMSettings *settings,
                        NMSettingsConnection *sett_conn,
-                       gboolean by_user,
+                       guint update_reason_u,
                        NMBluezDevice *self)
 {
 	if (_internal_track_connection (self, sett_conn,
@@ -1225,7 +1227,7 @@ dispose (GObject *object)
 	if (to_delete) {
 		nm_log_dbg (LOGD_BT, "bluez[%s] removing Bluetooth connection for NAP device: '%s' (%s)", priv->path,
 		            nm_settings_connection_get_id (to_delete), nm_settings_connection_get_uuid (to_delete));
-		nm_settings_connection_delete (to_delete, NULL);
+		nm_settings_connection_delete (to_delete, FALSE);
 		g_object_unref (to_delete);
 	}
 
diff --git a/src/devices/nm-device.c b/src/devices/nm-device.c
index 59afa1dca5..04a94db8a1 100644
--- a/src/devices/nm-device.c
+++ b/src/devices/nm-device.c
@@ -12404,16 +12404,28 @@ nm_device_set_ip_config (NMDevice *self,
 
 		if (   nm_device_sys_iface_state_is_external (self)
 		    && (settings_connection = nm_device_get_settings_connection (self))
-		    && NM_FLAGS_HAS (nm_settings_connection_get_flags (settings_connection),
-		                     NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED)
+		    && NM_FLAGS_ALL (nm_settings_connection_get_flags (settings_connection),
+		                       NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED
+		                     | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
+		                     | NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED)
 		    && nm_active_connection_get_activation_type (NM_ACTIVE_CONNECTION (priv->act_request.obj)) == NM_ACTIVATION_TYPE_EXTERNAL) {
-			g_object_freeze_notify (G_OBJECT (settings_connection));
-			/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-			nm_connection_add_setting (nm_settings_connection_get_connection (settings_connection),
+			gs_unref_object NMConnection *new_connection = NULL;
+
+			new_connection = nm_simple_connection_new_clone (nm_settings_connection_get_connection (settings_connection));
+
+			nm_connection_add_setting (new_connection,
 			                           IS_IPv4
 			                             ? nm_ip4_config_create_setting (priv->ip_config_4)
 			                             : nm_ip6_config_create_setting (priv->ip_config_6));
-			g_object_thaw_notify (G_OBJECT (settings_connection));
+
+			nm_settings_connection_update (settings_connection,
+			                               new_connection,
+			                               NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED,
+			                               NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+			                               NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+			                               NM_SETTINGS_CONNECTION_UPDATE_REASON_NONE,
+			                               "update-external",
+			                               NULL);
 		}
 
 		nm_device_queue_recheck_assume (self);
@@ -14365,7 +14377,7 @@ cp_connection_added (NMSettings *settings, NMSettingsConnection *sett_conn, gpoi
 }
 
 static void
-cp_connection_updated (NMSettings *settings, NMSettingsConnection *sett_conn, gboolean by_user, gpointer user_data)
+cp_connection_updated (NMSettings *settings, NMSettingsConnection *sett_conn, guint update_reason_u, gpointer user_data)
 {
 	cp_connection_added_or_updated (user_data, sett_conn);
 }
diff --git a/src/devices/wifi/nm-device-wifi.c b/src/devices/wifi/nm-device-wifi.c
index aff631f889..a1fa96be9f 100644
--- a/src/devices/wifi/nm-device-wifi.c
+++ b/src/devices/wifi/nm-device-wifi.c
@@ -1760,9 +1760,10 @@ supplicant_iface_wps_credentials_cb (NMSupplicantInterface *iface,
 	}
 	if (secrets) {
 		if (nm_settings_connection_new_secrets (nm_act_request_get_settings_connection (req),
-		                                         nm_act_request_get_applied_connection (req),
-		                                         NM_SETTING_WIRELESS_SECURITY_SETTING_NAME,
-		                                         secrets, &error)) {
+		                                        nm_act_request_get_applied_connection (req),
+		                                        NM_SETTING_WIRELESS_SECURITY_SETTING_NAME,
+		                                        secrets,
+		                                        &error)) {
 			wifi_secrets_cancel (self);
 			nm_device_activate_schedule_stage1_device_prepare (NM_DEVICE (self));
 		} else {
diff --git a/src/devices/wifi/nm-iwd-manager.c b/src/devices/wifi/nm-iwd-manager.c
index a2057a51d6..8889bb204a 100644
--- a/src/devices/wifi/nm-iwd-manager.c
+++ b/src/devices/wifi/nm-iwd-manager.c
@@ -469,19 +469,18 @@ mirror_8021x_connection (NMIwdManager *self,
 	if (!nm_connection_normalize (connection, NULL, NULL, NULL))
 		return NULL;
 
-	settings_connection = nm_settings_add_connection (priv->settings, connection,
-	                                                  FALSE, &error);
-	if (!settings_connection) {
+	if (!nm_settings_add_connection (priv->settings,
+	                                 connection,
+	                                 NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
+	                                 NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED,
+	                                 &settings_connection,
+	                                 &error)) {
 		_LOGW ("failed to add a mirror NMConnection for IWD's Known Network '%s': %s",
 		       name, error->message);
 		g_error_free (error);
 		return NULL;
 	}
 
-	nm_settings_connection_set_flags (settings_connection,
-	                                  NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED |
-	                                  NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED,
-	                                  TRUE);
 	return settings_connection;
 }
 
@@ -498,7 +497,7 @@ mirror_8021x_connection_take_and_delete (NMSettingsConnection *sett_conn)
 	/* If connection has not been saved since we created it
 	 * in interface_added it too can be removed now. */
 	if (NM_FLAGS_HAS (flags, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED))
-		nm_settings_connection_delete (sett_conn, NULL);
+		nm_settings_connection_delete (sett_conn, FALSE);
 
 	g_object_unref (sett_conn);
 }
diff --git a/src/meson.build b/src/meson.build
index aec6eceb44..f3f5ee581a 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -103,7 +103,7 @@ sources = files(
   'dnsmasq/nm-dnsmasq-manager.c',
   'dnsmasq/nm-dnsmasq-utils.c',
   'ppp/nm-ppp-manager-call.c',
-  'settings/plugins/keyfile/nms-keyfile-connection.c',
+  'settings/plugins/keyfile/nms-keyfile-storage.c',
   'settings/plugins/keyfile/nms-keyfile-plugin.c',
   'settings/plugins/keyfile/nms-keyfile-reader.c',
   'settings/plugins/keyfile/nms-keyfile-utils.c',
@@ -113,6 +113,8 @@ sources = files(
   'settings/nm-settings.c',
   'settings/nm-settings-connection.c',
   'settings/nm-settings-plugin.c',
+  'settings/nm-settings-storage.c',
+  'settings/nm-settings-utils.c',
   'supplicant/nm-supplicant-config.c',
   'supplicant/nm-supplicant-interface.c',
   'supplicant/nm-supplicant-manager.c',
diff --git a/src/nm-active-connection.c b/src/nm-active-connection.c
index 06932b5f15..b937f7329e 100644
--- a/src/nm-active-connection.c
+++ b/src/nm-active-connection.c
@@ -181,7 +181,7 @@ NM_UTILS_FLAGS2STR_DEFINE_STATIC (_state_flags_to_string, NMActivationStateFlags
 
 static void
 _settings_connection_updated (NMSettingsConnection *sett_conn,
-                              gboolean by_user,
+                              guint update_reason_u,
                               gpointer user_data)
 {
 	NMActiveConnection *self = user_data;
@@ -522,8 +522,9 @@ nm_active_connection_clear_secrets (NMActiveConnection *self)
 	if (nm_settings_connection_has_unmodified_applied_connection (priv->settings_connection.obj,
 	                                                              priv->applied_connection,
 	                                                              NM_SETTING_COMPARE_FLAG_NONE)) {
-		/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-		nm_connection_clear_secrets (nm_settings_connection_get_connection (priv->settings_connection.obj));
+		nm_settings_connection_clear_secrets (priv->settings_connection.obj,
+		                                      FALSE,
+		                                      FALSE);
 	}
 	nm_connection_clear_secrets (priv->applied_connection);
 }
diff --git a/src/nm-checkpoint.c b/src/nm-checkpoint.c
index 000edee5b7..d2cc52e7af 100644
--- a/src/nm-checkpoint.c
+++ b/src/nm-checkpoint.c
@@ -217,19 +217,35 @@ restore_and_activate_connection (NMCheckpoint *self,
 	gs_unref_object NMAuthSubject *subject = NULL;
 	GError *local_error = NULL;
 	gboolean need_update, need_activation;
+	NMSettingsConnectionPersistMode persist_mode;
+	NMSettingsConnectionIntFlags sett_flags;
+	NMSettingsConnectionIntFlags sett_mask;
 
 	connection = find_settings_connection (self,
 	                                       dev_checkpoint,
 	                                       &need_update,
 	                                       &need_activation);
+
+	/* FIXME: we need to ensure to re-create/update the profile for the
+	 *   same settings plugin. E.g. if it was a keyfile in /run or /etc,
+	 *   it must be again. If it was previously handled by a certain settings plugin,
+	 *   so it must again.
+	 *
+	 * FIXME: preserve and restore the right settings flags (volatile, nm-generated). */
+	sett_flags = NM_SETTINGS_CONNECTION_INT_FLAGS_NONE;
+	sett_mask = NM_SETTINGS_CONNECTION_INT_FLAGS_NONE;
+
 	if (connection) {
 		if (need_update) {
 			_LOGD ("rollback: updating connection %s",
 			       nm_settings_connection_get_uuid (connection));
+			persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP;
 			nm_settings_connection_update (connection,
 			                               dev_checkpoint->settings_connection,
-			                               NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
-			                               NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
+			                               persist_mode,
+			                               sett_flags,
+			                               sett_mask,
+			                               NM_SETTINGS_CONNECTION_UPDATE_REASON_NONE,
 			                               "checkpoint-rollback",
 			                               NULL);
 		}
@@ -238,11 +254,13 @@ restore_and_activate_connection (NMCheckpoint *self,
 		_LOGD ("rollback: adding connection %s again",
 		       nm_connection_get_uuid (dev_checkpoint->settings_connection));
 
-		connection = nm_settings_add_connection (NM_SETTINGS_GET,
-		                                         dev_checkpoint->settings_connection,
-		                                         TRUE,
-		                                         &local_error);
-		if (!connection) {
+		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
+		if (!nm_settings_add_connection (NM_SETTINGS_GET,
+		                                 dev_checkpoint->settings_connection,
+		                                 persist_mode,
+		                                 sett_flags,
+		                                 &connection,
+		                                 &local_error)) {
 			_LOGD ("rollback: connection add failure: %s", local_error->message);
 			g_clear_error (&local_error);
 			return FALSE;
@@ -428,7 +446,7 @@ next_dev:
 			                            nm_settings_connection_get_uuid (con))) {
 				_LOGD ("rollback: deleting new connection %s",
 				       nm_settings_connection_get_uuid (con));
-				nm_settings_connection_delete (con, NULL);
+				nm_settings_connection_delete (con, FALSE);
 			}
 		}
 	}
diff --git a/src/nm-manager.c b/src/nm-manager.c
index 52433049fa..605e04e238 100644
--- a/src/nm-manager.c
+++ b/src/nm-manager.c
@@ -94,7 +94,8 @@ typedef struct {
 				struct {
 					GDBusMethodInvocation *invocation;
 					NMConnection *connection;
-					NMSettingsConnectionPersistMode persist;
+					NMSettingsConnectionPersistMode persist_mode;
+					bool is_volatile:1;
 				} add_and_activate;
 			};
 		} ac_auth;
@@ -378,7 +379,8 @@ static void _add_and_activate_auth_done (NMManager *self,
                                          NMActiveConnection *active,
                                          NMConnection *connection,
                                          GDBusMethodInvocation *invocation,
-                                         NMSettingsConnectionPersistMode persist,
+                                         NMSettingsConnectionPersistMode persist_mode,
+                                         gboolean is_volatile,
                                          gboolean success,
                                          const char *error_desc);
 static void _activation_auth_done (NMManager *self,
@@ -495,7 +497,8 @@ _async_op_data_new_ac_auth_add_and_activate (NMManager *self,
                                              NMActiveConnection *active_take,
                                              GDBusMethodInvocation *invocation_take,
                                              NMConnection *connection_take,
-                                             NMSettingsConnectionPersistMode persist)
+                                             NMSettingsConnectionPersistMode persist_mode,
+                                             gboolean is_volatile)
 {
 	AsyncOpData *async_op_data;
 
@@ -508,7 +511,8 @@ _async_op_data_new_ac_auth_add_and_activate (NMManager *self,
 	async_op_data->ac_auth.active = active_take;
 	async_op_data->ac_auth.add_and_activate.invocation = invocation_take;
 	async_op_data->ac_auth.add_and_activate.connection = connection_take;
-	async_op_data->ac_auth.add_and_activate.persist = persist;
+	async_op_data->ac_auth.add_and_activate.persist_mode = persist_mode;
+	async_op_data->ac_auth.add_and_activate.is_volatile = is_volatile;
 	c_list_link_tail (&NM_MANAGER_GET_PRIVATE (self)->async_op_lst_head, &async_op_data->async_op_lst);
 	return async_op_data;
 }
@@ -550,7 +554,8 @@ _async_op_complete_ac_auth_cb (NMActiveConnection *active,
 		                             async_op_data->ac_auth.active,
 		                             async_op_data->ac_auth.add_and_activate.connection,
 		                             async_op_data->ac_auth.add_and_activate.invocation,
-		                             async_op_data->ac_auth.add_and_activate.persist,
+		                             async_op_data->ac_auth.add_and_activate.persist_mode,
+		                             async_op_data->ac_auth.add_and_activate.is_volatile,
 		                             success,
 		                             error_desc);
 		g_object_unref (async_op_data->ac_auth.add_and_activate.connection);
@@ -831,7 +836,7 @@ _delete_volatile_connection_do (NMManager *self,
 
 	_LOGD (LOGD_DEVICE, "volatile connection disconnected. Deleting connection '%s' (%s)",
 	       nm_settings_connection_get_id (connection), nm_settings_connection_get_uuid (connection));
-	nm_settings_connection_delete (connection, NULL);
+	nm_settings_connection_delete (connection, FALSE);
 }
 
 /* Returns: whether to notify D-Bus of the removal or not */
@@ -2140,7 +2145,7 @@ connection_added_cb (NMSettings *settings,
 static void
 connection_updated_cb (NMSettings *settings,
                        NMSettingsConnection *sett_conn,
-                       gboolean by_user,
+                       guint update_reason_u,
                        NMManager *self)
 {
 	connection_changed_on_idle (self, sett_conn);
@@ -2660,8 +2665,13 @@ get_existing_connection (NMManager *self,
 
 	nm_device_assume_state_reset (device);
 
-	added = nm_settings_add_connection (priv->settings, connection, FALSE, &error);
-	if (!added) {
+	if (!nm_settings_add_connection (priv->settings,
+	                                 connection,
+	                                 NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
+	                                   NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
+	                                 | NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED,
+	                                 &added,
+	                                 &error)) {
 		_LOG2W (LOGD_SETTINGS, device, "assume: failure to save generated connection '%s': %s",
 		       nm_connection_get_id (connection),
 		       error->message);
@@ -2669,10 +2679,6 @@ get_existing_connection (NMManager *self,
 		return NULL;
 	}
 
-	nm_settings_connection_set_flags (added,
-	                                  NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED |
-	                                  NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE,
-	                                  TRUE);
 	NM_SET_OUT (out_generated, TRUE);
 	return added;
 }
@@ -2774,7 +2780,7 @@ recheck_assume_connection (NMManager *self,
 
 			if (generated) {
 				_LOG2D (LOGD_DEVICE, device, "assume: deleting generated connection after assuming failed");
-				nm_settings_connection_delete (sett_conn, NULL);
+				nm_settings_connection_delete (sett_conn, FALSE);
 			} else {
 				if (nm_device_sys_iface_state_get (device) == NM_DEVICE_SYS_IFACE_STATE_ASSUME)
 					nm_device_sys_iface_state_set (device, NM_DEVICE_SYS_IFACE_STATE_EXTERNAL);
@@ -5315,14 +5321,11 @@ activation_add_done (NMSettings *settings,
 	NMManager *self;
 	gs_unref_object NMActiveConnection *active = NULL;
 	gs_free_error GError *local = NULL;
-	gpointer persist_ptr;
-	NMSettingsConnectionPersistMode persist;
 	gpointer async_op_type_ptr;
 	AsyncOpType async_op_type;
 	GVariant *result_floating;
 
-	nm_utils_user_data_unpack (user_data, &self, &active, &persist_ptr, &async_op_type_ptr);
-	persist = GPOINTER_TO_INT (persist_ptr);
+	nm_utils_user_data_unpack (user_data, &self, &active, &async_op_type_ptr);
 	async_op_type = GPOINTER_TO_INT (async_op_type_ptr);
 
 	if (error)
@@ -5330,17 +5333,8 @@ activation_add_done (NMSettings *settings,
 
 	nm_active_connection_set_settings_connection (active, new_connection);
 
-	if (!_internal_activate_generic (self, active, &local)) {
-		error = local;
+	if (!_internal_activate_generic (self, active, &local))
 		goto fail;
-	}
-
-	nm_settings_connection_update (new_connection,
-	                               NULL,
-	                               persist,
-	                               NM_SETTINGS_CONNECTION_COMMIT_REASON_USER_ACTION | NM_SETTINGS_CONNECTION_COMMIT_REASON_ID_CHANGED,
-	                               "add-and-activate",
-	                               NULL);
 
 	if (async_op_type == ASYNC_OP_TYPE_AC_AUTH_ADD_AND_ACTIVATE) {
 		result_floating = g_variant_new ("(oo)",
@@ -5363,13 +5357,17 @@ activation_add_done (NMSettings *settings,
 	return;
 
 fail:
-	nm_assert (error);
+	if (local) {
+		nm_assert (!error);
+		error = local;
+	} else
+		nm_assert (error);
 
 	nm_active_connection_set_state_fail (active,
 	                                     NM_ACTIVE_CONNECTION_STATE_REASON_UNKNOWN,
 	                                     error->message);
 	if (new_connection)
-		nm_settings_connection_delete (new_connection, NULL);
+		nm_settings_connection_delete (new_connection, FALSE);
 	g_dbus_method_invocation_return_gerror (context, error);
 	nm_audit_log_connection_op (NM_AUDIT_OP_CONN_ADD_ACTIVATE,
 	                            NULL,
@@ -5385,7 +5383,8 @@ _add_and_activate_auth_done (NMManager *self,
                              NMActiveConnection *active,
                              NMConnection *connection,
                              GDBusMethodInvocation *invocation,
-                             NMSettingsConnectionPersistMode persist,
+                             NMSettingsConnectionPersistMode persist_mode,
+                             gboolean is_volatile,
                              gboolean success,
                              const char *error_desc)
 {
@@ -5413,13 +5412,15 @@ _add_and_activate_auth_done (NMManager *self,
 	 * shutdown. */
 	nm_settings_add_connection_dbus (priv->settings,
 	                                 connection,
-	                                 FALSE,
+	                                 persist_mode,
+	                                 (  is_volatile
+	                                  ? NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
+	                                  : NM_SETTINGS_CONNECTION_INT_FLAGS_NONE),
 	                                 nm_active_connection_get_subject (active),
 	                                 invocation,
 	                                 activation_add_done,
 	                                 nm_utils_user_data_pack (self,
 	                                                          g_object_ref (active),
-	                                                          GINT_TO_POINTER (persist),
 	                                                          GINT_TO_POINTER (async_op_type)));
 }
 
@@ -5445,7 +5446,8 @@ impl_manager_add_and_activate_connection (NMDBusObject *obj,
 	const char *device_path;
 	const char *specific_object_path;
 	gs_free NMConnection **conns = NULL;
-	NMSettingsConnectionPersistMode persist = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
+	NMSettingsConnectionPersistMode persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
+	gboolean is_volatile = FALSE;
 	gboolean bind_dbus_client = FALSE;
 	AsyncOpType async_op_type;
 
@@ -5474,13 +5476,17 @@ impl_manager_add_and_activate_connection (NMDBusObject *obj,
 			    && g_variant_is_of_type (option_value, G_VARIANT_TYPE_STRING)) {
 				s = g_variant_get_string (option_value, NULL);
 
-				if (nm_streq (s, "volatile"))
-					persist = NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_ONLY;
-				else if (nm_streq (s, "memory"))
-					persist = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY;
-				else if (nm_streq (s, "disk"))
-					persist = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
-				else {
+				is_volatile = FALSE;
+				persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
+
+				if (nm_streq (s, "volatile")) {
+					persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY;
+					is_volatile = TRUE;
+				} else if (nm_streq (s, "memory"))
+					persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY;
+				else if (nm_streq (s, "disk")) {
+					/* pass */
+				} else {
 					error = g_error_new_literal (NM_MANAGER_ERROR,
 					                             NM_MANAGER_ERROR_INVALID_ARGUMENTS,
 					                             "Option \"persist\" must be one of \"volatile\", \"memory\" or \"disk\"");
@@ -5599,7 +5605,8 @@ impl_manager_add_and_activate_connection (NMDBusObject *obj,
 	                                                                             active,
 	                                                                             invocation,
 	                                                                             incompl_conn,
-	                                                                             persist));
+	                                                                             persist_mode,
+	                                                                             is_volatile));
 
 	/* we passed the pointers on to _async_op_data_new_ac_auth_add_and_activate() */
 	g_steal_pointer (&incompl_conn);
@@ -6549,8 +6556,9 @@ nm_manager_start (NMManager *self, GError **error)
 	gs_free NMSettingsConnection **connections = NULL;
 	guint i;
 
-	if (!nm_settings_start (priv->settings, error))
-		return FALSE;
+	nm_device_factory_manager_load_factories (_register_device_factory, self);
+
+	nm_device_factory_manager_for_each_factory (start_factory, NULL);
 
 	/* Set initial radio enabled/disabled state */
 	for (i = 0; i < RFKILL_TYPE_MAX; i++) {
@@ -6573,16 +6581,15 @@ nm_manager_start (NMManager *self, GError **error)
 		manager_update_radio_enabled (self, rstate, enabled);
 	}
 
-	/* Log overall networking status - enabled/disabled */
 	_LOGI (LOGD_CORE, "Networking is %s by state file",
 	       priv->net_enabled ? "enabled" : "disabled");
 
 	system_unmanaged_devices_changed_cb (priv->settings, NULL, self);
+
 	hostname_changed_cb (priv->hostname_manager, NULL, self);
 
-	/* Start device factories */
-	nm_device_factory_manager_load_factories (_register_device_factory, self);
-	nm_device_factory_manager_for_each_factory (start_factory, NULL);
+	if (!nm_settings_start (priv->settings, error))
+		return FALSE;
 
 	nm_platform_process_events (priv->platform);
 
@@ -6596,10 +6603,11 @@ nm_manager_start (NMManager *self, GError **error)
 	/* Load VPN plugins */
 	priv->vpn_manager = g_object_ref (nm_vpn_manager_get ());
 
-	/* Connections added before the manager is started do not emit
-	 * connection-added signals thus devices have to be created manually.
-	 */
 	_LOGD (LOGD_CORE, "creating virtual devices...");
+	g_signal_connect (priv->settings, NM_SETTINGS_SIGNAL_CONNECTION_ADDED,
+	                  G_CALLBACK (connection_added_cb), self);
+	g_signal_connect (priv->settings, NM_SETTINGS_SIGNAL_CONNECTION_UPDATED,
+	                  G_CALLBACK (connection_updated_cb), self);
 	connections = nm_settings_get_connections_clone (priv->settings, NULL,
 	                                                 NULL, NULL,
 	                                                 nm_settings_connection_cmp_autoconnect_priority_p_with_data, NULL);
@@ -7334,10 +7342,6 @@ constructed (GObject *object)
 	                  G_CALLBACK (settings_startup_complete_changed), self);
 	g_signal_connect (priv->settings, "notify::" NM_SETTINGS_UNMANAGED_SPECS,
 	                  G_CALLBACK (system_unmanaged_devices_changed_cb), self);
-	g_signal_connect (priv->settings, NM_SETTINGS_SIGNAL_CONNECTION_ADDED,
-	                  G_CALLBACK (connection_added_cb), self);
-	g_signal_connect (priv->settings, NM_SETTINGS_SIGNAL_CONNECTION_UPDATED,
-	                  G_CALLBACK (connection_updated_cb), self);
 	g_signal_connect (priv->settings, NM_SETTINGS_SIGNAL_CONNECTION_FLAGS_CHANGED, G_CALLBACK (connection_flags_changed), self);
 
 	priv->hostname_manager = g_object_ref (nm_hostname_manager_get ());
diff --git a/src/nm-policy.c b/src/nm-policy.c
index affd541303..559babed3a 100644
--- a/src/nm-policy.c
+++ b/src/nm-policy.c
@@ -1846,8 +1846,7 @@ device_state_changed (NMDevice *device,
 				}
 			}
 
-			/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-			nm_connection_clear_secrets (nm_settings_connection_get_connection (sett_conn));
+			nm_settings_connection_clear_secrets (sett_conn, FALSE, FALSE);
 		}
 		break;
 	case NM_DEVICE_STATE_ACTIVATED:
@@ -1858,9 +1857,7 @@ device_state_changed (NMDevice *device,
 			/* And clear secrets so they will always be requested from the
 			 * settings service when the next connection is made.
 			 */
-
-			/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-			nm_connection_clear_secrets (nm_settings_connection_get_connection (sett_conn));
+			nm_settings_connection_clear_secrets (sett_conn, FALSE, FALSE);
 		}
 
 		/* Add device's new IPv4 and IPv6 configs to DNS */
@@ -2399,13 +2396,14 @@ dns_config_changed (NMDnsManager *dns_manager, gpointer user_data)
 static void
 connection_updated (NMSettings *settings,
                     NMSettingsConnection *connection,
-                    gboolean by_user,
+                    guint update_reason_u,
                     gpointer user_data)
 {
 	NMPolicyPrivate *priv = user_data;
 	NMPolicy *self = _PRIV_TO_SELF (priv);
+	NMSettingsConnectionUpdateReason update_reason = update_reason_u;
 
-	if (by_user) {
+	if (NM_FLAGS_HAS (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_REAPPLY_PARTIAL)) {
 		const CList *tmp_lst;
 		NMDevice *device;
 
@@ -2414,9 +2412,6 @@ connection_updated (NMSettings *settings,
 			if (nm_device_get_settings_connection (device) == connection)
 				nm_device_reapply_settings_immediately (device);
 		}
-
-		/* Reset auto retries back to default since connection was updated */
-		nm_settings_connection_autoconnect_retries_reset (connection);
 	}
 
 	schedule_activate_all (self);
diff --git a/src/settings/nm-settings-connection.c b/src/settings/nm-settings-connection.c
index 6368b3d5a0..06b1561b6b 100644
--- a/src/settings/nm-settings-connection.c
+++ b/src/settings/nm-settings-connection.c
@@ -37,6 +37,8 @@
 #include "NetworkManagerUtils.h"
 #include "nm-core-internal.h"
 #include "nm-audit-manager.h"
+#include "nm-settings.h"
+#include "settings/plugins/keyfile/nms-keyfile-storage.h"
 
 #define AUTOCONNECT_RETRIES_UNSET        -2
 #define AUTOCONNECT_RETRIES_FOREVER      -1
@@ -67,13 +69,11 @@ nm_settings_connections_array_to_connections (NMSettingsConnection *const*connec
 
 NM_GOBJECT_PROPERTIES_DEFINE (NMSettingsConnection,
 	PROP_UNSAVED,
-	PROP_READY,
 	PROP_FLAGS,
 	PROP_FILENAME,
 );
 
 enum {
-	REMOVED,
 	UPDATED_INTERNAL,
 	FLAGS_CHANGED,
 	LAST_SIGNAL
@@ -83,21 +83,12 @@ static guint signals[LAST_SIGNAL] = { 0 };
 
 typedef struct _NMSettingsConnectionPrivate {
 
+	NMSettings *settings;
+
 	NMKeyFileDB *kf_db_timestamps;
 	NMKeyFileDB *kf_db_seen_bssids;
 
 	NMAgentManager *agent_mgr;
-	NMSessionMonitor *session_monitor;
-	gulong session_changed_id;
-
-	NMSettingsConnectionIntFlags flags:5;
-
-	bool removed:1;
-	bool ready:1;
-
-	bool timestamp_set:1;
-
-	NMSettingsAutoconnectBlockedReason autoconnect_blocked_reason:4;
 
 	/* List of pending authentication requests */
 	CList auth_lst_head;
@@ -106,6 +97,12 @@ typedef struct _NMSettingsConnectionPrivate {
 
 	NMConnection *connection;
 
+	NMSettingsStorage *storage;
+
+	char *filename;
+
+	NMDevice *default_wired_device;
+
 	/* Caches secrets from on-disk connections; were they not cached any
 	 * call to nm_connection_clear_secrets() wipes them out and we'd have
 	 * to re-read them from disk which defeats the purpose of having the
@@ -121,8 +118,6 @@ typedef struct _NMSettingsConnectionPrivate {
 	 */
 	GVariant *agent_secrets;
 
-	char *filename;
-
 	GHashTable *seen_bssids; /* Up-to-date BSSIDs that's been seen for the connection */
 
 	guint64 timestamp;   /* Up-to-date timestamp of connection use */
@@ -130,10 +125,21 @@ typedef struct _NMSettingsConnectionPrivate {
 	guint64 last_secret_agent_version_id;
 
 	int autoconnect_retries;
+
 	gint32 autoconnect_retries_blocked_until;
 
+	bool timestamp_set:1;
+
+	NMSettingsAutoconnectBlockedReason autoconnect_blocked_reason:4;
+
+	NMSettingsConnectionIntFlags flags:5;
+
 } NMSettingsConnectionPrivate;
 
+struct _NMSettingsConnectionClass {
+	NMDBusObjectClass parent;
+};
+
 G_DEFINE_TYPE (NMSettingsConnection, nm_settings_connection, NM_TYPE_DBUS_OBJECT)
 
 #define NM_SETTINGS_CONNECTION_GET_PRIVATE(self) _NM_GET_PRIVATE_PTR (self, NMSettingsConnection, NM_IS_SETTINGS_CONNECTION)
@@ -167,6 +173,84 @@ static const GDBusSignalInfo signal_info_updated;
 static const GDBusSignalInfo signal_info_removed;
 static const NMDBusInterfaceInfoExtended interface_info_settings_connection;
 
+static void update_system_secrets_cache (NMSettingsConnection *self, NMConnection *new);
+static void update_agent_secrets_cache (NMSettingsConnection *self, NMConnection *new);
+
+/*****************************************************************************/
+
+NMDevice *
+nm_settings_connection_default_wired_get_device (NMSettingsConnection *self)
+{
+	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+
+	nm_assert (!priv->default_wired_device || NM_IS_DEVICE (priv->default_wired_device));
+
+	return priv->default_wired_device;
+}
+
+void
+nm_settings_connection_default_wired_set_device (NMSettingsConnection *self,
+                                                 NMDevice *device)
+{
+	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+
+	nm_assert (!priv->default_wired_device || NM_IS_DEVICE (priv->default_wired_device));
+	nm_assert (!device || NM_IS_DEVICE (device));
+
+	nm_assert ((!!priv->default_wired_device) != (!!device));
+
+	priv->default_wired_device = device;
+}
+
+/*****************************************************************************/
+
+NMSettingsStorage *
+nm_settings_connection_get_storage (NMSettingsConnection *self)
+{
+	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), NULL);
+
+	return NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->storage;
+}
+
+void
+_nm_settings_connection_set_storage (NMSettingsConnection *self,
+                                     NMSettingsStorage *storage)
+{
+	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+	const char *filename;
+
+	nm_assert (NM_IS_SETTINGS_STORAGE (storage));
+	nm_assert (   !priv->storage
+	           || nm_streq (nm_settings_storage_get_uuid (storage),
+	                        nm_settings_storage_get_uuid (priv->storage)));
+
+	nm_g_object_ref_set (&priv->storage, storage);
+
+	filename = nm_settings_storage_get_filename (priv->storage);
+
+	if (!nm_streq0 (priv->filename, filename)) {
+		g_free (priv->filename);
+		priv->filename = g_strdup (filename);
+		_notify (self, PROP_FILENAME);
+	}
+}
+
+/*****************************************************************************/
+
+gboolean
+nm_settings_connection_still_valid (NMSettingsConnection *self)
+{
+	gboolean valid;
+
+	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), FALSE);
+
+	valid = !c_list_is_empty (&self->_connections_lst);
+
+	nm_assert (valid == nm_settings_has_connection (NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->settings, self));
+
+	return valid;
+}
+
 /*****************************************************************************/
 
 static GHashTable *
@@ -185,6 +269,50 @@ nm_settings_connection_get_connection (NMSettingsConnection *self)
 	return NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->connection;
 }
 
+void
+_nm_settings_connection_set_connection (NMSettingsConnection *self,
+                                        NMConnection *new_connection,
+                                        NMConnection **out_connection_old,
+                                        NMSettingsConnectionUpdateReason update_reason)
+{
+	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+	gs_unref_object NMConnection *connection_old = NULL;
+
+	nm_assert (NM_IS_CONNECTION (new_connection));
+	nm_assert (NM_IS_SETTINGS_STORAGE (priv->storage));
+	nm_assert (nm_streq0 (nm_settings_storage_get_uuid (priv->storage), nm_connection_get_uuid (new_connection)));
+	nm_assert (!out_connection_old || !*out_connection_old);
+
+	if (   !priv->connection
+	    || !nm_connection_compare (priv->connection,
+	                               new_connection,
+	                               NM_SETTING_COMPARE_FLAG_EXACT)) {
+		connection_old = priv->connection;
+		priv->connection = g_object_ref (new_connection);
+		nmtst_connection_assert_unchanging (priv->connection);
+
+		/* note that we only return @connection_old if the new connection actually differs from
+		 * before.
+		 *
+		 * So, there are three cases:
+		 *
+		 *  - return %NULL when setting the connection the first time.
+		 *  - return %NULL if setting a profile with the same content that we already have.
+		 *  - return the previous pointer if the connection changed. */
+		NM_SET_OUT (out_connection_old, g_steal_pointer (&connection_old));
+	}
+
+	if (NM_FLAGS_HAS (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_SYSTEM_SECRETS))
+		update_system_secrets_cache (self, NULL);
+	else if (NM_FLAGS_HAS (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS))
+		update_system_secrets_cache (self, priv->connection);
+
+	if (NM_FLAGS_HAS (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_AGENT_SECRETS))
+		update_agent_secrets_cache (self, NULL);
+	else if (NM_FLAGS_HAS (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS))
+		update_agent_secrets_cache (self, priv->connection);
+}
+
 /*****************************************************************************/
 
 gboolean
@@ -214,34 +342,23 @@ nm_settings_connection_get_last_secret_agent_version_id (NMSettingsConnection *s
 
 /*****************************************************************************/
 
-static void
-set_visible (NMSettingsConnection *self, gboolean new_visible)
-{
-	nm_settings_connection_set_flags (self,
-	                                  NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE,
-	                                  new_visible);
-}
-
-void
-nm_settings_connection_recheck_visibility (NMSettingsConnection *self)
+gboolean
+nm_settings_connection_check_visibility (NMSettingsConnection *self,
+                                         NMSessionMonitor *session_monitor)
 {
-	NMSettingsConnectionPrivate *priv;
 	NMSettingConnection *s_con;
 	guint32 num, i;
 
-	g_return_if_fail (NM_IS_SETTINGS_CONNECTION (self));
+	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), FALSE);
 
-	priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+	nm_assert (NM_IS_SESSION_MONITOR (session_monitor));
 
 	s_con = nm_connection_get_setting_connection (nm_settings_connection_get_connection (self));
 
 	/* Check every user in the ACL for a session */
 	num = nm_setting_connection_get_num_permissions (s_con);
-	if (num == 0) {
-		/* Visible to all */
-		set_visible (self, TRUE);
-		return;
-	}
+	if (num == 0)
+		return TRUE;
 
 	for (i = 0; i < num; i++) {
 		const char *user;
@@ -251,20 +368,13 @@ nm_settings_connection_recheck_visibility (NMSettingsConnection *self)
 			continue;
 		if (!nm_session_monitor_user_to_uid (user, &uid))
 			continue;
-		if (!nm_session_monitor_session_exists (priv->session_monitor, uid, FALSE))
+		if (!nm_session_monitor_session_exists (session_monitor, uid, FALSE))
 			continue;
 
-		set_visible (self, TRUE);
-		return;
+		return TRUE;
 	}
 
-	set_visible (self, FALSE);
-}
-
-static void
-session_changed_cb (NMSessionMonitor *self, NMSettingsConnection *sett_conn)
-{
-	nm_settings_connection_recheck_visibility (sett_conn);
+	return FALSE;
 }
 
 /*****************************************************************************/
@@ -310,7 +420,8 @@ nm_settings_connection_check_permission (NMSettingsConnection *self,
 		if (nm_setting_connection_get_permission (s_con, i, NULL, &puser, NULL)) {
 			NMSecretAgent *agent = nm_agent_manager_get_agent_by_user (priv->agent_mgr, puser);
 
-			if (agent && nm_secret_agent_has_permission (agent, permission))
+			if (   agent
+			    && nm_secret_agent_has_permission (agent, permission))
 				return TRUE;
 		}
 	}
@@ -321,20 +432,37 @@ nm_settings_connection_check_permission (NMSettingsConnection *self,
 /*****************************************************************************/
 
 static void
-update_system_secrets_cache (NMSettingsConnection *self)
+update_system_secrets_cache (NMSettingsConnection *self, NMConnection *new)
 {
 	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
 	gs_unref_object NMConnection *connection_cloned = NULL;
+	gs_unref_variant GVariant *old_secrets = NULL;
 
-	nm_clear_pointer (&priv->system_secrets, g_variant_unref);
+	old_secrets = g_steal_pointer (&priv->system_secrets);
 
-	connection_cloned = nm_simple_connection_new_clone (nm_settings_connection_get_connection (self));
+	if (!new)
+		goto out;
+
+	/* FIXME: improve NMConnection API so we can avoid the overhead of cloning the connection,
+	 *   in particular if there are no secrets to begin with. */
+
+	connection_cloned = nm_simple_connection_new_clone (new);
 
 	/* Clear out non-system-owned and not-saved secrets */
 	_nm_connection_clear_secrets_by_secret_flags (connection_cloned,
 	                                              NM_SETTING_SECRET_FLAG_NONE);
 
 	priv->system_secrets = nm_g_variant_ref_sink (nm_connection_to_dbus (connection_cloned, NM_CONNECTION_SERIALIZE_ONLY_SECRETS));
+
+out:
+	if (_LOGT_ENABLED ()) {
+		if ((!!old_secrets) != (!!priv->system_secrets)) {
+			_LOGT ("update system secrets: secrets %s",
+			       old_secrets ? "cleared" : "set");
+		} else if (   priv->system_secrets
+		           && !g_variant_equal (old_secrets, priv->system_secrets))
+			_LOGT ("update system secrets: secrets updated");
+	}
 }
 
 static void
@@ -342,11 +470,17 @@ update_agent_secrets_cache (NMSettingsConnection *self, NMConnection *new)
 {
 	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
 	gs_unref_object NMConnection *connection_cloned = NULL;
+	gs_unref_variant GVariant *old_secrets = NULL;
 
-	nm_clear_pointer (&priv->agent_secrets, g_variant_unref);
+	old_secrets = g_steal_pointer (&priv->agent_secrets);
 
-	connection_cloned = nm_simple_connection_new_clone (   new
-	                                                    ?: nm_settings_connection_get_connection (self));
+	if (!new)
+		goto out;
+
+	/* FIXME: improve NMConnection API so we can avoid the overhead of cloning the connection,
+	 *   in particular if there are no secrets to begin with. */
+
+	connection_cloned = nm_simple_connection_new_clone (new);
 
 	/* Clear out non-system-owned secrets */
 	_nm_connection_clear_secrets_by_secret_flags (connection_cloned,
@@ -354,328 +488,139 @@ update_agent_secrets_cache (NMSettingsConnection *self, NMConnection *new)
 	                                              | NM_SETTING_SECRET_FLAG_AGENT_OWNED);
 
 	priv->agent_secrets = nm_g_variant_ref_sink (nm_connection_to_dbus (connection_cloned, NM_CONNECTION_SERIALIZE_ONLY_SECRETS));
-}
 
-static void
-secrets_cleared_cb (NMConnection *connection, NMSettingsConnection *self)
-{
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-
-	/* Clear agent secrets when connection's secrets are cleared since agent
-	 * secrets are transient.
-	 */
-	nm_clear_pointer (&priv->agent_secrets, g_variant_unref);
+out:
+	if (_LOGT_ENABLED ()) {
+		if ((!!old_secrets) != (!!priv->agent_secrets)) {
+			_LOGT ("update agent secrets: secrets %s",
+			       old_secrets ? "cleared" : "set");
+		} else if (   priv->agent_secrets
+		           && !g_variant_equal (old_secrets, priv->agent_secrets))
+			_LOGT ("update agent secrets: secrets updated");
+	}
 }
 
-static void
-set_persist_mode (NMSettingsConnection *self, NMSettingsConnectionPersistMode persist_mode)
+void
+nm_settings_connection_clear_secrets (NMSettingsConnection *self,
+                                      gboolean clear_cached_system_secrets,
+                                      gboolean persist)
 {
-	NMSettingsConnectionIntFlags flags = NM_SETTINGS_CONNECTION_INT_FLAGS_NONE;
-	const NMSettingsConnectionIntFlags ALL =   NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED
-	                                         | NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
-	                                         | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE;
+	gs_unref_object NMConnection *connection_cloned = NULL;
 
-	switch (persist_mode) {
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK:
-		flags = NM_SETTINGS_CONNECTION_INT_FLAGS_NONE;
-		break;
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY:
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED:
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY:
-		flags = NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED;
-		break;
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_DETACHED:
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_ONLY:
-		flags = NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED |
-		        NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE;
-		break;
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_UNSAVED:
-		/* only set the connection as unsaved, but preserve the nm-generated
-		 * and volatile flag. */
-		nm_settings_connection_set_flags (self,
-		                                  NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED,
-		                                  TRUE);
-		return;
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP:
-	case NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP_SAVED:
-		/* Nothing to do */
+	if (!nm_settings_connection_still_valid (self))
 		return;
-	}
 
-	nm_settings_connection_set_flags_full (self, ALL, flags);
-}
-
-static void
-_emit_updated (NMSettingsConnection *self, gboolean by_user)
-{
-	nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
-	                            &interface_info_settings_connection,
-	                            &signal_info_updated,
-	                            "()");
-	g_signal_emit (self, signals[UPDATED_INTERNAL], 0, by_user);
-}
+	/* FIXME: add API to NMConnection so that we can clone a profile without secrets. */
 
-static void
-connection_changed_cb (NMConnection *connection, NMSettingsConnection *self)
-{
-	set_persist_mode (self, NM_SETTINGS_CONNECTION_PERSIST_MODE_UNSAVED);
-	_emit_updated (self, FALSE);
-}
-
-static gboolean
-_delete (NMSettingsConnection *self, GError **error)
-{
-	NMSettingsConnectionClass *klass;
-	GError *local = NULL;
-	const char *filename;
+	connection_cloned = nm_simple_connection_new_clone (nm_settings_connection_get_connection (self));
 
-	nm_assert (NM_IS_SETTINGS_CONNECTION (self));
+	nm_connection_clear_secrets (connection_cloned);
 
-	klass = NM_SETTINGS_CONNECTION_GET_CLASS (self);
-	if (!klass->delete) {
-		g_set_error (&local,
-		             NM_SETTINGS_ERROR,
-		             NM_SETTINGS_ERROR_FAILED,
-		             "delete not supported");
-		goto fail;
-	}
-	if (!klass->delete (self,
-	                    &local))
-		goto fail;
-
-	filename = nm_settings_connection_get_filename (self);
-	if (filename) {
-		_LOGD ("delete: success deleting connection (\"%s\")", filename);
-		nm_settings_connection_set_filename (self, NULL);
-	} else
-		_LOGT ("delete: success deleting connection (no-file)");
-	return TRUE;
-fail:
-	_LOGD ("delete: failure deleting connection: %s", local->message);
-	g_propagate_error (error, local);
-	return FALSE;
+	if (!nm_settings_connection_update (self,
+	                                    connection_cloned,
+	                                       persist
+	                                    ? NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP
+	                                    : NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                      NM_SETTINGS_CONNECTION_UPDATE_REASON_IGNORE_PERSIST_FAILURE
+	                                    | (clear_cached_system_secrets ? NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_SYSTEM_SECRETS : NM_SETTINGS_CONNECTION_UPDATE_REASON_NONE)
+	                                    | NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_AGENT_SECRETS,
+	                                    "clear-secrets",
+	                                    NULL))
+		nm_assert_not_reached ();
 }
 
 static gboolean
-_update_prepare (NMSettingsConnection *self,
-                 NMConnection *new_connection,
+_secrets_update (NMConnection *connection,
+                 const char *setting_name,
+                 GVariant *secrets,
+                 NMConnection **out_new_connection,
                  GError **error)
 {
-	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), FALSE);
-	g_return_val_if_fail (NM_IS_CONNECTION (new_connection), FALSE);
+	gs_unref_variant GVariant *secrets_setting = NULL;
 
-	if (!nm_connection_normalize (new_connection, NULL, NULL, error))
-		return FALSE;
+	nm_assert (NM_IS_CONNECTION (connection));
 
-	if (   nm_dbus_object_get_path (NM_DBUS_OBJECT (self))
-	    && g_strcmp0 (nm_settings_connection_get_uuid (self), nm_connection_get_uuid (new_connection)) != 0) {
-		/* Updating the UUID is not allowed once the path is exported. */
-		g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-		             "connection %s cannot change the UUID from %s to %s", nm_settings_connection_get_id (self),
-		             nm_settings_connection_get_uuid (self), nm_connection_get_uuid (new_connection));
+	if (   setting_name
+	    && !nm_connection_get_setting_by_name (connection, setting_name)) {
+		g_set_error_literal (error,
+		                     NM_CONNECTION_ERROR,
+		                     NM_CONNECTION_ERROR_SETTING_NOT_FOUND,
+		                     setting_name);
 		return FALSE;
 	}
 
-	return TRUE;
-}
-
-gboolean
-nm_settings_connection_update (NMSettingsConnection *self,
-                               NMConnection *new_connection,
-                               NMSettingsConnectionPersistMode persist_mode,
-                               NMSettingsConnectionCommitReason commit_reason,
-                               const char *log_diff_name,
-                               GError **error)
-{
-	NMSettingsConnectionPrivate *priv;
-	NMSettingsConnectionClass *klass = NULL;
-	gs_unref_object NMConnection *reread_connection = NULL;
-	NMConnection *replace_connection;
-	gboolean replaced = FALSE;
-	gs_free char *logmsg_change = NULL;
-	GError *local = NULL;
-	gs_unref_variant GVariant *con_agent_secrets = NULL;
-	gs_unref_variant GVariant *new_agent_secrets = NULL;
-
-	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), FALSE);
-
-	priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-
-	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK) {
-		klass = NM_SETTINGS_CONNECTION_GET_CLASS (self);
-		if (!klass->commit_changes) {
-			g_set_error (&local,
-			             NM_SETTINGS_ERROR,
-			             NM_SETTINGS_ERROR_FAILED,
-			             "writing settings not supported");
-			goto out;
-		}
-	}
-
-	if (   new_connection
-	    && !_update_prepare (self,
-	                         new_connection,
-	                         &local))
-		goto out;
-
-	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK) {
-		if (!klass->commit_changes (self,
-		                            new_connection ?: nm_settings_connection_get_connection (self),
-		                            commit_reason,
-		                            &reread_connection,
-		                            &logmsg_change,
-		                            &local))
-			goto out;
-
-		if (   reread_connection
-		    && !_update_prepare (self,
-		                         reread_connection,
-		                         &local))
-			goto out;
-	}
-
-	replace_connection = reread_connection ?: new_connection;
-
-	/* Save agent-owned secrets from the new connection for later use */
-	if (new_connection) {
-		new_agent_secrets = nm_connection_to_dbus (new_connection,   NM_CONNECTION_SERIALIZE_ONLY_SECRETS
-		                                                           | NM_CONNECTION_SERIALIZE_WITH_SECRETS_AGENT_OWNED);
-	}
-
-	/* Disconnect the changed signal to ensure we don't set Unsaved when
-	 * it's not required.
-	 */
-	g_signal_handlers_block_by_func (priv->connection, G_CALLBACK (connection_changed_cb), self);
-
-	/* Do nothing if there's nothing to update */
-	if (   replace_connection
-	    && !nm_connection_compare (nm_settings_connection_get_connection (self),
-	                               replace_connection,
-	                               NM_SETTING_COMPARE_FLAG_EXACT)) {
-
-		if (log_diff_name) {
-			nm_utils_log_connection_diff (replace_connection, nm_settings_connection_get_connection (self), LOGL_DEBUG, LOGD_CORE, log_diff_name, "++ ",
-			                              nm_dbus_object_get_path (NM_DBUS_OBJECT (self)));
-		}
-
-		/* Make a copy of agent-owned secrets because they won't be present in
-		 * the connection returned by plugins, as plugins return only what was
-		 * reread from the file. */
-		con_agent_secrets = nm_connection_to_dbus (nm_settings_connection_get_connection (self),
-		                                             NM_CONNECTION_SERIALIZE_ONLY_SECRETS
-		                                           | NM_CONNECTION_SERIALIZE_WITH_SECRETS_AGENT_OWNED);
-
-		nm_connection_replace_settings_from_connection (nm_settings_connection_get_connection (self), replace_connection);
-
-		replaced = TRUE;
-	}
+	if (!secrets)
+		return TRUE;
 
-	nm_settings_connection_set_flags (self,
-	                                  NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE,
-	                                  FALSE);
+	nm_assert (   g_variant_is_of_type (secrets, NM_VARIANT_TYPE_SETTING)
+	           || g_variant_is_of_type (secrets, NM_VARIANT_TYPE_CONNECTION));
 
-	if (replaced) {
-		/* Cache the just-updated system secrets in case something calls
-		 * nm_connection_clear_secrets() and clears them.
-		 */
-		update_system_secrets_cache (self);
+	if (g_variant_n_children (secrets) == 0)
+		return TRUE;
 
-		/* Add agent and always-ask secrets back; they won't necessarily be
-		 * in the replacement connection data if it was eg reread from disk.
-		 */
-		if (priv->agent_secrets) {
-			/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-			nm_connection_update_secrets (nm_settings_connection_get_connection (self), NULL, priv->agent_secrets, NULL);
-		}
-		if (con_agent_secrets) {
-			/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-			nm_connection_update_secrets (nm_settings_connection_get_connection (self), NULL, con_agent_secrets, NULL);
+	if (   setting_name
+	    && g_variant_is_of_type (secrets, NM_VARIANT_TYPE_CONNECTION)) {
+		secrets_setting = g_variant_lookup_value (secrets, setting_name, NM_VARIANT_TYPE_SETTING);
+		if (!secrets_setting) {
+			/* The connection dictionary didn't contain any secrets for
+			 * @setting_name; just return success.
+			 */
+			return TRUE;
 		}
+		secrets = secrets_setting;
 	}
 
-	/* Apply agent-owned secrets from the new connection so that
-	 * they can be sent to agents */
-	if (new_agent_secrets) {
-		/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-		nm_connection_update_secrets (nm_settings_connection_get_connection (self),
-		                              NULL,
-		                              new_agent_secrets,
-		                              NULL);
+	/* if @out_new_connection is provided, we don't modify @connection but clone
+	 * and return it. Otherwise, we update @connection inplace. */
+	if (out_new_connection) {
+		nm_assert (!*out_new_connection);
+		connection = nm_simple_connection_new_clone (connection);
+		*out_new_connection = connection;
 	}
 
-	nm_settings_connection_recheck_visibility (self);
-
-	if (   replaced
-	    && persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP)
-		set_persist_mode (self, NM_SETTINGS_CONNECTION_PERSIST_MODE_UNSAVED);
-	else
-		set_persist_mode (self, persist_mode);
-
-	if (NM_IN_SET (persist_mode, NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
-	                             NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_ONLY))
-		_delete (self, NULL);
-	else if (NM_IN_SET (persist_mode, NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED,
-	                                  NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_DETACHED))
-		nm_settings_connection_set_filename (self, NULL);
-
-	g_signal_handlers_unblock_by_func (priv->connection, G_CALLBACK (connection_changed_cb), self);
-
-	_emit_updated (self, TRUE);
-
-out:
-	if (local) {
-		_LOGI ("write: failure to update connection: %s", local->message);
-		g_propagate_error (error, local);
+	if (!nm_connection_update_secrets (connection,
+	                                   setting_name,
+	                                   secrets,
+	                                   error))
 		return FALSE;
-	}
 
-	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK) {
-		if (reread_connection)
-			_LOGI ("write: successfully updated (%s), connection was modified in the process", logmsg_change);
-		else if (new_connection)
-			_LOGI ("write: successfully updated (%s)", logmsg_change);
-		else
-			_LOGI ("write: successfully committed (%s)", logmsg_change);
-	}
 	return TRUE;
 }
 
 gboolean
-nm_settings_connection_delete (NMSettingsConnection *self,
+nm_settings_connection_update (NMSettingsConnection *self,
+                               NMConnection *new_connection,
+                               NMSettingsConnectionPersistMode persist_mode,
+                               NMSettingsConnectionIntFlags sett_flags,
+                               NMSettingsConnectionIntFlags sett_mask,
+                               NMSettingsConnectionUpdateReason update_reason,
+                               const char *log_context_name,
                                GError **error)
 {
-	gs_unref_object NMSettingsConnection *self_keep_alive = NULL;
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-	NMConnection *for_agents;
-	const char *connection_uuid;
-
 	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), FALSE);
 
-	self_keep_alive = g_object_ref (self);
-
-	if (!_delete (self, error))
-		return FALSE;
-
-	set_visible (self, FALSE);
-
-	/* Tell agents to remove secrets for this connection */
-	for_agents = nm_simple_connection_new_clone (nm_settings_connection_get_connection (self));
-	nm_connection_clear_secrets (for_agents);
-	nm_agent_manager_delete_secrets (priv->agent_mgr,
-	                                 nm_dbus_object_get_path (NM_DBUS_OBJECT (self)),
-	                                 for_agents);
-	g_object_unref (for_agents);
-
-	connection_uuid = nm_settings_connection_get_uuid (self);
-
-	if (priv->kf_db_timestamps)
-		nm_key_file_db_remove_key (priv->kf_db_timestamps, connection_uuid);
+	return nm_settings_update_connection (NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->settings,
+	                                      self,
+	                                      new_connection,
+	                                      persist_mode,
+	                                      sett_flags,
+	                                      sett_mask,
+	                                      update_reason,
+	                                      log_context_name,
+	                                      error);
+}
 
-	if (priv->kf_db_seen_bssids)
-		nm_key_file_db_remove_key (priv->kf_db_seen_bssids, connection_uuid);
+void
+nm_settings_connection_delete (NMSettingsConnection *self,
+                               gboolean allow_add_to_no_auto_default)
+{
+	g_return_if_fail (NM_IS_SETTINGS_CONNECTION (self));
 
-	nm_settings_connection_signal_remove (self);
-	return TRUE;
+	nm_settings_delete_connection (NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->settings,
+	                               self,
+	                               allow_add_to_no_auto_default);
 }
 
 /*****************************************************************************/
@@ -860,26 +805,37 @@ nm_settings_connection_new_secrets (NMSettingsConnection *self,
                                     GVariant *secrets,
                                     GError **error)
 {
-	if (!nm_settings_connection_has_unmodified_applied_connection (self, applied_connection,
-	                                                              NM_SETTING_COMPARE_FLAG_NONE)) {
+	gs_unref_object NMConnection *new_connection = NULL;
+	NMConnection *connection;
+
+	if (!nm_settings_connection_has_unmodified_applied_connection (self,
+	                                                               applied_connection,
+	                                                               NM_SETTING_COMPARE_FLAG_NONE)) {
 		g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
 		                     "The connection was modified since activation");
 		return FALSE;
 	}
 
-	/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-	if (!nm_connection_update_secrets (nm_settings_connection_get_connection (self), setting_name, secrets, error))
-		return FALSE;
+	connection = nm_settings_connection_get_connection (self);
 
-	update_system_secrets_cache (self);
-	update_agent_secrets_cache (self, NULL);
+	if (!_secrets_update (connection,
+	                      setting_name,
+	                      secrets,
+	                      &new_connection,
+	                      error))
+		return FALSE;
 
-	nm_settings_connection_update (self,
-	                               NULL,
-	                               NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
-	                               NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-	                               "new-secrets",
-	                               NULL);
+	if (!nm_settings_connection_update (self,
+	                                    new_connection ?: connection,
+	                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                      NM_SETTINGS_CONNECTION_UPDATE_REASON_IGNORE_PERSIST_FAILURE
+	                                    | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS
+	                                    | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS,
+	                                    "new-secrets",
+	                                    NULL))
+		nm_assert_not_reached ();
 	return TRUE;
 }
 
@@ -901,8 +857,10 @@ get_secrets_done_cb (NMAgentManager *manager,
 	NMConnection *applied_connection;
 	gs_free_error GError *local = NULL;
 	gs_unref_variant GVariant *system_secrets = NULL;
+	gs_unref_object NMConnection *new_connection = NULL;
 	gboolean agent_had_system = FALSE;
 	ForEachSecretFlags cmp_flags = { NM_SETTING_SECRET_FLAG_NONE, NM_SETTING_SECRET_FLAG_NONE };
+	gs_unref_variant GVariant *filtered_secrets = NULL;
 
 	if (g_error_matches (error, G_IO_ERROR, G_IO_ERROR_CANCELLED))
 		return;
@@ -965,66 +923,67 @@ get_secrets_done_cb (NMAgentManager *manager,
 
 	system_secrets = nm_g_variant_ref (priv->system_secrets);
 
-	/* Update the connection with our existing secrets from backing storage */
-	nm_connection_clear_secrets (nm_settings_connection_get_connection (self));
-
-	if (   !system_secrets
-	       /* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-	    || nm_connection_update_secrets (nm_settings_connection_get_connection (self),
-	                                     setting_name,
-	                                     system_secrets,
-	                                     &local)) {
-		gs_unref_variant GVariant *filtered_secrets = NULL;
-
-		/* Update the connection with the agent's secrets; by this point if any
-		 * system-owned secrets exist in 'secrets' the agent that provided them
-		 * will have been authenticated, so those secrets can replace the existing
-		 * system secrets.
-		 */
-		filtered_secrets = validate_secret_flags (nm_settings_connection_get_connection (self), secrets, &cmp_flags);
-		/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-		if (nm_connection_update_secrets (nm_settings_connection_get_connection (self), setting_name, filtered_secrets, &local)) {
-			/* Now that all secrets are updated, copy and cache new secrets,
-			 * then save them to backing storage.
-			 */
-			update_system_secrets_cache (self);
-			update_agent_secrets_cache (self, NULL);
+	new_connection = nm_simple_connection_new_clone (nm_settings_connection_get_connection (self));
 
-			/* Only save secrets to backing storage if the agent returned any
-			 * new system secrets.  If it didn't, then the secrets are agent-
-			 * owned and there's no point to writing out the connection when
-			 * nothing has changed, since agent-owned secrets don't get saved here.
-			 */
-			if (agent_had_system) {
-				_LOGD ("(%s:%p) saving new secrets to backing storage",
-				       setting_name,
-				       call_id);
-
-				nm_settings_connection_update (self,
-				                               NULL,
-				                               NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
-				                               NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-				                               "get-new-secrets",
-				                               NULL);
-			} else {
-				_LOGD ("(%s:%p) new agent secrets processed",
-				       setting_name,
-				       call_id);
-			}
+	nm_connection_clear_secrets (new_connection);
 
-		} else {
-			_LOGD ("(%s:%p) failed to update with agent secrets: %s",
-			       setting_name,
-			       call_id,
-			       local->message);
-		}
-	} else {
+	if (!_secrets_update (new_connection,
+	                      setting_name,
+	                      system_secrets,
+	                      NULL,
+	                      &local)) {
 		_LOGD ("(%s:%p) failed to update with existing secrets: %s",
 		       setting_name,
 		       call_id,
 		       local->message);
 	}
 
+	/* Update the connection with the agent's secrets; by this point if any
+	 * system-owned secrets exist in 'secrets' the agent that provided them
+	 * will have been authenticated, so those secrets can replace the existing
+	 * system secrets.
+	 */
+	filtered_secrets = validate_secret_flags (new_connection, secrets, &cmp_flags);
+
+	if (!_secrets_update (new_connection,
+	                      setting_name,
+	                      filtered_secrets,
+	                      NULL,
+	                      &local)) {
+		_LOGD ("(%s:%p) failed to update with agent secrets: %s",
+		       setting_name,
+		       call_id,
+		       local->message);
+	}
+
+	/* Only save secrets to backing storage if the agent returned any
+	 * new system secrets.  If it didn't, then the secrets are agent-
+	 * owned and there's no point to writing out the connection when
+	 * nothing has changed, since agent-owned secrets don't get saved here.
+	 */
+	if (agent_had_system) {
+		_LOGD ("(%s:%p) saving new secrets to backing storage",
+		       setting_name,
+		       call_id);
+	} else {
+		_LOGD ("(%s:%p) new agent secrets processed",
+		       setting_name,
+		       call_id);
+	}
+	if (!nm_settings_connection_update (self,
+	                                    new_connection,
+	                                      agent_had_system
+	                                    ? NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP
+	                                    : NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                      NM_SETTINGS_CONNECTION_UPDATE_REASON_IGNORE_PERSIST_FAILURE
+	                                    | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS
+	                                    | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS,
+	                                    "get-new-secrets",
+	                                    NULL))
+		nm_assert_not_reached ();
+
 	applied_connection = call_id->applied_connection;
 	if (applied_connection) {
 		get_cmp_flags (self,
@@ -1042,10 +1001,10 @@ get_secrets_done_cb (NMAgentManager *manager,
 
 		if (   !system_secrets
 		    || nm_connection_update_secrets (applied_connection, setting_name, system_secrets, NULL)) {
-			gs_unref_variant GVariant *filtered_secrets = NULL;
+			gs_unref_variant GVariant *filtered_secrets2 = NULL;
 
-			filtered_secrets = validate_secret_flags (applied_connection, secrets, &cmp_flags);
-			nm_connection_update_secrets (applied_connection, setting_name, filtered_secrets, NULL);
+			filtered_secrets2 = validate_secret_flags (applied_connection, secrets, &cmp_flags);
+			nm_connection_update_secrets (applied_connection, setting_name, filtered_secrets2, NULL);
 		}
 	}
 
@@ -1502,10 +1461,8 @@ update_auth_cb (NMSettingsConnection *self,
 {
 	NMSettingsConnectionPrivate *priv;
 	UpdateInfo *info = data;
-	NMSettingsConnectionCommitReason commit_reason;
 	gs_free_error GError *local = NULL;
 	NMSettingsConnectionPersistMode persist_mode;
-	const char *log_diff_name;
 
 	if (error) {
 		update_complete (self, info, error);
@@ -1547,34 +1504,16 @@ update_auth_cb (NMSettingsConnection *self,
 		}
 	}
 
-	commit_reason = NM_SETTINGS_CONNECTION_COMMIT_REASON_USER_ACTION;
-	if (   info->new_settings
-	    && !nm_streq0 (nm_connection_get_id (nm_settings_connection_get_connection (self)),
-	                   nm_connection_get_id (info->new_settings)))
-		commit_reason |= NM_SETTINGS_CONNECTION_COMMIT_REASON_ID_CHANGED;
-
 	if (NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_TO_DISK))
 		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
-	else if (NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY))
-		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY;
-	else if (NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED)) {
-		persist_mode = NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_VOLATILE)
-		               ? NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_DETACHED
-		               : NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED;
-	} else if (NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_ONLY)) {
-		persist_mode = NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_VOLATILE)
-		               ? NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_ONLY
-		               : NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY;
+	else if (NM_FLAGS_ANY (info->flags,   NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY
+	                                      | NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED))
+		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED;
+	else if (NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_ONLY)) {
+		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY;
 	} else
 		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP;
 
-	if (   persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK
-	    || (   persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP
-	        && !nm_settings_connection_get_unsaved (self)))
-		log_diff_name = info->new_settings ? "update-settings" : "write-out-to-disk";
-	else
-		log_diff_name = info->new_settings ? "update-unsaved" : "make-unsaved";
-
 	if (NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_BLOCK_AUTOCONNECT)) {
 		nm_settings_connection_autoconnect_blocked_reason_set (self,
 		                                                       NM_SETTINGS_AUTO_CONNECT_BLOCKED_REASON_USER_REQUEST,
@@ -1584,8 +1523,16 @@ update_auth_cb (NMSettingsConnection *self,
 	nm_settings_connection_update (self,
 	                               info->new_settings,
 	                               persist_mode,
-	                               commit_reason,
-	                               log_diff_name,
+	                               (  NM_FLAGS_HAS (info->flags, NM_SETTINGS_UPDATE2_FLAG_VOLATILE)
+	                                ? NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
+	                                : NM_SETTINGS_CONNECTION_INT_FLAGS_NONE),
+	                                 NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+	                               | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE,
+	                                 NM_SETTINGS_CONNECTION_UPDATE_REASON_FORCE_RENAME
+	                               | NM_SETTINGS_CONNECTION_UPDATE_REASON_REAPPLY_PARTIAL
+	                               | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS
+	                               | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS,
+	                               "update-from-dbus",
 	                               &local);
 
 	if (!local) {
@@ -1604,6 +1551,9 @@ update_auth_cb (NMSettingsConnection *self,
 		                               info->subject);
 	}
 
+	/* Reset auto retries back to default since connection was updated */
+	nm_settings_connection_autoconnect_retries_reset (self);
+
 	update_complete (self, info, local);
 }
 
@@ -1839,7 +1789,6 @@ delete_auth_cb (NMSettingsConnection *self,
                 gpointer data)
 {
 	gs_unref_object NMSettingsConnection *self_keep_alive = NULL;
-	gs_free_error GError *local = NULL;
 
 	self_keep_alive = g_object_ref (self);
 
@@ -1850,15 +1799,11 @@ delete_auth_cb (NMSettingsConnection *self,
 		return;
 	}
 
-	nm_settings_connection_delete (self, &local);
+	nm_settings_connection_delete (self, TRUE);
 
 	nm_audit_log_connection_op (NM_AUDIT_OP_CONN_DELETE, self,
-	                            !local, NULL, subject, local ? local->message : NULL);
-
-	if (local)
-		g_dbus_method_invocation_return_gerror (context, local);
-	else
-		g_dbus_method_invocation_return_value (context, NULL);
+	                            TRUE, NULL, subject, NULL);
+	g_dbus_method_invocation_return_value (context, NULL);
 }
 
 static const char *
@@ -1890,6 +1835,8 @@ impl_settings_connection_delete (NMDBusObject *obj,
 	gs_unref_object NMAuthSubject *subject = NULL;
 	GError *error = NULL;
 
+	nm_assert (nm_settings_connection_still_valid (self));
+
 	if (!check_writable (nm_settings_connection_get_connection (self), &error))
 		goto err;
 
@@ -2006,26 +1953,13 @@ dbus_clear_secrets_auth_cb (NMSettingsConnection *self,
 		return;
 	}
 
-	/* Clear secrets in connection and caches */
-
-	/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-	nm_connection_clear_secrets (nm_settings_connection_get_connection (self));
-
-	nm_clear_pointer (&priv->system_secrets, g_variant_unref);
-	nm_clear_pointer (&priv->agent_secrets, g_variant_unref);
+	nm_settings_connection_clear_secrets (self, TRUE, TRUE);
 
 	/* Tell agents to remove secrets for this connection */
 	nm_agent_manager_delete_secrets (priv->agent_mgr,
 	                                 nm_dbus_object_get_path (NM_DBUS_OBJECT (self)),
 	                                 nm_settings_connection_get_connection (self));
 
-	nm_settings_connection_update (self,
-	                               NULL,
-	                               NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
-	                               NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-	                               "clear-secrets",
-	                               &local);
-
 	nm_audit_log_connection_op (NM_AUDIT_OP_CONN_CLEAR_SECRETS, self,
 	                            !local, NULL, subject, local ? local->message : NULL);
 
@@ -2066,40 +2000,28 @@ impl_settings_connection_clear_secrets (NMDBusObject *obj,
 /*****************************************************************************/
 
 void
-nm_settings_connection_added (NMSettingsConnection *self)
+_nm_settings_connection_emit_dbus_signal_updated (NMSettingsConnection *self)
 {
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-
-	/* FIXME: we should always dispose connections that are removed
-	 * and not reuse them, but currently plugins keep alive unmanaged
-	 * (e.g. NM_CONTROLLED=no) connections. */
-	priv->removed = FALSE;
+	nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
+	                            &interface_info_settings_connection,
+	                            &signal_info_updated,
+	                            "()");
 }
 
 void
-nm_settings_connection_signal_remove (NMSettingsConnection *self)
+_nm_settings_connection_emit_dbus_signal_removed (NMSettingsConnection *self)
 {
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-	AuthData *auth_data;
-
-	if (priv->removed)
-		return;
-	priv->removed = TRUE;
-
-	while ((auth_data = c_list_first_entry (&priv->auth_lst_head, AuthData, auth_lst)))
-		nm_auth_manager_check_authorization_cancel (auth_data->call_id);
-
 	nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
 	                            &interface_info_settings_connection,
 	                            &signal_info_removed,
 	                            "()");
-	g_signal_emit (self, signals[REMOVED], 0);
 }
 
-gboolean
-nm_settings_connection_get_unsaved (NMSettingsConnection *self)
+void
+_nm_settings_connection_emit_signal_updated_internal (NMSettingsConnection *self,
+                                                      NMSettingsConnectionUpdateReason update_reason)
 {
-	return NM_FLAGS_HAS (nm_settings_connection_get_flags (self), NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED);
+	g_signal_emit (self, signals[UPDATED_INTERNAL], 0, (guint) update_reason);
 }
 
 /*****************************************************************************/
@@ -2121,14 +2043,6 @@ nm_settings_connection_get_flags (NMSettingsConnection *self)
 }
 
 NMSettingsConnectionIntFlags
-nm_settings_connection_set_flags (NMSettingsConnection *self, NMSettingsConnectionIntFlags flags, gboolean set)
-{
-	return nm_settings_connection_set_flags_full (self,
-	                                              flags,
-	                                              set ? flags : NM_SETTINGS_CONNECTION_INT_FLAGS_NONE);
-}
-
-NMSettingsConnectionIntFlags
 nm_settings_connection_set_flags_full (NMSettingsConnection *self,
                                        NMSettingsConnectionIntFlags mask,
                                        NMSettingsConnectionIntFlags value)
@@ -2137,7 +2051,8 @@ nm_settings_connection_set_flags_full (NMSettingsConnection *self,
 	NMSettingsConnectionIntFlags old_flags;
 
 	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), NM_SETTINGS_CONNECTION_INT_FLAGS_NONE);
-	nm_assert (mask && !NM_FLAGS_ANY (mask, ~NM_SETTINGS_CONNECTION_INT_FLAGS_ALL));
+
+	nm_assert (!NM_FLAGS_ANY (mask, ~_NM_SETTINGS_CONNECTION_INT_FLAGS_ALL));
 	nm_assert (!NM_FLAGS_ANY (value, ~mask));
 
 	priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
@@ -2298,17 +2213,10 @@ nm_settings_connection_update_timestamp (NMSettingsConnection *self,
 	}
 }
 
-/**
- * nm_settings_connection_read_and_fill_timestamp:
- * @self: the #NMSettingsConnection
- *
- * Retrieves timestamp of the connection's last usage from database file and
- * stores it into the connection private data.
- **/
 void
-nm_settings_connection_register_kf_dbs (NMSettingsConnection *self,
-                                        NMKeyFileDB *kf_db_timestamps,
-                                        NMKeyFileDB *kf_db_seen_bssids)
+_nm_settings_connection_register_kf_dbs (NMSettingsConnection *self,
+                                         NMKeyFileDB *kf_db_timestamps,
+                                         NMKeyFileDB *kf_db_seen_bssids)
 {
 	NMSettingsConnectionPrivate *priv;
 	const char *connection_uuid;
@@ -2625,46 +2533,6 @@ nm_settings_connection_autoconnect_is_blocked (NMSettingsConnection *self)
 
 /*****************************************************************************/
 
-gboolean
-nm_settings_connection_get_ready (NMSettingsConnection *self)
-{
-	return NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->ready;
-}
-
-void
-nm_settings_connection_set_ready (NMSettingsConnection *self,
-                                  gboolean ready)
-{
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-
-	ready = !!ready;
-	if (priv->ready != ready) {
-		priv->ready = ready;
-		_notify (self, PROP_READY);
-	}
-}
-
-/**
- * nm_settings_connection_set_filename:
- * @self: an #NMSettingsConnection
- * @filename: @self's filename
- *
- * Called by a backend to sets the filename that @self is read
- * from/written to.
- */
-void
-nm_settings_connection_set_filename (NMSettingsConnection *self,
-                                     const char *filename)
-{
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
-
-	if (g_strcmp0 (filename, priv->filename) != 0) {
-		g_free (priv->filename);
-		priv->filename = g_strdup (filename);
-		_notify (self, PROP_FILENAME);
-	}
-}
-
 /**
  * nm_settings_connection_get_filename:
  * @self: an #NMSettingsConnection
@@ -2678,9 +2546,9 @@ nm_settings_connection_set_filename (NMSettingsConnection *self,
 const char *
 nm_settings_connection_get_filename (NMSettingsConnection *self)
 {
-	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), NULL);
 
-	return priv->filename;
+	return NM_SETTINGS_CONNECTION_GET_PRIVATE (self)->filename;
 }
 
 const char *
@@ -2692,7 +2560,18 @@ nm_settings_connection_get_id (NMSettingsConnection *self)
 const char *
 nm_settings_connection_get_uuid (NMSettingsConnection *self)
 {
-	return nm_connection_get_uuid (nm_settings_connection_get_connection (self));
+	NMSettingsConnectionPrivate *priv;
+	const char *uuid;
+
+	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (self), NULL);
+
+	priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+
+	uuid = nm_settings_storage_get_uuid (priv->storage);
+
+	nm_assert (uuid && nm_streq0 (uuid, nm_connection_get_uuid (nm_settings_connection_get_connection (self))));
+
+	return uuid;
 }
 
 const char *
@@ -2703,6 +2582,43 @@ nm_settings_connection_get_connection_type (NMSettingsConnection *self)
 
 /*****************************************************************************/
 
+void
+_nm_settings_connection_cleanup_after_remove (NMSettingsConnection *self)
+{
+	NMSettingsConnectionPrivate *priv = NM_SETTINGS_CONNECTION_GET_PRIVATE (self);
+	AuthData *auth_data;
+
+	while ((auth_data = c_list_first_entry (&priv->auth_lst_head, AuthData, auth_lst)))
+		nm_auth_manager_check_authorization_cancel (auth_data->call_id);
+}
+
+/*****************************************************************************/
+
+static void
+get_property (GObject *object, guint prop_id,
+              GValue *value, GParamSpec *pspec)
+{
+	NMSettingsConnection *self = NM_SETTINGS_CONNECTION (object);
+
+	switch (prop_id) {
+	case PROP_UNSAVED:
+		g_value_set_boolean (value, nm_settings_connection_get_unsaved (self));
+		break;
+	case PROP_FLAGS:
+		g_value_set_uint (value,
+		                  nm_settings_connection_get_flags (self) & _NM_SETTINGS_CONNECTION_INT_FLAGS_EXPORTED_MASK);
+		break;
+	case PROP_FILENAME:
+		g_value_set_string (value, nm_settings_connection_get_filename (self));
+		break;
+	default:
+		G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+		break;
+	}
+}
+
+/*****************************************************************************/
+
 static void
 nm_settings_connection_init (NMSettingsConnection *self)
 {
@@ -2713,33 +2629,19 @@ nm_settings_connection_init (NMSettingsConnection *self)
 
 	c_list_init (&self->_connections_lst);
 
-	priv->ready = TRUE;
 	c_list_init (&priv->call_ids_lst_head);
 	c_list_init (&priv->auth_lst_head);
 
-	priv->session_monitor = g_object_ref (nm_session_monitor_get ());
-	priv->session_changed_id = g_signal_connect (priv->session_monitor,
-	                                             NM_SESSION_MONITOR_CHANGED,
-	                                             G_CALLBACK (session_changed_cb), self);
-
 	priv->agent_mgr = g_object_ref (nm_agent_manager_get ());
+	priv->settings = g_object_ref (nm_settings_get ());
 
 	priv->autoconnect_retries = AUTOCONNECT_RETRIES_UNSET;
-
-	priv->connection = nm_simple_connection_new ();
-
-	g_signal_connect (priv->connection, NM_CONNECTION_SECRETS_CLEARED, G_CALLBACK (secrets_cleared_cb), self);
-	g_signal_connect (priv->connection, NM_CONNECTION_CHANGED, G_CALLBACK (connection_changed_cb), self);
 }
 
-static void
-constructed (GObject *object)
+NMSettingsConnection *
+nm_settings_connection_new (void)
 {
-	NMSettingsConnection *self = NM_SETTINGS_CONNECTION (object);
-
-	_LOGD ("constructed (%s)", G_OBJECT_TYPE_NAME (self));
-
-	G_OBJECT_CLASS (nm_settings_connection_parent_class)->constructed (object);
+	return g_object_new (NM_TYPE_SETTINGS_CONNECTION, NULL);
 }
 
 static void
@@ -2751,6 +2653,8 @@ dispose (GObject *object)
 
 	_LOGD ("disposing");
 
+	nm_assert (!priv->default_wired_device);
+
 	nm_assert (c_list_is_empty (&self->_connections_lst));
 	nm_assert (c_list_is_empty (&priv->auth_lst_head));
 
@@ -2760,83 +2664,29 @@ dispose (GObject *object)
 			_get_secrets_cancel (self, call_id, TRUE);
 	}
 
-	set_visible (self, FALSE);
-
-	if (priv->connection) {
-		/* Disconnect handlers.
-		 * connection_changed_cb() has to be disconnected *before* nm_connection_clear_secrets(),
-		 * because nm_connection_clear_secrets() emits NM_CONNECTION_CHANGED signal.
-		 */
-		g_signal_handlers_disconnect_by_func (priv->connection, G_CALLBACK (secrets_cleared_cb), self);
-		g_signal_handlers_disconnect_by_func (priv->connection, G_CALLBACK (connection_changed_cb), self);
-
-		/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-		nm_connection_clear_secrets (priv->connection);
-	}
-
 	nm_clear_pointer (&priv->system_secrets, g_variant_unref);
 	nm_clear_pointer (&priv->agent_secrets, g_variant_unref);
 
 	g_clear_pointer (&priv->seen_bssids, g_hash_table_destroy);
 
-	nm_clear_g_signal_handler (priv->session_monitor, &priv->session_changed_id);
-	g_clear_object (&priv->session_monitor);
-
 	g_clear_object (&priv->agent_mgr);
 
 	g_clear_object (&priv->connection);
 
-	g_clear_pointer (&priv->filename, g_free);
-
 	g_clear_pointer (&priv->kf_db_timestamps, nm_key_file_db_unref);
 	g_clear_pointer (&priv->kf_db_seen_bssids, nm_key_file_db_unref);
 
 	G_OBJECT_CLASS (nm_settings_connection_parent_class)->dispose (object);
-}
 
-static void
-get_property (GObject *object, guint prop_id,
-              GValue *value, GParamSpec *pspec)
-{
-	NMSettingsConnection *self = NM_SETTINGS_CONNECTION (object);
-
-	switch (prop_id) {
-	case PROP_UNSAVED:
-		g_value_set_boolean (value, nm_settings_connection_get_unsaved (self));
-		break;
-	case PROP_READY:
-		g_value_set_boolean (value, nm_settings_connection_get_ready (self));
-		break;
-	case PROP_FLAGS:
-		g_value_set_uint (value,
-		                  nm_settings_connection_get_flags (self) & NM_SETTINGS_CONNECTION_INT_FLAGS_EXPORTED_MASK);
-		break;
-	case PROP_FILENAME:
-		g_value_set_string (value, nm_settings_connection_get_filename (self));
-		break;
-	default:
-		G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
-		break;
-	}
-}
+	g_clear_object (&priv->storage);
 
-static void
-set_property (GObject *object, guint prop_id,
-              const GValue *value, GParamSpec *pspec)
-{
-	NMSettingsConnection *self = NM_SETTINGS_CONNECTION (object);
+	nm_clear_g_free (&priv->filename);
 
-	switch (prop_id) {
-	case PROP_FILENAME:
-		/* construct-only */
-		nm_settings_connection_set_filename (self, g_value_get_string (value));
-		break;
-	default:
-		G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
-		break;
-	}
+	g_clear_object (&priv->settings);
 }
 
+/*****************************************************************************/
+
 static const GDBusSignalInfo signal_info_updated = NM_DEFINE_GDBUS_SIGNAL_INFO_INIT (
 	"Updated",
 );
@@ -2946,10 +2796,8 @@ nm_settings_connection_class_init (NMSettingsConnectionClass *klass)
 	dbus_object_class->export_path = NM_DBUS_EXPORT_PATH_NUMBERED (NM_DBUS_PATH_SETTINGS);
 	dbus_object_class->interface_infos = NM_DBUS_INTERFACE_INFOS (&interface_info_settings_connection);
 
-	object_class->constructed = constructed;
 	object_class->dispose = dispose;
 	object_class->get_property = get_property;
-	object_class->set_property = set_property;
 
 	obj_properties[PROP_UNSAVED] =
 	     g_param_spec_boolean (NM_SETTINGS_CONNECTION_UNSAVED, "", "",
@@ -2957,12 +2805,6 @@ nm_settings_connection_class_init (NMSettingsConnectionClass *klass)
 	                           G_PARAM_READABLE |
 	                           G_PARAM_STATIC_STRINGS);
 
-	obj_properties[PROP_READY] =
-	     g_param_spec_boolean (NM_SETTINGS_CONNECTION_READY, "", "",
-	                           TRUE,
-	                           G_PARAM_READABLE |
-	                           G_PARAM_STATIC_STRINGS);
-
 	obj_properties[PROP_FLAGS] =
 	     g_param_spec_uint (NM_SETTINGS_CONNECTION_FLAGS, "", "",
 	                        0, G_MAXUINT32, 0,
@@ -2972,29 +2814,20 @@ nm_settings_connection_class_init (NMSettingsConnectionClass *klass)
 	obj_properties[PROP_FILENAME] =
 	     g_param_spec_string (NM_SETTINGS_CONNECTION_FILENAME, "", "",
 	                          NULL,
-	                          G_PARAM_READWRITE |
-	                          G_PARAM_CONSTRUCT_ONLY |
+	                          G_PARAM_READABLE |
 	                          G_PARAM_STATIC_STRINGS);
 
 	g_object_class_install_properties (object_class, _PROPERTY_ENUMS_LAST, obj_properties);
 
-	/* internal signal, with an argument (gboolean by_user). */
+	/* internal signal, with an argument (NMSettingsConnectionUpdateReason update_reason) as
+	 * guint. */
 	signals[UPDATED_INTERNAL] =
 	    g_signal_new (NM_SETTINGS_CONNECTION_UPDATED_INTERNAL,
 	                  G_TYPE_FROM_CLASS (klass),
 	                  G_SIGNAL_RUN_FIRST,
 	                  0, NULL, NULL,
-	                  g_cclosure_marshal_VOID__BOOLEAN,
-	                  G_TYPE_NONE, 1, G_TYPE_BOOLEAN);
-
-	signals[REMOVED] =
-	    g_signal_new (NM_SETTINGS_CONNECTION_REMOVED,
-	                  G_TYPE_FROM_CLASS (klass),
-	                  G_SIGNAL_RUN_FIRST,
-	                  0,
-	                  NULL, NULL,
-	                  g_cclosure_marshal_VOID__VOID,
-	                  G_TYPE_NONE, 0);
+	                  g_cclosure_marshal_VOID__UINT,
+	                  G_TYPE_NONE, 1, G_TYPE_UINT);
 
 	signals[FLAGS_CHANGED] =
 	    g_signal_new (NM_SETTINGS_CONNECTION_FLAGS_CHANGED,
diff --git a/src/settings/nm-settings-connection.h b/src/settings/nm-settings-connection.h
index 38e34d8fa5..9dfc473da7 100644
--- a/src/settings/nm-settings-connection.h
+++ b/src/settings/nm-settings-connection.h
@@ -21,11 +21,78 @@
 #ifndef __NETWORKMANAGER_SETTINGS_CONNECTION_H__
 #define __NETWORKMANAGER_SETTINGS_CONNECTION_H__
 
-#include <net/ethernet.h>
-
 #include "nm-dbus-object.h"
 #include "nm-connection.h"
 
+#include "nm-settings-storage.h"
+
+/*****************************************************************************/
+
+typedef enum {
+
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_NONE                      = 0,
+
+	/* with persist-mode != NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY, and
+	 * update tries to update the profile on disk (which can always fail).
+	 * In some cases we want to ignore such failure and proceed. For example,
+	 * when we receive secrets from a secret-agent, we want to update the connection
+	 * at all cost and ignore failures to write them to disk. */
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_IGNORE_PERSIST_FAILURE    = (1u << 0),
+
+	/* When updating the profile, force renaming the file on disk. That matters
+	 * only for keyfile plugin. Keyfile prefers a filename based on connection.id.
+	 * When the connection.id changes we might want to rename the file on disk
+	 * (that is, don't overwrite the existing file, but delete it and write it
+	 * with the new name).
+	 * This flag forces such rename. */
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_FORCE_RENAME              = (1u << 1),
+
+	/* Usually, changing a profile that is currently active does not immediately
+	 * reapply the changes. The exception are connection.zone and connection.metered
+	 * properties. When this flag is set, then these two properties are reapplied
+	 * right away. */
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_REAPPLY_PARTIAL           = (1u << 2),
+
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_SYSTEM_SECRETS      = (1u << 3),
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS      = (1u << 4),
+
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_AGENT_SECRETS       = (1u << 5),
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS       = (1u << 6),
+
+	/* if a profile was greated as default-wired connection for a device, then
+	 * when the user modifies it via D-Bus, the profile should become persisted
+	 * to disk and it the purpose why the profile was created should be forgotten. */
+	NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_DEFAULT_WIRED       = (1u << 7),
+
+} NMSettingsConnectionUpdateReason;
+
+typedef enum {
+
+	/* if the profile is in-memory, update it in-memory and keep it.
+	 * if the profile is on-disk, update it on-disk, and keep it. */
+	NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP,
+
+	/* persist to disk. If the profile is currenly in-memory, remove
+	 * it from /run. */
+	NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
+
+	/* persist to /run (in-memory). If the profile is currently on disk,
+	 * delete it from disk. */
+	NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
+
+	/* persist to /run (in-memory). If the profile is currently on disk,
+	 * forget about it, but don't delete it from disk. */
+	NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED,
+
+	/* this only updates the connection in-memory. Note that "in-memory" above
+	 * means to write to keyfile in /run. This really means to not notify the
+	 * settings plugin about the change. */
+	NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST,
+
+} NMSettingsConnectionPersistMode;
+
+/*****************************************************************************/
+
 #define NM_TYPE_SETTINGS_CONNECTION            (nm_settings_connection_get_type ())
 #define NM_SETTINGS_CONNECTION(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NM_TYPE_SETTINGS_CONNECTION, NMSettingsConnection))
 #define NM_SETTINGS_CONNECTION_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NM_TYPE_SETTINGS_CONNECTION, NMSettingsConnectionClass))
@@ -33,7 +100,6 @@
 #define NM_IS_SETTINGS_CONNECTION_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NM_TYPE_SETTINGS_CONNECTION))
 #define NM_SETTINGS_CONNECTION_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NM_TYPE_SETTINGS_CONNECTION, NMSettingsConnectionClass))
 
-#define NM_SETTINGS_CONNECTION_REMOVED "removed"
 #define NM_SETTINGS_CONNECTION_GET_SECRETS "get-secrets"
 #define NM_SETTINGS_CONNECTION_CANCEL_SECRETS "cancel-secrets"
 #define NM_SETTINGS_CONNECTION_UPDATED_INTERNAL "updated-internal"
@@ -44,9 +110,6 @@
 #define NM_SETTINGS_CONNECTION_FLAGS    "flags"
 #define NM_SETTINGS_CONNECTION_FILENAME "filename"
 
-/* Internal properties */
-#define NM_SETTINGS_CONNECTION_READY    "ready"
-
 /**
  * NMSettingsConnectionIntFlags:
  * @NM_SETTINGS_CONNECTION_INT_FLAGS_NONE: no flag set
@@ -61,38 +124,37 @@
  *  currently active but cleanup on disconnect.
  *  See also #NM_SETTINGS_CONNECTION_FLAG_VOLATILE.
  * @NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE: The connection is visible
- * @NM_SETTINGS_CONNECTION_INT_FLAGS_EXPORTED_MASK: the entire enum is
+ * @_NM_SETTINGS_CONNECTION_INT_FLAGS_EXPORTED_MASK: the entire enum is
  *   internal, however, parts of it is public API as #NMSettingsConnectionFlags.
  *   This mask, are the public flags.
- * @NM_SETTINGS_CONNECTION_INT_FLAGS_ALL: special mask, for all known flags
+ * @_NM_SETTINGS_CONNECTION_INT_FLAGS_ALL: special mask, for all known flags
  *
  * #NMSettingsConnection flags.
  **/
-typedef enum {
+typedef enum _NMSettingsConnectionIntFlags {
 	NM_SETTINGS_CONNECTION_INT_FLAGS_NONE                   = 0,
 
 	NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED                = NM_SETTINGS_CONNECTION_FLAG_UNSAVED,
 	NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED           = NM_SETTINGS_CONNECTION_FLAG_NM_GENERATED,
 	NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE               = NM_SETTINGS_CONNECTION_FLAG_VOLATILE,
 
-	NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE                = (1LL <<  3),
+	NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE                = 0x08,
 
-	__NM_SETTINGS_CONNECTION_INT_FLAGS_LAST,
+	_NM_SETTINGS_CONNECTION_INT_FLAGS_LAST,
 
-	NM_SETTINGS_CONNECTION_INT_FLAGS_EXPORTED_MASK          = 0
+	_NM_SETTINGS_CONNECTION_INT_FLAGS_EXPORTED_MASK         = 0
 	                                                          | NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED
 	                                                          | NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
 	                                                          | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
 	                                                          | 0,
 
-	NM_SETTINGS_CONNECTION_INT_FLAGS_ALL = ((__NM_SETTINGS_CONNECTION_INT_FLAGS_LAST - 1) << 1) - 1,
-} NMSettingsConnectionIntFlags;
+	_NM_SETTINGS_CONNECTION_INT_FLAGS_PERSISTENT_MASK       = 0
+	                                                          | NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+	                                                          | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
+	                                                          | 0,
 
-typedef enum {
-	NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE                       = 0,
-	NM_SETTINGS_CONNECTION_COMMIT_REASON_USER_ACTION                = (1LL << 0),
-	NM_SETTINGS_CONNECTION_COMMIT_REASON_ID_CHANGED                 = (1LL << 1),
-} NMSettingsConnectionCommitReason;
+	_NM_SETTINGS_CONNECTION_INT_FLAGS_ALL = ((_NM_SETTINGS_CONNECTION_INT_FLAGS_LAST - 1) << 1) - 1,
+} NMSettingsConnectionIntFlags;
 
 typedef enum {
 	NM_SETTINGS_AUTO_CONNECT_BLOCKED_REASON_NONE                    = 0,
@@ -106,7 +168,6 @@ typedef enum {
 	                                                                   | NM_SETTINGS_AUTO_CONNECT_BLOCKED_REASON_NO_SECRETS),
 } NMSettingsAutoconnectBlockedReason;
 
-struct _NMSettingsConnectionCallId;
 typedef struct _NMSettingsConnectionCallId NMSettingsConnectionCallId;
 
 typedef struct _NMSettingsConnectionClass NMSettingsConnectionClass;
@@ -115,62 +176,48 @@ struct _NMSettingsConnectionPrivate;
 
 struct _NMSettingsConnection {
 	NMDBusObject parent;
-	struct _NMSettingsConnectionPrivate *_priv;
 	CList _connections_lst;
-};
-
-struct _NMSettingsConnectionClass {
-	NMDBusObjectClass parent;
-
-	gboolean (*commit_changes) (NMSettingsConnection *self,
-	                            NMConnection *new_connection,
-	                            NMSettingsConnectionCommitReason commit_reason,
-	                            NMConnection **out_reread_connection,
-	                            char **out_logmsg_change,
-	                            GError **error);
-
-	gboolean (*delete) (NMSettingsConnection *self,
-	                    GError **error);
+	struct _NMSettingsConnectionPrivate *_priv;
 };
 
 GType nm_settings_connection_get_type (void);
 
+NMSettingsConnection *nm_settings_connection_new (void);
+
 NMConnection *nm_settings_connection_get_connection (NMSettingsConnection *self);
 
-guint64 nm_settings_connection_get_last_secret_agent_version_id (NMSettingsConnection *self);
+void _nm_settings_connection_set_connection (NMSettingsConnection *self,
+                                             NMConnection *new_connection,
+                                             NMConnection **out_old_connection,
+                                             NMSettingsConnectionUpdateReason update_reason);
 
-gboolean nm_settings_connection_has_unmodified_applied_connection (NMSettingsConnection *self,
-                                                                   NMConnection *applied_connection,
-                                                                   NMSettingCompareFlags compare_flage);
+NMSettingsStorage *nm_settings_connection_get_storage (NMSettingsConnection *self);
 
-typedef enum {
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP,
+void _nm_settings_connection_set_storage (NMSettingsConnection *self,
+                                          NMSettingsStorage *storage);
 
-	/* like KEEP, but always clears the UNSAVED flag */
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP_SAVED,
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
+gboolean nm_settings_connection_still_valid (NMSettingsConnection *self);
 
-	/* unsaved, only sets the unsaved flag, but it doesn't touch
-	 * the NM_GENERATED nor VOLATILE flag. */
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_UNSAVED,
+const char *nm_settings_connection_get_filename (NMSettingsConnection *self);
 
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY,
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED,
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_DETACHED,
-	NM_SETTINGS_CONNECTION_PERSIST_MODE_VOLATILE_ONLY,
-} NMSettingsConnectionPersistMode;
+guint64 nm_settings_connection_get_last_secret_agent_version_id (NMSettingsConnection *self);
 
-gboolean  nm_settings_connection_update (NMSettingsConnection *self,
-                                         NMConnection *new_connection,
-                                         NMSettingsConnectionPersistMode persist_mode,
-                                         NMSettingsConnectionCommitReason commit_reason,
-                                         const char *log_diff_name,
-                                         GError **error);
+gboolean nm_settings_connection_has_unmodified_applied_connection (NMSettingsConnection *self,
+                                                                   NMConnection *applied_connection,
+                                                                   NMSettingCompareFlags compare_flage);
 
-gboolean nm_settings_connection_delete (NMSettingsConnection *self,
+gboolean nm_settings_connection_update (NMSettingsConnection *self,
+                                        NMConnection *new_connection,
+                                        NMSettingsConnectionPersistMode persist_mode,
+                                        NMSettingsConnectionIntFlags sett_flags,
+                                        NMSettingsConnectionIntFlags sett_mask,
+                                        NMSettingsConnectionUpdateReason update_reason,
+                                        const char *log_context_name,
                                         GError **error);
 
+void nm_settings_connection_delete (NMSettingsConnection *self,
+                                    gboolean allow_add_to_no_auto_default);
+
 typedef void (*NMSettingsConnectionSecretsFunc) (NMSettingsConnection *self,
                                                  NMSettingsConnectionCallId *call_id,
                                                  const char *agent_username,
@@ -196,21 +243,44 @@ NMSettingsConnectionCallId *nm_settings_connection_get_secrets (NMSettingsConnec
 void nm_settings_connection_cancel_secrets (NMSettingsConnection *self,
                                             NMSettingsConnectionCallId *call_id);
 
-void nm_settings_connection_recheck_visibility (NMSettingsConnection *self);
+void nm_settings_connection_clear_secrets (NMSettingsConnection *self,
+                                           gboolean clear_cached_system_secrets,
+                                           gboolean persist);
+
+gboolean nm_settings_connection_check_visibility (NMSettingsConnection *self,
+                                                  NMSessionMonitor *session_monitor);
 
 gboolean nm_settings_connection_check_permission (NMSettingsConnection *self,
                                                   const char *permission);
 
-void nm_settings_connection_added (NMSettingsConnection *self);
+/*****************************************************************************/
 
-void nm_settings_connection_signal_remove (NMSettingsConnection *self);
+NMDevice *nm_settings_connection_default_wired_get_device (NMSettingsConnection *self);
+void      nm_settings_connection_default_wired_set_device (NMSettingsConnection *self,
+                                                           NMDevice *device);
 
-gboolean nm_settings_connection_get_unsaved (NMSettingsConnection *self);
+/*****************************************************************************/
 
 NMSettingsConnectionIntFlags nm_settings_connection_get_flags (NMSettingsConnection *self);
-NMSettingsConnectionIntFlags nm_settings_connection_set_flags (NMSettingsConnection *self, NMSettingsConnectionIntFlags flags, gboolean set);
+
+static inline gboolean
+nm_settings_connection_get_unsaved (NMSettingsConnection *self)
+{
+	return NM_FLAGS_HAS (nm_settings_connection_get_flags (self), NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED);
+}
+
 NMSettingsConnectionIntFlags nm_settings_connection_set_flags_full (NMSettingsConnection *self, NMSettingsConnectionIntFlags mask, NMSettingsConnectionIntFlags value);
 
+static inline NMSettingsConnectionIntFlags
+nm_settings_connection_set_flags (NMSettingsConnection *self, NMSettingsConnectionIntFlags flags, gboolean set)
+{
+	return nm_settings_connection_set_flags_full (self,
+	                                              flags,
+	                                              set ? flags : NM_SETTINGS_CONNECTION_INT_FLAGS_NONE);
+}
+
+/*****************************************************************************/
+
 int nm_settings_connection_cmp_timestamp (NMSettingsConnection *ac, NMSettingsConnection *ab);
 int nm_settings_connection_cmp_timestamp_p_with_data (gconstpointer pa, gconstpointer pb, gpointer user_data);
 int nm_settings_connection_cmp_autoconnect_priority (NMSettingsConnection *a, NMSettingsConnection *b);
@@ -218,9 +288,9 @@ int nm_settings_connection_cmp_autoconnect_priority_p_with_data (gconstpointer p
 
 struct _NMKeyFileDB;
 
-void nm_settings_connection_register_kf_dbs (NMSettingsConnection *self,
-                                             struct _NMKeyFileDB *kf_db_timestamps,
-                                             struct _NMKeyFileDB *kf_db_seen_bssids);
+void _nm_settings_connection_register_kf_dbs (NMSettingsConnection *self,
+                                              struct _NMKeyFileDB *kf_db_timestamps,
+                                              struct _NMKeyFileDB *kf_db_seen_bssids);
 
 gboolean nm_settings_connection_get_timestamp (NMSettingsConnection *self,
                                                guint64 *out_timestamp);
@@ -259,14 +329,6 @@ nm_settings_connection_autoconnect_blocked_reason_set (NMSettingsConnection *sel
 
 gboolean nm_settings_connection_autoconnect_is_blocked (NMSettingsConnection *self);
 
-gboolean nm_settings_connection_get_ready (NMSettingsConnection *self);
-void     nm_settings_connection_set_ready (NMSettingsConnection *self,
-                                           gboolean ready);
-
-void        nm_settings_connection_set_filename (NMSettingsConnection *self,
-                                                 const char *filename);
-const char *nm_settings_connection_get_filename (NMSettingsConnection *self);
-
 const char *nm_settings_connection_get_id              (NMSettingsConnection *connection);
 const char *nm_settings_connection_get_uuid            (NMSettingsConnection *connection);
 const char *nm_settings_connection_get_connection_type (NMSettingsConnection *connection);
@@ -276,4 +338,14 @@ const char *nm_settings_connection_get_connection_type (NMSettingsConnection *co
 NMConnection **nm_settings_connections_array_to_connections (NMSettingsConnection *const*connections,
                                                              gssize n_connections);
 
+/*****************************************************************************/
+
+void _nm_settings_connection_emit_dbus_signal_updated (NMSettingsConnection *self);
+void _nm_settings_connection_emit_dbus_signal_removed (NMSettingsConnection *self);
+
+void _nm_settings_connection_emit_signal_updated_internal (NMSettingsConnection *self,
+                                                           NMSettingsConnectionUpdateReason update_reason);
+
+void _nm_settings_connection_cleanup_after_remove (NMSettingsConnection *self);
+
 #endif /* __NETWORKMANAGER_SETTINGS_CONNECTION_H__ */
diff --git a/src/settings/nm-settings-plugin.c b/src/settings/nm-settings-plugin.c
index 81a168b24d..8ae1e6528a 100644
--- a/src/settings/nm-settings-plugin.c
+++ b/src/settings/nm-settings-plugin.c
@@ -22,12 +22,14 @@
 
 #include "nm-settings-plugin.h"
 
+#include "nm-utils.h"
+#include "nm-core-internal.h"
+
 #include "nm-settings-connection.h"
 
 /*****************************************************************************/
 
 enum {
-	CONNECTION_ADDED,
 	UNMANAGED_SPECS_CHANGED,
 	UNRECOGNIZED_SPECS_CHANGED,
 
@@ -41,104 +43,201 @@ G_DEFINE_TYPE (NMSettingsPlugin, nm_settings_plugin, G_TYPE_OBJECT)
 /*****************************************************************************/
 
 GSList *
-nm_settings_plugin_get_connections (NMSettingsPlugin *self)
+nm_settings_plugin_get_unmanaged_specs (NMSettingsPlugin *self)
 {
+	NMSettingsPluginClass *klass;
+
 	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), NULL);
 
-	if (NM_SETTINGS_PLUGIN_GET_CLASS (self)->get_connections)
-		return NM_SETTINGS_PLUGIN_GET_CLASS (self)->get_connections (self);
-	return NULL;
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+	if (!klass->get_unmanaged_specs)
+		return NULL;
+	return klass->get_unmanaged_specs (self);
 }
 
-gboolean
-nm_settings_plugin_load_connection (NMSettingsPlugin *self,
-                                    const char *filename)
+GSList *
+nm_settings_plugin_get_unrecognized_specs (NMSettingsPlugin *self)
 {
-	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), FALSE);
+	NMSettingsPluginClass *klass;
+
+	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), NULL);
 
-	if (NM_SETTINGS_PLUGIN_GET_CLASS (self)->load_connection)
-		return NM_SETTINGS_PLUGIN_GET_CLASS (self)->load_connection (self, filename);
-	return FALSE;
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+	if (!klass->get_unrecognized_specs)
+		return NULL;
+	return klass->get_unrecognized_specs (self);
 }
 
 void
-nm_settings_plugin_reload_connections (NMSettingsPlugin *self)
+nm_settings_plugin_reload_connections (NMSettingsPlugin *self,
+                                       NMSettingsPluginConnectionLoadCallback callback,
+                                       gpointer user_data)
 {
+	NMSettingsPluginClass *klass;
+
 	g_return_if_fail (NM_IS_SETTINGS_PLUGIN (self));
+	g_return_if_fail (callback);
 
-	if (NM_SETTINGS_PLUGIN_GET_CLASS (self)->reload_connections)
-		NM_SETTINGS_PLUGIN_GET_CLASS (self)->reload_connections (self);
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+	if (klass->reload_connections)
+		klass->reload_connections (self, callback, user_data);
 }
 
-GSList *
-nm_settings_plugin_get_unmanaged_specs (NMSettingsPlugin *self)
+NMSettingsPluginConnectionLoadEntry *
+nm_settings_plugin_create_connection_load_entries (const char *const*filenames,
+                                                   gsize *out_len)
 {
-	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), NULL);
+	NMSettingsPluginConnectionLoadEntry *entries;
+	gsize len;
+	gsize i;
+
+	len = NM_PTRARRAY_LEN (filenames);
+	if (len == 0) {
+		*out_len = 0;
+		return NULL;
+	}
+
+	entries = g_new (NMSettingsPluginConnectionLoadEntry, len);
+	for (i = 0; i < len; i++) {
+		entries[i] = (NMSettingsPluginConnectionLoadEntry) {
+			.filename = filenames[i],
+			.error    = NULL,
+			.handled  = FALSE,
+		};
+	}
 
-	if (NM_SETTINGS_PLUGIN_GET_CLASS (self)->get_unmanaged_specs)
-		return NM_SETTINGS_PLUGIN_GET_CLASS (self)->get_unmanaged_specs (self);
-	return NULL;
+	*out_len = len;
+	return entries;
 }
 
-GSList *
-nm_settings_plugin_get_unrecognized_specs (NMSettingsPlugin *self)
+void
+nm_settings_plugin_load_connections (NMSettingsPlugin *self,
+                                     NMSettingsPluginConnectionLoadEntry *entries,
+                                     gsize n_entries,
+                                     NMSettingsPluginConnectionLoadCallback callback,
+                                     gpointer user_data)
 {
-	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), NULL);
+	NMSettingsPluginClass *klass;
+
+	g_return_if_fail (NM_IS_SETTINGS_PLUGIN (self));
 
-	if (NM_SETTINGS_PLUGIN_GET_CLASS (self)->get_unrecognized_specs)
-		return NM_SETTINGS_PLUGIN_GET_CLASS (self)->get_unrecognized_specs (self);
-	return NULL;
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+	if (klass->load_connections)
+		klass->load_connections (self, entries, n_entries, callback, user_data);
 }
 
-/**
- * nm_settings_plugin_add_connection:
- * @self: the #NMSettingsPlugin
- * @connection: the source connection to create a plugin-specific
- * #NMSettingsConnection from
- * @save_to_disk: %TRUE to save the connection to disk immediately, %FALSE to
- * not save to disk
- * @error: on return, a location to store any errors that may occur
- *
- * Creates a new #NMSettingsConnection for the given source @connection.  If the
- * plugin cannot handle the given connection type, it should return %NULL and
- * set @error.  The plugin owns the returned object and the caller must reference
- * the object if it wishes to continue using it.
- *
- * Returns: the new #NMSettingsConnection or %NULL
- */
-NMSettingsConnection *
+void
+nm_settings_plugin_load_connections_done (NMSettingsPlugin *self)
+{
+	NMSettingsPluginClass *klass;
+
+	g_return_if_fail (NM_IS_SETTINGS_PLUGIN (self));
+
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+	if (klass->load_connections_done)
+		klass->load_connections_done (self);
+}
+
+gboolean
 nm_settings_plugin_add_connection (NMSettingsPlugin *self,
                                    NMConnection *connection,
-                                   gboolean save_to_disk,
+                                   NMSettingsStorage **out_storage,
+                                   NMConnection **out_connection,
                                    GError **error)
 {
 	NMSettingsPluginClass *klass;
 
-	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), NULL);
-	g_return_val_if_fail (NM_IS_CONNECTION (connection), NULL);
+	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), FALSE);
+	g_return_val_if_fail (NM_IS_CONNECTION (connection), FALSE);
+
+#if NM_MORE_ASSERTS > 5
+	nm_assert (nm_connection_verify (connection, NULL));
+#endif
+
+	NM_SET_OUT (out_storage, NULL);
+	NM_SET_OUT (out_connection, NULL);
 
 	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
 	if (!klass->add_connection) {
 		g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_NOT_SUPPORTED,
-		                     "Plugin does not support adding connections");
-		return NULL;
+		                     "settings plugin does not support adding connections");
+		return FALSE;
 	}
-
-	return klass->add_connection (self, connection, save_to_disk, error);
+	return klass->add_connection (self,
+	                              connection,
+	                              out_storage,
+	                              out_connection,
+	                              error);
 }
 
-/*****************************************************************************/
+gboolean
+nm_settings_plugin_update_connection (NMSettingsPlugin *self,
+                                      NMSettingsStorage *storage,
+                                      NMConnection *connection,
+                                      NMSettingsStorage **out_storage,
+                                      NMConnection **out_connection,
+                                      GError **error)
+{
+	NMSettingsPluginClass *klass = NULL;
 
-void
-_nm_settings_plugin_emit_signal_connection_added (NMSettingsPlugin *self,
-                                                  NMSettingsConnection *sett_conn)
+	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), FALSE);
+	g_return_val_if_fail (NM_IS_SETTINGS_STORAGE (storage), FALSE);
+	g_return_val_if_fail (nm_settings_storage_get_plugin (storage) == self, FALSE);
+	g_return_val_if_fail (NM_IS_CONNECTION (connection), FALSE);
+
+#if NM_MORE_ASSERTS > 5
+	nm_assert (nm_connection_verify (connection, NULL));
+	nm_assert (nm_streq (nm_connection_get_uuid (connection), nm_settings_storage_get_uuid (storage)));
+#endif
+
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+
+	NM_SET_OUT (out_storage, NULL);
+	NM_SET_OUT (out_connection, NULL);
+
+	if (!klass->update_connection) {
+		g_set_error (error,
+		             NM_SETTINGS_ERROR,
+		             NM_SETTINGS_ERROR_NOT_SUPPORTED,
+		             "settings plugin does not support modifying connections");
+		return FALSE;
+	}
+	return klass->update_connection (self,
+	                                 storage,
+	                                 connection,
+	                                 out_storage,
+	                                 out_connection,
+	                                 error);
+}
+
+gboolean
+nm_settings_plugin_delete_connection (NMSettingsPlugin *self,
+                                      NMSettingsStorage *storage,
+                                      GError **error)
 {
-	nm_assert (NM_IS_SETTINGS_PLUGIN (self));
-	nm_assert (NM_IS_SETTINGS_CONNECTION (sett_conn));
+	NMSettingsPluginClass *klass = NULL;
+
+	g_return_val_if_fail (NM_IS_SETTINGS_PLUGIN (self), FALSE);
+	g_return_val_if_fail (NM_IS_SETTINGS_STORAGE (storage), FALSE);
+	g_return_val_if_fail (nm_settings_storage_get_plugin (storage) == self, FALSE);
 
-	g_signal_emit (self, signals[CONNECTION_ADDED], 0, sett_conn);
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+
+	if (!klass->delete_connection) {
+		g_set_error (error,
+		             NM_SETTINGS_ERROR,
+		             NM_SETTINGS_ERROR_NOT_SUPPORTED,
+		             "settings plugin does not support deleting connections");
+		return FALSE;
+	}
+
+	return klass->delete_connection (self,
+	                                 storage,
+	                                 error);
 }
 
+/*****************************************************************************/
+
 void
 _nm_settings_plugin_emit_signal_unmanaged_specs_changed (NMSettingsPlugin *self)
 {
@@ -167,15 +266,6 @@ nm_settings_plugin_class_init (NMSettingsPluginClass *klass)
 {
 	GObjectClass *object_class = G_OBJECT_CLASS (klass);
 
-	signals[CONNECTION_ADDED] =
-	    g_signal_new (NM_SETTINGS_PLUGIN_CONNECTION_ADDED,
-	                  G_OBJECT_CLASS_TYPE (object_class),
-	                  G_SIGNAL_RUN_FIRST,
-	                  0, NULL, NULL,
-	                  g_cclosure_marshal_VOID__OBJECT,
-	                  G_TYPE_NONE, 1,
-	                  NM_TYPE_SETTINGS_CONNECTION);
-
 	signals[UNMANAGED_SPECS_CHANGED] =
 	    g_signal_new (NM_SETTINGS_PLUGIN_UNMANAGED_SPECS_CHANGED,
 	                  G_OBJECT_CLASS_TYPE (object_class),
diff --git a/src/settings/nm-settings-plugin.h b/src/settings/nm-settings-plugin.h
index 11b859978a..188614bed7 100644
--- a/src/settings/nm-settings-plugin.h
+++ b/src/settings/nm-settings-plugin.h
@@ -23,6 +23,21 @@
 
 #include "nm-connection.h"
 
+#include "nm-settings-storage.h"
+
+typedef struct _NMSettingsPlugin NMSettingsPlugin;
+
+typedef void (*NMSettingsPluginConnectionLoadCallback) (NMSettingsPlugin *self,
+                                                        NMSettingsStorage *storage,
+                                                        NMConnection *connection,
+                                                        gpointer user_data);
+
+typedef struct {
+	const char *filename;
+	GError *error;
+	bool handled:1;
+} NMSettingsPluginConnectionLoadEntry;
+
 #define NM_TYPE_SETTINGS_PLUGIN               (nm_settings_plugin_get_type ())
 #define NM_SETTINGS_PLUGIN(obj)               (G_TYPE_CHECK_INSTANCE_CAST ((obj), NM_TYPE_SETTINGS_PLUGIN, NMSettingsPlugin))
 #define NM_SETTINGS_PLUGIN_CLASS(klass)       (G_TYPE_CHECK_CLASS_CAST ((klass), NM_TYPE_SETTINGS_PLUGIN, NMSettingsPluginClass))
@@ -32,32 +47,14 @@
 
 #define NM_SETTINGS_PLUGIN_UNMANAGED_SPECS_CHANGED    "unmanaged-specs-changed"
 #define NM_SETTINGS_PLUGIN_UNRECOGNIZED_SPECS_CHANGED "unrecognized-specs-changed"
-#define NM_SETTINGS_PLUGIN_CONNECTION_ADDED           "connection-added"
 
-typedef struct {
+struct _NMSettingsPlugin {
 	GObject parent;
-} NMSettingsPlugin;
+};
 
 typedef struct {
 	GObjectClass parent;
 
-	/* Returns a GSList of NMSettingsConnection objects that represent
-	 * connections the plugin knows about.  The returned list is freed by the
-	 * system settings service.
-	 */
-	GSList * (*get_connections) (NMSettingsPlugin *plugin);
-
-	/* Requests that the plugin load/reload a single connection, if it
-	 * recognizes the filename. Returns success or failure.
-	 */
-	gboolean (*load_connection) (NMSettingsPlugin *plugin,
-	                             const char *filename);
-
-	/* Requests that the plugin reload all connection files from disk,
-	 * and emit signals reflecting new, changed, and removed connections.
-	 */
-	void (*reload_connections) (NMSettingsPlugin *plugin);
-
 	/*
 	 * Return a string list of specifications of devices which NetworkManager
 	 * should not manage.  Returned list will be freed by the system settings
@@ -67,7 +64,7 @@ typedef struct {
 	 * Each string in the list must be in one of the formats recognized by
 	 * nm_device_spec_match_list().
 	 */
-	GSList * (*get_unmanaged_specs) (NMSettingsPlugin *plugin);
+	GSList * (*get_unmanaged_specs) (NMSettingsPlugin *self);
 
 	/*
 	 * Return a string list of specifications of devices for which at least
@@ -79,49 +76,131 @@ typedef struct {
 	 * Each string in the list must be in one of the formats recognized by
 	 * nm_device_spec_match_list().
 	 */
-	GSList * (*get_unrecognized_specs) (NMSettingsPlugin *plugin);
+	GSList * (*get_unrecognized_specs) (NMSettingsPlugin *self);
 
-	/*
-	 * Initialize the plugin-specific connection and return a new
-	 * NMSettingsConnection subclass that contains the same settings as the
-	 * original connection.  The connection should only be saved to backing
-	 * storage if @save_to_disk is TRUE.  The returned object is owned by the
-	 * plugin and must be referenced by the owner if necessary.
+	/* Requests that the plugin load/reload a set of filenames.
 	 */
-	NMSettingsConnection * (*add_connection) (NMSettingsPlugin *plugin,
-	                                          NMConnection *connection,
-	                                          gboolean save_to_disk,
-	                                          GError **error);
+	void (*load_connections) (NMSettingsPlugin *self,
+	                          NMSettingsPluginConnectionLoadEntry *entries,
+	                          gsize n_entries,
+	                          NMSettingsPluginConnectionLoadCallback callback,
+	                          gpointer user_data);
+
+	/* Requests that the plugin reload all connection files from disk,
+	 * and emit signals reflecting new, changed, and removed connections.
+	 */
+	void (*reload_connections) (NMSettingsPlugin *self,
+	                            NMSettingsPluginConnectionLoadCallback callback,
+	                            gpointer user_data);
+
+	void (*load_connections_done) (NMSettingsPlugin *self);
+
+	gboolean (*add_connection) (NMSettingsPlugin *self,
+	                            NMConnection *connection,
+	                            NMSettingsStorage **out_storage,
+	                            NMConnection **out_connection,
+	                            GError **error);
+
+	gboolean (*update_connection) (NMSettingsPlugin *self,
+	                               NMSettingsStorage *storage,
+	                               NMConnection *connection,
+	                               NMSettingsStorage **out_storage,
+	                               NMConnection **out_connection,
+	                               GError **error);
+
+	gboolean (*delete_connection) (NMSettingsPlugin *self,
+	                               NMSettingsStorage *storage,
+	                               GError **error);
+
+	const char *plugin_name;
+
 } NMSettingsPluginClass;
 
+/*****************************************************************************/
+
 GType nm_settings_plugin_get_type (void);
 
-typedef NMSettingsPlugin *(*NMSettingsPluginFactoryFunc) (void);
+/*****************************************************************************/
 
-/* Plugin's factory function that returns a #NMSettingsPlugin */
-NMSettingsPlugin *nm_settings_plugin_factory (void);
+#define NM_SETTINGS_STORAGE_PRINT_FMT \
+	NM_HASH_OBFUSCATE_PTR_FMT"/%s"
+
+#define NM_SETTINGS_STORAGE_PRINT_ARG(storage)  \
+	NM_HASH_OBFUSCATE_PTR (storage), \
+	nm_settings_plugin_get_plugin_name (nm_settings_storage_get_plugin (storage))
+
+static inline const char *
+nm_settings_plugin_get_plugin_name (NMSettingsPlugin *self)
+{
+	NMSettingsPluginClass *klass;
+
+	nm_assert (NM_SETTINGS_PLUGIN (self));
+
+	klass = NM_SETTINGS_PLUGIN_GET_CLASS (self);
+
+	nm_assert (klass && klass->plugin_name && strlen (klass->plugin_name) > 0);
+
+	return klass->plugin_name;
+}
+
+/*****************************************************************************/
 
-GSList *nm_settings_plugin_get_connections (NMSettingsPlugin *plugin);
+GSList *nm_settings_plugin_get_unmanaged_specs (NMSettingsPlugin *self);
+GSList *nm_settings_plugin_get_unrecognized_specs (NMSettingsPlugin *self);
 
-gboolean nm_settings_plugin_load_connection (NMSettingsPlugin *plugin,
-                                             const char *filename);
-void nm_settings_plugin_reload_connections (NMSettingsPlugin *plugin);
+void nm_settings_plugin_reload_connections (NMSettingsPlugin *self,
+                                            NMSettingsPluginConnectionLoadCallback callback,
+                                            gpointer user_data);
+
+NMSettingsPluginConnectionLoadEntry *nm_settings_plugin_create_connection_load_entries (const char *const*filenames,
+                                                                                        gsize *out_len);
+
+void nm_settings_plugin_load_connections (NMSettingsPlugin *self,
+                                          NMSettingsPluginConnectionLoadEntry *entries,
+                                          gsize n_entries,
+                                          NMSettingsPluginConnectionLoadCallback callback,
+                                          gpointer user_data);
+
+void nm_settings_plugin_load_connections_done (NMSettingsPlugin *self);
+
+gboolean nm_settings_plugin_add_connection (NMSettingsPlugin *self,
+                                            NMConnection *connection,
+                                            NMSettingsStorage **out_storage,
+                                            NMConnection **out_connection,
+                                            GError **error);
+
+gboolean nm_settings_plugin_update_connection (NMSettingsPlugin *self,
+                                               NMSettingsStorage *storage,
+                                               NMConnection *connection,
+                                               NMSettingsStorage **out_storage,
+                                               NMConnection **out_connection,
+                                               GError **error);
+
+gboolean nm_settings_plugin_delete_connection (NMSettingsPlugin *self,
+                                               NMSettingsStorage *storage,
+                                               GError **error);
+
+/*****************************************************************************/
+
+typedef NMSettingsPlugin *(*NMSettingsPluginFactoryFunc) (void);
+
+NMSettingsPlugin *nm_settings_plugin_factory (void);
 
-GSList *nm_settings_plugin_get_unmanaged_specs (NMSettingsPlugin *plugin);
-GSList *nm_settings_plugin_get_unrecognized_specs (NMSettingsPlugin *plugin);
+/*****************************************************************************
+ * Internal API
+ *****************************************************************************/
 
-NMSettingsConnection *nm_settings_plugin_add_connection (NMSettingsPlugin *plugin,
-                                                         NMConnection *connection,
-                                                         gboolean save_to_disk,
-                                                         GError **error);
+void _nm_settings_plugin_emit_signal_unmanaged_specs_changed (NMSettingsPlugin *self);
 
-/* internal API */
+void _nm_settings_plugin_emit_signal_unrecognized_specs_changed (NMSettingsPlugin *self);
 
-void _nm_settings_plugin_emit_signal_connection_added (NMSettingsPlugin *plugin,
-                                                       NMSettingsConnection *sett_conn);
+/*****************************************************************************/
 
-void _nm_settings_plugin_emit_signal_unmanaged_specs_changed (NMSettingsPlugin *plugin);
+/* forward declare this function from NMSettings. It's used by the ifcfg-rh plugin,
+ * but that shouldn't include all "nm-settings.h" header. */
+NMSettings *nm_settings_get (void);
 
-void _nm_settings_plugin_emit_signal_unrecognized_specs_changed (NMSettingsPlugin *plugin);
+const char *nm_settings_get_dbus_path_for_uuid (NMSettings *self,
+                                                const char *uuid);
 
 #endif /* __NM_SETTINGS_PLUGIN_H__ */
diff --git a/src/settings/nm-settings-storage.c b/src/settings/nm-settings-storage.c
new file mode 100644
index 0000000000..c5942c6e02
--- /dev/null
+++ b/src/settings/nm-settings-storage.c
@@ -0,0 +1,197 @@
+/*
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301 USA.
+ *
+ * Copyright 2018 Red Hat, Inc.
+ */
+
+#include "nm-default.h"
+
+#include "nm-settings-storage.h"
+
+#include "nm-utils.h"
+#include "nm-settings-plugin.h"
+
+#include "settings/plugins/keyfile/nms-keyfile-storage.h"
+
+/*****************************************************************************/
+
+int
+nm_settings_storage_cmp (NMSettingsStorage *a,
+                         NMSettingsStorage *b,
+                         const GSList *plugin_list)
+{
+	NMSettingsStorageClass *klass;
+	NMSettingsPlugin *plugin_a;
+	NMSettingsPlugin *plugin_b;
+
+	/* Sort by priority.
+	 *
+	 * If a > b (by priority), we return a positive number (as one
+	 * would expect by a cmp() function). */
+
+	nm_assert (NM_IS_SETTINGS_STORAGE (a));
+	nm_assert (NM_IS_SETTINGS_STORAGE (b));
+	nm_assert (a != b);
+	nm_assert (nm_streq (nm_settings_storage_get_uuid (a), nm_settings_storage_get_uuid (b)));
+
+	/* in-memory has always higher priority */
+	NM_CMP_DIRECT (nm_settings_storage_is_keyfile_run (a),
+	               nm_settings_storage_is_keyfile_run (b));
+
+	plugin_a = nm_settings_storage_get_plugin (a);
+	plugin_b = nm_settings_storage_get_plugin (b);
+
+	if (plugin_a != plugin_b) {
+		int idx_a = g_slist_index ((GSList *) plugin_list, plugin_a);
+		int idx_b = g_slist_index ((GSList *) plugin_list, plugin_b);
+
+		/* the plugins must be found in the list. */
+		nm_assert (idx_a >= 0);
+		nm_assert (idx_b >= 0);
+		nm_assert (idx_a != idx_b);
+
+		/* plugins that appear first in @plugin_list have higher priority.
+		 * That means: smaller index -> higher priority. Reverse sort. */
+		NM_CMP_DIRECT (idx_b, idx_a);
+
+		/* undecided. We really don't expect unknown plugins here. */
+		return 0;
+	}
+
+	klass = NM_SETTINGS_STORAGE_GET_CLASS (a);
+
+	if (klass != NM_SETTINGS_STORAGE_GET_CLASS (b)) {
+		/* one plugin must return storages of the same type. Otherwise, it's
+		 * unclear how cmp_fcn() should compare them. */
+		nm_assert_not_reached ();
+		return 0;
+	}
+
+	if (klass->cmp_fcn)
+		NM_CMP_RETURN (klass->cmp_fcn (a, b));
+
+	return 0;
+}
+
+/*****************************************************************************/
+
+NM_GOBJECT_PROPERTIES_DEFINE_BASE (
+	PROP_PLUGIN,
+	PROP_UUID,
+	PROP_FILENAME,
+);
+
+G_DEFINE_TYPE (NMSettingsStorage, nm_settings_storage, G_TYPE_OBJECT)
+
+/*****************************************************************************/
+
+static void
+set_property (GObject *object, guint prop_id,
+              const GValue *value, GParamSpec *pspec)
+{
+	NMSettingsStorage *self = NM_SETTINGS_STORAGE (object);
+
+	switch (prop_id) {
+	case PROP_PLUGIN:
+		/* construct-only */
+		self->_plugin = g_object_ref (g_value_get_object (value));
+		nm_assert (NM_IS_SETTINGS_PLUGIN (self->_plugin));
+		break;
+	case PROP_UUID:
+		/* construct-only */
+		self->_uuid = g_value_dup_string (value);
+		nm_assert (!self->_uuid || nm_utils_is_uuid (self->_uuid));
+		break;
+	case PROP_FILENAME:
+		/* construct-only */
+		self->_filename = g_value_dup_string (value);
+		break;
+	default:
+		G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+		break;
+	}
+}
+
+/*****************************************************************************/
+
+static void
+nm_settings_storage_init (NMSettingsStorage *self)
+{
+	c_list_init (&self->_storage_lst);
+	c_list_init (&self->_storage_by_uuid_lst);
+}
+
+NMSettingsStorage *
+nm_settings_storage_new (NMSettingsPlugin *plugin,
+                         const char *uuid,
+                         const char *filename)
+{
+	nm_assert (NM_IS_SETTINGS_PLUGIN (plugin));
+	nm_assert (nm_utils_is_uuid (uuid));
+
+	return g_object_new (NM_TYPE_SETTINGS_STORAGE,
+	                     NM_SETTINGS_STORAGE_PLUGIN, plugin,
+	                     NM_SETTINGS_STORAGE_UUID, uuid,
+	                     NM_SETTINGS_STORAGE_FILENAME, filename,
+	                     NULL);
+}
+
+static void
+finalize (GObject *object)
+{
+	NMSettingsStorage *self = NM_SETTINGS_STORAGE (object);
+
+	c_list_unlink_stale (&self->_storage_lst);
+	c_list_unlink_stale (&self->_storage_by_uuid_lst);
+
+	g_object_unref (self->_plugin);
+	g_free (self->_uuid);
+	g_free (self->_filename);
+
+	G_OBJECT_CLASS (nm_settings_storage_parent_class)->finalize (object);
+}
+
+static void
+nm_settings_storage_class_init (NMSettingsStorageClass *klass)
+{
+	GObjectClass *object_class = G_OBJECT_CLASS (klass);
+
+	object_class->set_property = set_property;
+	object_class->finalize     = finalize;
+
+	obj_properties[PROP_PLUGIN] =
+	    g_param_spec_object (NM_SETTINGS_STORAGE_PLUGIN, "", "",
+	                         NM_TYPE_SETTINGS_PLUGIN,
+	                         G_PARAM_WRITABLE |
+	                         G_PARAM_CONSTRUCT_ONLY |
+	                         G_PARAM_STATIC_STRINGS);
+
+	obj_properties[PROP_UUID] =
+	    g_param_spec_string (NM_SETTINGS_STORAGE_UUID, "", "",
+	                         NULL,
+	                         G_PARAM_WRITABLE |
+	                         G_PARAM_CONSTRUCT_ONLY |
+	                         G_PARAM_STATIC_STRINGS);
+
+	obj_properties[PROP_FILENAME] =
+	    g_param_spec_string (NM_SETTINGS_STORAGE_FILENAME, "", "",
+	                         NULL,
+	                         G_PARAM_WRITABLE |
+	                         G_PARAM_CONSTRUCT_ONLY |
+	                         G_PARAM_STATIC_STRINGS);
+
+	g_object_class_install_properties (object_class, _PROPERTY_ENUMS_LAST, obj_properties);
+}
diff --git a/src/settings/nm-settings-storage.h b/src/settings/nm-settings-storage.h
new file mode 100644
index 0000000000..c43145b5af
--- /dev/null
+++ b/src/settings/nm-settings-storage.h
@@ -0,0 +1,122 @@
+/*
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301 USA.
+ *
+ * Copyright 2018 Red Hat, Inc.
+ */
+
+#ifndef __NM_SETTINGS_STORAGE_H__
+#define __NM_SETTINGS_STORAGE_H__
+
+/*****************************************************************************/
+
+#include "c-list/src/c-list.h"
+
+#define NM_TYPE_SETTINGS_STORAGE            (nm_settings_storage_get_type ())
+#define NM_SETTINGS_STORAGE(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NM_TYPE_SETTINGS_STORAGE, NMSettingsStorage))
+#define NM_SETTINGS_STORAGE_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NM_TYPE_SETTINGS_STORAGE, NMSettingsStorageClass))
+#define NM_IS_SETTINGS_STORAGE(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NM_TYPE_SETTINGS_STORAGE))
+#define NM_IS_SETTINGS_STORAGE_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NM_TYPE_SETTINGS_STORAGE))
+#define NM_SETTINGS_STORAGE_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NM_TYPE_SETTINGS_STORAGE, NMSettingsStorageClass))
+
+#define NM_SETTINGS_STORAGE_PLUGIN   "plugin"
+#define NM_SETTINGS_STORAGE_UUID     "uuid"
+#define NM_SETTINGS_STORAGE_FILENAME "filename"
+
+struct _NMSettingsPlugin;
+
+typedef struct NMSettingsStorage {
+	GObject parent;
+	struct _NMSettingsPlugin *_plugin;
+	char *_uuid;
+	char *_filename;
+	CList _storage_lst;
+	CList _storage_by_uuid_lst;
+} NMSettingsStorage;
+
+typedef struct {
+	GObjectClass parent;
+
+	int (*cmp_fcn) (NMSettingsStorage *a,
+	                NMSettingsStorage *b);
+
+} NMSettingsStorageClass;
+
+GType nm_settings_storage_get_type (void);
+
+NMSettingsStorage *nm_settings_storage_new (struct _NMSettingsPlugin *plugin,
+                                            const char *uuid,
+                                            const char *filename);
+
+static inline struct _NMSettingsPlugin *
+nm_settings_storage_get_plugin (const NMSettingsStorage *self)
+{
+	GType nm_settings_plugin_get_type (void);
+
+	g_return_val_if_fail (NM_IS_SETTINGS_STORAGE (self), NULL);
+
+	nm_assert (G_TYPE_CHECK_INSTANCE_TYPE (self->_plugin, nm_settings_plugin_get_type ()));
+	return self->_plugin;
+}
+
+static inline const char *
+nm_settings_storage_get_uuid (const NMSettingsStorage *self)
+{
+	gboolean nm_utils_is_uuid (const char *str);
+
+	g_return_val_if_fail (NM_IS_SETTINGS_STORAGE (self), NULL);
+
+	nm_assert (nm_utils_is_uuid (self->_uuid));
+	return self->_uuid;
+}
+
+static inline const char *
+nm_settings_storage_get_uuid_opt (const NMSettingsStorage *self)
+{
+	gboolean nm_utils_is_uuid (const char *str);
+
+	g_return_val_if_fail (NM_IS_SETTINGS_STORAGE (self), NULL);
+
+	nm_assert (!self->_uuid || nm_utils_is_uuid (self->_uuid));
+	return self->_uuid;
+}
+
+static inline const char *
+nm_settings_storage_get_filename (const NMSettingsStorage *self)
+{
+	g_return_val_if_fail (NM_IS_SETTINGS_STORAGE (self), NULL);
+
+	return self->_filename;
+}
+
+/*****************************************************************************/
+
+#define nm_assert_valid_settings_storage(plugin, storage) \
+	G_STMT_START { \
+		NMSettingsPlugin *const _plugin = (plugin); \
+		NMSettingsStorage *const _storage = (storage); \
+		\
+		nm_assert (!_plugin || NM_IS_SETTINGS_PLUGIN (_plugin)); \
+		nm_assert (NM_IS_SETTINGS_STORAGE (_storage)); \
+		nm_assert (!_plugin || nm_settings_storage_get_plugin (_storage) == _plugin); \
+	} G_STMT_END
+
+/*****************************************************************************/
+
+int nm_settings_storage_cmp (NMSettingsStorage *sd_a,
+                             NMSettingsStorage *sd_b,
+                             const GSList *plugin_list);
+
+#endif /* __NM_SETTINGS_STORAGE_H__ */
diff --git a/src/settings/nm-settings-utils.c b/src/settings/nm-settings-utils.c
new file mode 100644
index 0000000000..0d636537b6
--- /dev/null
+++ b/src/settings/nm-settings-utils.c
@@ -0,0 +1,175 @@
+/* NetworkManager system settings service - keyfile plugin
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Copyright (C) 2019 Red Hat, Inc.
+ */
+
+#include "nm-default.h"
+
+#include "nm-settings-utils.h"
+
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "nm-settings-plugin.h"
+
+/*****************************************************************************/
+
+const struct timespec *
+nm_sett_util_stat_mtime (const char *filename,
+                         gboolean do_lstat,
+                         struct timespec *out_val)
+{
+	struct stat st;
+	struct timeval now_tv;
+
+	if (filename) {
+		if (do_lstat) {
+			if (lstat (filename, &st) == 0) {
+				*out_val = st.st_mtim;
+				return out_val;
+			}
+		} else {
+			if (stat (filename, &st) == 0) {
+				*out_val = st.st_mtim;
+				return out_val;
+			}
+		}
+	}
+
+	if (gettimeofday (&now_tv, NULL) == 0) {
+		*out_val = (struct timespec) {
+			.tv_sec  = now_tv.tv_sec,
+			.tv_nsec = now_tv.tv_usec * 1000u,
+		};
+		return out_val;
+	}
+
+	*out_val = (struct timespec) { };
+	return out_val;
+}
+
+/*****************************************************************************/
+
+gboolean
+nm_sett_util_allow_filename_cb (const char *filename,
+                                gpointer user_data)
+{
+	const NMSettUtilAllowFilenameData *allow_filename_data = user_data;
+
+	if (   allow_filename_data->allowed_filename
+	    && nm_streq (allow_filename_data->allowed_filename, filename))
+		return TRUE;
+
+	return !g_hash_table_contains (allow_filename_data->idx_by_filename, filename);
+}
+
+/*****************************************************************************/
+
+void
+nm_sett_util_storage_by_uuid_head_destroy (NMSettUtilStorageByUuidHead *sbuh)
+{
+	CList *iter;
+
+	while ((iter = c_list_first (&sbuh->_storage_by_uuid_lst_head)))
+		c_list_unlink (iter);
+	g_free (sbuh);
+}
+
+/*****************************************************************************/
+
+void
+nm_sett_util_storages_clear (NMSettUtilStorages *storages)
+{
+	nm_clear_pointer (&storages->idx_by_uuid, g_hash_table_destroy);
+	nm_clear_pointer (&storages->idx_by_filename, g_hash_table_destroy);
+	nm_assert (c_list_is_empty (&storages->_storage_lst_head));
+}
+
+void
+nm_sett_util_storages_add_take (NMSettUtilStorages *storages,
+                                gpointer storage_take_p /* NMSettingsStorage *, take reference */)
+{
+	NMSettingsStorage *storage_take = storage_take_p;
+	NMSettUtilStorageByUuidHead *sbuh;
+	const char *uuid;
+
+	nm_assert (storage_take);
+	nm_assert (c_list_is_empty (&storage_take->_storage_lst));
+	nm_assert (c_list_is_empty (&storage_take->_storage_by_uuid_lst));
+	nm_assert (nm_settings_storage_get_filename (storage_take));
+
+	if (!g_hash_table_replace (storages->idx_by_filename,
+	                           (char *) nm_settings_storage_get_filename (storage_take),
+	                           storage_take /* takes ownership of reference. */))
+		nm_assert_not_reached ();
+
+	uuid = nm_settings_storage_get_uuid_opt (storage_take);
+
+	if (uuid) {
+		sbuh = nm_sett_util_storages_lookup_by_uuid (storages, uuid);
+		if (!sbuh) {
+			gsize l = strlen (uuid) + 1;
+
+			sbuh = g_malloc (sizeof (NMSettUtilStorageByUuidHead) + l);
+			sbuh->uuid = sbuh->uuid_data;
+			c_list_init (&sbuh->_storage_by_uuid_lst_head);
+			memcpy (sbuh->uuid_data, uuid, l);
+			g_hash_table_add (storages->idx_by_uuid, sbuh);
+		}
+		c_list_link_tail (&sbuh->_storage_by_uuid_lst_head, &storage_take->_storage_by_uuid_lst);
+	}
+
+	c_list_link_tail (&storages->_storage_lst_head, &storage_take->_storage_lst);
+}
+
+gpointer /* NMSettingsStorage * */
+nm_sett_util_storages_steal (NMSettUtilStorages *storages,
+                             gpointer storage_p /* NMSettingsStorage **/)
+{
+	NMSettingsStorage *storage = storage_p;
+	NMSettUtilStorageByUuidHead *sbuh;
+	const char *uuid;
+
+	nm_assert (storage);
+	nm_assert (nm_sett_util_storages_lookup_by_filename (storages, nm_settings_storage_get_filename (storage)) == storage);
+	nm_assert (c_list_contains (&storages->_storage_lst_head, &storage->_storage_lst));
+
+	uuid = nm_settings_storage_get_uuid_opt (storage);
+
+	if (!uuid) {
+		nm_assert (c_list_is_empty (&storage->_storage_by_uuid_lst));
+	} else {
+		nm_assert (!c_list_is_empty (&storage->_storage_by_uuid_lst));
+
+		sbuh = nm_sett_util_storages_lookup_by_uuid (storages, uuid);
+
+		nm_assert (sbuh);
+		nm_assert (c_list_contains (&sbuh->_storage_by_uuid_lst_head, &storage->_storage_by_uuid_lst));
+		c_list_unlink (&storage->_storage_by_uuid_lst);
+
+		if (c_list_is_empty (&sbuh->_storage_by_uuid_lst_head))
+			g_hash_table_remove (storages->idx_by_uuid, sbuh);
+	}
+
+	c_list_unlink (&storage->_storage_lst);
+
+	g_hash_table_steal (storages->idx_by_filename, nm_settings_storage_get_filename (storage));
+
+	return storage;
+}
diff --git a/src/settings/nm-settings-utils.h b/src/settings/nm-settings-utils.h
new file mode 100644
index 0000000000..a2a22dc415
--- /dev/null
+++ b/src/settings/nm-settings-utils.h
@@ -0,0 +1,110 @@
+/*
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301 USA.
+ *
+ * Copyright 2019 Red Hat, Inc.
+ */
+
+#ifndef __NM_SETTINGS_UTILS_H__
+#define __NM_SETTINGS_UTILS_H__
+
+#include "nm-settings-storage.h"
+
+/*****************************************************************************/
+
+struct timespec;
+
+const struct timespec *nm_sett_util_stat_mtime (const char *filename,
+                                                gboolean do_lstat,
+                                                struct timespec *out_val);
+
+/*****************************************************************************/
+
+typedef struct {
+	const char *uuid;
+
+	CList _storage_by_uuid_lst_head;
+
+	char uuid_data[];
+} NMSettUtilStorageByUuidHead;
+
+typedef struct {
+	CList _storage_lst_head;
+	GHashTable *idx_by_filename;
+	GHashTable *idx_by_uuid;
+} NMSettUtilStorages;
+
+void nm_sett_util_storage_by_uuid_head_destroy (NMSettUtilStorageByUuidHead *sbuh);
+
+#define NM_SETT_UTIL_STORAGES_INIT(storages, storage_destroy_fcn) \
+	{ \
+		._storage_lst_head = C_LIST_INIT (((storages)._storage_lst_head)), \
+		.idx_by_filename   = g_hash_table_new_full (nm_str_hash, \
+		                                            g_str_equal, \
+		                                            NULL, \
+		                                            (GDestroyNotify) storage_destroy_fcn), \
+		.idx_by_uuid       = g_hash_table_new_full (nm_pstr_hash, \
+		                                            nm_pstr_equal, \
+		                                            NULL, \
+		                                            (GDestroyNotify) nm_sett_util_storage_by_uuid_head_destroy), \
+	}
+
+void nm_sett_util_storages_clear (NMSettUtilStorages *storages);
+
+#define nm_auto_clear_sett_util_storages nm_auto(nm_sett_util_storages_clear)
+
+void nm_sett_util_storages_add_take (NMSettUtilStorages *storages,
+                                     gpointer storage_take_p);
+
+gpointer nm_sett_util_storages_steal (NMSettUtilStorages *storages,
+                                      gpointer storage_p);
+
+/*****************************************************************************/
+
+static inline gpointer /* NMSettingsStorage * */
+nm_sett_util_storages_lookup_by_filename (NMSettUtilStorages *storages,
+                                          const char *filename)
+{
+	nm_assert (filename);
+
+	return g_hash_table_lookup (storages->idx_by_filename, filename);
+}
+
+static inline NMSettUtilStorageByUuidHead *
+nm_sett_util_storages_lookup_by_uuid (NMSettUtilStorages *storages,
+                                      const char *uuid)
+{
+	nm_assert (uuid);
+
+	return g_hash_table_lookup (storages->idx_by_uuid, &uuid);
+}
+
+/*****************************************************************************/
+
+typedef struct {
+	GHashTable *idx_by_filename;
+	const char *allowed_filename;
+} NMSettUtilAllowFilenameData;
+
+#define NM_SETT_UTIL_ALLOW_FILENAME_DATA(_storages, _allowed_filename) \
+    (&((NMSettUtilAllowFilenameData) { \
+       .idx_by_filename  = (_storages)->idx_by_filename, \
+       .allowed_filename = (_allowed_filename), \
+    }))
+
+gboolean nm_sett_util_allow_filename_cb (const char *filename,
+                                         gpointer user_data);
+
+#endif /* __NM_SETTINGS_UTILS_H__ */
diff --git a/src/settings/nm-settings.c b/src/settings/nm-settings.c
index 7bf47a5757..ef242e564a 100644
--- a/src/settings/nm-settings.c
+++ b/src/settings/nm-settings.c
@@ -37,6 +37,7 @@
 
 #include "nm-libnm-core-intern/nm-common-macros.h"
 #include "nm-glib-aux/nm-keyfile-aux.h"
+#include "nm-keyfile-internal.h"
 #include "nm-dbus-interface.h"
 #include "nm-connection.h"
 #include "nm-setting-8021x.h"
@@ -60,6 +61,7 @@
 #include "nm-utils.h"
 #include "nm-core-internal.h"
 
+#include "nm-std-aux/c-list-util.h"
 #include "nm-glib-aux/nm-c-list.h"
 #include "nm-dbus-object.h"
 #include "devices/nm-device-ethernet.h"
@@ -70,6 +72,7 @@
 #include "nm-auth-subject.h"
 #include "nm-session-monitor.h"
 #include "plugins/keyfile/nms-keyfile-plugin.h"
+#include "plugins/keyfile/nms-keyfile-storage.h"
 #include "nm-agent-manager.h"
 #include "nm-config.h"
 #include "nm-audit-manager.h"
@@ -79,15 +82,155 @@
 
 /*****************************************************************************/
 
-#define EXPORT(sym) void * __export_##sym = &sym;
+static NM_CACHED_QUARK_FCN ("default-wired-connection", _default_wired_connection_quark)
+
+/*****************************************************************************/
+
+typedef struct _StorageData {
+	CList sd_lst;
+	NMSettingsStorage *storage;
+	NMConnection *connection;
+	bool prioritize:1;
+} StorageData;
+
+static StorageData *
+_storage_data_new_stale (NMSettingsStorage *storage,
+                         NMConnection *connection)
+{
+	StorageData *sd;
+
+	sd = g_slice_new (StorageData);
+	sd->storage    = g_object_ref (storage);
+	sd->connection = nm_g_object_ref (connection);
+	sd->prioritize = FALSE;
+	return sd;
+}
+
+static void
+_storage_data_destroy (StorageData *sd)
+{
+	c_list_unlink_stale (&sd->sd_lst);
+	g_object_unref (sd->storage);
+	nm_g_object_unref (sd->connection);
+	g_slice_free (StorageData, sd);
+}
+
+static StorageData *
+_storage_data_find_in_lst (CList *head,
+                           NMSettingsStorage *storage)
+{
+	StorageData *sd;
+
+	nm_assert (head);
+	nm_assert (NM_IS_SETTINGS_STORAGE (storage));
+
+	c_list_for_each_entry (sd, head, sd_lst) {
+		if (sd->storage == storage)
+			return sd;
+	}
+	return NULL;
+}
+
+static void
+nm_assert_storage_data_lst (CList *head)
+{
+#if NM_MORE_ASSERTS > 5
+	const char *uuid = NULL;
+	StorageData *sd;
+	CList *iter;
+
+	nm_assert (head);
+
+	if (c_list_is_empty (head))
+		return;
+
+	c_list_for_each_entry (sd, head, sd_lst) {
+		const char *u;
+
+		nm_assert (NM_IS_SETTINGS_STORAGE (sd->storage));
+		nm_assert (!sd->connection || NM_IS_CONNECTION (sd->connection));
+		u = nm_settings_storage_get_uuid (sd->storage);
+		if (!uuid) {
+			uuid = u;
+			nm_assert (nm_utils_is_uuid (uuid));
+		} else
+			nm_assert (nm_streq0 (uuid, u));
+	}
+
+	/* assert that all storages are unique. */
+	c_list_for_each_entry (sd, head, sd_lst) {
+		for (iter = sd->sd_lst.next; iter != head; iter = iter->next)
+			nm_assert (c_list_entry (iter, StorageData, sd_lst)->storage != sd->storage);
+	}
+#endif
+}
+
+static gboolean
+_storage_data_is_alive (StorageData *sd)
+{
+	if (sd->connection)
+		return TRUE;
+
+	if (nm_settings_storage_is_keyfile_tombstone (sd->storage)) {
+		/* entry does not have a profile, but it's here as a tombstone to
+		 * hide/shadow other connections. That's also relevant. */
+		return TRUE;
+	}
 
-EXPORT(nm_settings_connection_get_type)
-EXPORT(nm_settings_connection_update)
+	return FALSE;
+}
 
 /*****************************************************************************/
 
-static NM_CACHED_QUARK_FCN ("default-wired-connection", _default_wired_connection_quark)
-static NM_CACHED_QUARK_FCN ("default-wired-device", _default_wired_device_quark)
+typedef struct {
+	const char *uuid;
+	NMSettingsConnection *sett_conn;
+	NMSettingsStorage *storage;
+	CList sd_lst_head;
+	CList dirty_sd_lst_head;
+
+	CList sce_dirty_lst;
+
+	char _uuid_data[];
+} SettConnEntry;
+
+static SettConnEntry *
+_sett_conn_entry_new (const char *uuid)
+{
+	SettConnEntry *sett_conn_entry;
+	gsize l_p_1;
+
+	nm_assert (nm_utils_is_uuid (uuid));
+
+	l_p_1 = strlen (uuid) + 1;
+
+	sett_conn_entry = g_malloc (sizeof (SettConnEntry) + l_p_1);
+	sett_conn_entry->uuid = sett_conn_entry->_uuid_data;
+	sett_conn_entry->sett_conn = NULL;
+	sett_conn_entry->storage = NULL;
+	c_list_init (&sett_conn_entry->sd_lst_head);
+	c_list_init (&sett_conn_entry->dirty_sd_lst_head);
+	c_list_init (&sett_conn_entry->sce_dirty_lst);
+	memcpy (sett_conn_entry->_uuid_data, uuid, l_p_1);
+	return sett_conn_entry;
+}
+
+static void
+_sett_conn_entry_free (SettConnEntry *sett_conn_entry)
+{
+	c_list_unlink_stale (&sett_conn_entry->sce_dirty_lst);
+	nm_c_list_free_all (&sett_conn_entry->sd_lst_head,       StorageData, sd_lst, _storage_data_destroy);
+	nm_c_list_free_all (&sett_conn_entry->dirty_sd_lst_head, StorageData, sd_lst, _storage_data_destroy);
+	nm_g_object_unref (sett_conn_entry->sett_conn);
+	nm_g_object_unref (sett_conn_entry->storage);
+	g_free (sett_conn_entry);
+}
+
+static NMSettingsConnection *
+_sett_conn_entry_get_conn (SettConnEntry *sett_conn_entry)
+{
+	return sett_conn_entry ? sett_conn_entry->sett_conn : NULL;
+}
 
 /*****************************************************************************/
 
@@ -114,8 +257,12 @@ typedef struct {
 
 	NMConfig *config;
 
+	NMPlatform *platform;
+
 	NMHostnameManager *hostname_manager;
 
+	NMSessionMonitor *session_monitor;
+
 	CList auth_lst_head;
 
 	NMSKeyfilePlugin *keyfile_plugin;
@@ -125,6 +272,10 @@ typedef struct {
 	NMKeyFileDB *kf_db_timestamps;
 	NMKeyFileDB *kf_db_seen_bssids;
 
+	GHashTable *sce_idx;
+
+	CList sce_dirty_lst_head;
+
 	CList connections_lst_head;
 
 	NMSettingsConnection **connections_cached_list;
@@ -132,16 +283,19 @@ typedef struct {
 	GSList *unmanaged_specs;
 	GSList *unrecognized_specs;
 
+	GHashTable *startup_complete_idx;
 	NMSettingsConnection *startup_complete_blocked_by;
+	gulong startup_complete_platform_change_id;
+	guint startup_complete_timeout_id;
 
 	guint connections_len;
 
+	guint connections_generation;
+
 	guint kf_db_flush_idle_id_timestamps;
 	guint kf_db_flush_idle_id_seen_bssids;
 
 	bool started:1;
-	bool startup_complete:1;
-	bool connections_loaded:1;
 
 } NMSettingsPrivate;
 
@@ -160,6 +314,9 @@ G_DEFINE_TYPE (NMSettings, nm_settings, NM_TYPE_DBUS_OBJECT);
 
 /*****************************************************************************/
 
+/* FIXME: a lot of logging lines are directly connected to a profile. Set the @con_uuid
+ *   argument for structured logging. */
+
 #define _NMLOG_DOMAIN         LOGD_SETTINGS
 #define _NMLOG(level, ...) __NMLOG_DEFAULT (level, _NMLOG_DOMAIN, "settings", __VA_ARGS__)
 
@@ -169,57 +326,267 @@ static const NMDBusInterfaceInfoExtended interface_info_settings;
 static const GDBusSignalInfo signal_info_new_connection;
 static const GDBusSignalInfo signal_info_connection_removed;
 
-static void claim_connection (NMSettings *self,
-                              NMSettingsConnection *connection);
-
-static void connection_ready_changed (NMSettingsConnection *conn,
-                                      GParamSpec *pspec,
-                                      gpointer user_data);
-
 static void default_wired_clear_tag (NMSettings *self,
                                      NMDevice *device,
-                                     NMSettingsConnection *connection,
+                                     NMSettingsConnection *sett_conn,
                                      gboolean add_to_no_auto_default);
 
 static void _clear_connections_cached_list (NMSettingsPrivate *priv);
 
+static void _startup_complete_check (NMSettings *self,
+                                     gint64 now_us);
+
 /*****************************************************************************/
 
 static void
-check_startup_complete (NMSettings *self)
+_emit_connection_added (NMSettings *self,
+                        NMSettingsConnection *sett_conn)
 {
-	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	g_signal_emit (self, signals[CONNECTION_ADDED], 0, sett_conn);
+}
+
+static void
+_emit_connection_updated (NMSettings *self,
+                          NMSettingsConnection *sett_conn,
+                          NMSettingsConnectionUpdateReason update_reason)
+{
+	_nm_settings_connection_emit_signal_updated_internal (sett_conn, update_reason);
+	g_signal_emit (self, signals[CONNECTION_UPDATED], 0, sett_conn, (guint) update_reason);
+}
+
+static void
+_emit_connection_removed (NMSettings *self,
+                          NMSettingsConnection *sett_conn)
+{
+	g_signal_emit (self, signals[CONNECTION_REMOVED], 0, sett_conn);
+}
+
+static void
+_emit_connection_flags_changed (NMSettings *self,
+                                NMSettingsConnection *sett_conn)
+{
+	g_signal_emit (self, signals[CONNECTION_FLAGS_CHANGED], 0, sett_conn);
+}
+
+/*****************************************************************************/
+
+typedef struct {
 	NMSettingsConnection *sett_conn;
+	gint64 start_at;
+	gint64 timeout;
+} StartupCompleteData;
+
+static void
+_startup_complete_data_destroy (StartupCompleteData *scd)
+{
+	g_object_unref (scd->sett_conn);
+	g_slice_free (StartupCompleteData, scd);
+}
+
+static gboolean
+_startup_complete_check_is_ready (NMPlatform *platform,
+                                  NMSettingsConnection *sett_conn)
+{
+	const NMPlatformLink *plink;
+	const char *ifname;
+
+	/* FIXME: instead of just looking for the interface name, it would be better
+	 *        to wait for a device that is compatible with the profile. */
+
+	ifname = nm_connection_get_interface_name (nm_settings_connection_get_connection (sett_conn));
+
+	if (!ifname)
+		return TRUE;
+
+	plink = nm_platform_link_get_by_ifname (platform, ifname);
+	return plink && plink->initialized;
+}
+
+static gboolean
+_startup_complete_timeout_cb (gpointer user_data)
+{
+	NMSettings *self = user_data;
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+
+	priv->startup_complete_timeout_id = 0;
+	_startup_complete_check (self, 0);
+	return G_SOURCE_REMOVE;
+}
+
+static void
+_startup_complete_platform_change_cb (NMPlatform *platform,
+                                      int obj_type_i,
+                                      int ifindex,
+                                      const NMPlatformLink *link,
+                                      int change_type_i,
+                                      NMSettings *self)
+{
+	const NMPlatformSignalChangeType change_type = change_type_i;
+	NMSettingsPrivate *priv;
+	const char *ifname;
 
-	if (priv->startup_complete)
+	if (change_type == NM_PLATFORM_SIGNAL_REMOVED)
 		return;
 
-	c_list_for_each_entry (sett_conn, &priv->connections_lst_head, _connections_lst) {
-		if (!nm_settings_connection_get_ready (sett_conn)) {
-			nm_g_object_ref_set (&priv->startup_complete_blocked_by, sett_conn);
-			return;
+	if (!link->initialized)
+		return;
+
+	priv = NM_SETTINGS_GET_PRIVATE (self);
+
+	ifname = nm_connection_get_interface_name (nm_settings_connection_get_connection (priv->startup_complete_blocked_by));
+	if (   ifname
+	    && !nm_streq (ifname, link->name))
+		return;
+
+	nm_assert (priv->startup_complete_timeout_id > 0);
+
+	nm_clear_g_source (&priv->startup_complete_timeout_id);
+	priv->startup_complete_timeout_id = g_idle_add (_startup_complete_timeout_cb, self);
+}
+
+static void
+_startup_complete_check (NMSettings *self,
+                         gint64 now_us)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	gint64 next_expiry;
+	StartupCompleteData *scd;
+	NMSettingsConnection *next_sett_conn = NULL;
+	GHashTableIter iter;
+
+	if (!priv->started) {
+		/* before we are started, we don't setup the timers... */
+		return;
+	}
+
+	if (!priv->startup_complete_idx)
+		goto ready;
+
+	if (!now_us)
+		now_us = nm_utils_get_monotonic_timestamp_us ();
+
+	next_expiry = 0;
+
+	g_hash_table_iter_init (&iter, priv->startup_complete_idx);
+	while (g_hash_table_iter_next (&iter, (gpointer *) &scd, NULL)) {
+		gint64 expiry;
+
+		if (scd->start_at == 0) {
+			/* once ready, the decision is remembered and there is nothing
+			 * left to check. */
+			continue;
+		}
+
+		expiry = scd->start_at + scd->timeout;
+		if (expiry <= now_us) {
+			scd->start_at = 0;
+			continue;
+		}
+
+		if (_startup_complete_check_is_ready (priv->platform, scd->sett_conn)) {
+			scd->start_at = 0;
+			continue;
 		}
+
+		next_expiry = expiry;
+		next_sett_conn = scd->sett_conn;
+		/* we found one timeout for which to wait. that's good enough. */
+		break;
 	}
 
-	g_clear_object (&priv->startup_complete_blocked_by);
+	nm_clear_g_source (&priv->startup_complete_timeout_id);
+	nm_g_object_ref_set (&priv->startup_complete_blocked_by, next_sett_conn);
+	if (next_expiry > 0) {
+		nm_assert (priv->startup_complete_blocked_by);
+		if (priv->startup_complete_platform_change_id == 0) {
+			priv->startup_complete_platform_change_id = g_signal_connect (priv->platform,
+			                                                              NM_PLATFORM_SIGNAL_LINK_CHANGED,
+			                                                              G_CALLBACK (_startup_complete_platform_change_cb),
+			                                                              self);
+		}
+		priv->startup_complete_timeout_id = g_timeout_add (NM_MIN (3600u*1000u, (next_expiry - now_us) / 1000u),
+		                                                   _startup_complete_timeout_cb,
+		                                                   self);
+		_LOGT ("startup-complete: wait for device \"%s\" due to connection %s (%s)",
+		       nm_connection_get_interface_name (nm_settings_connection_get_connection (priv->startup_complete_blocked_by)),
+		       nm_settings_connection_get_uuid (priv->startup_complete_blocked_by),
+		       nm_settings_connection_get_id (priv->startup_complete_blocked_by));
+		return;
+	}
 
-	/* the connection_ready_changed signal handler is no longer needed. */
-	c_list_for_each_entry (sett_conn, &priv->connections_lst_head, _connections_lst)
-		g_signal_handlers_disconnect_by_func (sett_conn, G_CALLBACK (connection_ready_changed), self);
+	nm_clear_pointer (&priv->startup_complete_idx, g_hash_table_destroy);
+	nm_clear_g_signal_handler (priv->platform, &priv->startup_complete_platform_change_id);
 
-	priv->startup_complete = TRUE;
+ready:
+	_LOGT ("startup-complete: ready, no profiles to wait for");
+	nm_assert (priv->started);
+	nm_assert (!priv->startup_complete_blocked_by);
+	nm_assert (!priv->startup_complete_idx);
+	nm_assert (priv->startup_complete_timeout_id == 0);
+	nm_assert (priv->startup_complete_platform_change_id == 0);
 	_notify (self, PROP_STARTUP_COMPLETE);
 }
 
 static void
-connection_ready_changed (NMSettingsConnection *conn,
-                          GParamSpec *pspec,
-                          gpointer user_data)
+_startup_complete_notify_connection (NMSettings *self,
+                                     NMSettingsConnection *sett_conn,
+                                     gboolean forget)
 {
-	NMSettings *self = NM_SETTINGS (user_data);
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	gint64 timeout;
+	gint64 now_us = 0;
+
+	nm_assert (   !priv->started
+	           || priv->startup_complete_idx);
+
+	timeout = 0;
+	if (!forget) {
+		NMSettingConnection *s_con;
+		gint32 v;
+
+		s_con = nm_connection_get_setting_connection (nm_settings_connection_get_connection (sett_conn));
+		v = nm_setting_connection_get_wait_device_timeout (s_con);
+		if (v > 0) {
+			nm_assert (nm_setting_connection_get_interface_name (s_con));
+			timeout = ((gint64) v) * 1000;
+		}
+	}
 
-	if (nm_settings_connection_get_ready (conn))
-		check_startup_complete (self);
+	if (timeout == 0) {
+		if (   !priv->startup_complete_idx
+		    || !g_hash_table_remove (priv->startup_complete_idx, &sett_conn))
+			return;
+	} else {
+		StartupCompleteData *scd;
+
+		if (!priv->startup_complete_idx) {
+			nm_assert (!priv->started);
+			priv->startup_complete_idx = g_hash_table_new_full (nm_pdirect_hash,
+			                                                    nm_pdirect_equal,
+			                                                    NULL,
+			                                                    (GDestroyNotify) _startup_complete_data_destroy);
+			scd = NULL;
+		} else
+			scd = g_hash_table_lookup (priv->startup_complete_idx, &sett_conn);
+		if (!scd) {
+			now_us = nm_utils_get_monotonic_timestamp_us ();
+			scd = g_slice_new (StartupCompleteData);
+			*scd = (StartupCompleteData) {
+				.sett_conn = g_object_ref (sett_conn),
+				.start_at  = now_us,
+				.timeout   = timeout,
+			};
+			g_hash_table_add (priv->startup_complete_idx, scd);
+		} else {
+			if (scd->start_at == 0) {
+				/* the entry already is ready and no longer relevant. Ignore it. */
+				return;
+			}
+			scd->timeout = timeout;
+		}
+	}
+
+	_startup_complete_check (self, now_us);
 }
 
 const char *
@@ -228,10 +595,12 @@ nm_settings_get_startup_complete_blocked_reason (NMSettings *self)
 	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
 	const char *uuid = NULL;
 
-	if (priv->startup_complete)
-		return NULL;
-	if (priv->startup_complete_blocked_by)
-		uuid = nm_settings_connection_get_uuid (priv->startup_complete_blocked_by);
+	if (priv->started) {
+		if (!priv->startup_complete_idx)
+			return NULL;
+		if (priv->startup_complete_blocked_by)
+			uuid = nm_settings_connection_get_uuid (priv->startup_complete_blocked_by);
+	}
 	return uuid ?: "unknown";
 }
 
@@ -283,8 +652,8 @@ update_specs (NMSettings *self, GSList **specs_ptr,
 }
 
 static void
-unmanaged_specs_changed (NMSettingsPlugin *config,
-                         gpointer user_data)
+_plugin_unmanaged_specs_changed (NMSettingsPlugin *config,
+                                 gpointer user_data)
 {
 	NMSettings *self = NM_SETTINGS (user_data);
 	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
@@ -295,8 +664,8 @@ unmanaged_specs_changed (NMSettingsPlugin *config,
 }
 
 static void
-unrecognized_specs_changed (NMSettingsPlugin *config,
-                               gpointer user_data)
+_plugin_unrecognized_specs_changed (NMSettingsPlugin *config,
+                                    gpointer user_data)
 {
 	NMSettings *self = NM_SETTINGS (user_data);
 	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
@@ -308,320 +677,1224 @@ unrecognized_specs_changed (NMSettingsPlugin *config,
 /*****************************************************************************/
 
 static void
-plugin_connection_added (NMSettingsPlugin *config,
-                         NMSettingsConnection *connection,
-                         NMSettings *self)
+connection_flags_changed (NMSettingsConnection *sett_conn,
+                          gpointer user_data)
+{
+	_emit_connection_flags_changed (NM_SETTINGS (user_data), sett_conn);
+}
+
+/*****************************************************************************/
+
+static SettConnEntry *
+_sett_conn_entries_get (NMSettings *self,
+                        const char *uuid)
 {
-	claim_connection (self, connection);
+	nm_assert (uuid);
+	return g_hash_table_lookup (NM_SETTINGS_GET_PRIVATE (self)->sce_idx, &uuid);
+}
+
+static SettConnEntry *
+_sett_conn_entries_create_and_add (NMSettings *self,
+                                   const char *uuid)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	SettConnEntry *sett_conn_entry;
+
+	sett_conn_entry = _sett_conn_entry_new (uuid);
+
+	if (!g_hash_table_add (priv->sce_idx, sett_conn_entry))
+		nm_assert_not_reached ();
+	else if (g_hash_table_size (priv->sce_idx) == 1)
+		g_object_ref (self);
+
+	return sett_conn_entry;
 }
 
 static void
-load_connections (NMSettings *self)
+_sett_conn_entries_remove_and_destroy (NMSettings *self,
+                                       SettConnEntry *sett_conn_entry)
 {
 	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
-	GSList *iter;
 
-	for (iter = priv->plugins; iter; iter = g_slist_next (iter)) {
-		NMSettingsPlugin *plugin = NM_SETTINGS_PLUGIN (iter->data);
-		GSList *plugin_connections;
-		GSList *elt;
+	if (!g_hash_table_remove (priv->sce_idx, sett_conn_entry))
+		nm_assert_not_reached ();
+	else if (g_hash_table_size (priv->sce_idx) == 0)
+		g_object_unref (self);
+}
+
+/*****************************************************************************/
+
+static int
+_sett_conn_entry_sds_update_cmp (const CList *ls_a,
+                                 const CList *ls_b,
+                                 gconstpointer user_data)
+{
+	StorageData *sd_a = c_list_entry (ls_a, StorageData, sd_lst);
+	StorageData *sd_b = c_list_entry (ls_b, StorageData, sd_lst);
+
+	/* prioritized entries are sorted first (higher priority). */
+	NM_CMP_FIELD_UNSAFE (sd_b, sd_a, prioritize);
+
+	/* nm_settings_storage_cmp() compares in ascending order. Meaning,
+	 * if the storage has higher priority, it gives a positive number (as one
+	 * would expect).
+	 *
+	 * We want to sort the list in reverse though, with highest priority first. */
+	return nm_settings_storage_cmp (sd_b->storage, sd_a->storage, user_data);
+}
 
-		plugin_connections = nm_settings_plugin_get_connections (plugin);
+static void
+_sett_conn_entry_sds_update (NMSettings *self,
+                             SettConnEntry *sett_conn_entry)
+{
+	StorageData *sd;
+	StorageData *sd_safe;
+	StorageData *sd_dirty;
+	gboolean reprioritize;
+
+	nm_assert_storage_data_lst (&sett_conn_entry->sd_lst_head);
+	nm_assert_storage_data_lst (&sett_conn_entry->dirty_sd_lst_head);
+
+	/* we merge the dirty list with the previous list.
+	 *
+	 * The idea is:
+	 *
+	 *  - _connection_changed_track() appends events for the same UUID. Meaning:
+	 *    if the storage is new, it get appended (having lower priority).
+	 *    If it already exist and is an update for an event that we already
+	 *    track it, it keeps the list position in @dirty_sd_lst_head unchanged.
+	 *
+	 *  - during merge, we want to preserve the previous order (with higher
+	 *    priority first in the list).
+	 */
 
-		// FIXME: ensure connections from plugins loaded with a lower priority
-		// get rejected when they conflict with connections from a higher
-		// priority plugin.
+	/* first go through all storages that we track and check whether they
+	 * got an update...*/
 
-		for (elt = plugin_connections; elt; elt = g_slist_next (elt))
-			claim_connection (self, elt->data);
+	reprioritize = FALSE;
+	c_list_for_each_entry (sd, &sett_conn_entry->dirty_sd_lst_head, sd_lst) {
+		if (sd->prioritize) {
+			reprioritize = TRUE;
+			break;
+		}
+	}
 
-		g_slist_free (plugin_connections);
+	nm_assert_storage_data_lst (&sett_conn_entry->sd_lst_head);
 
-		g_signal_connect (plugin, NM_SETTINGS_PLUGIN_CONNECTION_ADDED,
-		                  G_CALLBACK (plugin_connection_added), self);
-		g_signal_connect (plugin, NM_SETTINGS_PLUGIN_UNMANAGED_SPECS_CHANGED,
-		                  G_CALLBACK (unmanaged_specs_changed), self);
-		g_signal_connect (plugin, NM_SETTINGS_PLUGIN_UNRECOGNIZED_SPECS_CHANGED,
-		                  G_CALLBACK (unrecognized_specs_changed), self);
+	c_list_for_each_entry_safe (sd, sd_safe, &sett_conn_entry->sd_lst_head, sd_lst) {
+
+		sd_dirty = _storage_data_find_in_lst (&sett_conn_entry->dirty_sd_lst_head, sd->storage);
+		if (!sd_dirty) {
+			/* there is no update for this storage (except maybe reprioritize). */
+			if (reprioritize)
+				sd->prioritize = FALSE;
+			continue;
+		}
+
+		nm_g_object_ref_set (&sd->connection, sd_dirty->connection);
+		sd->prioritize = sd_dirty->prioritize;
+
+		_storage_data_destroy (sd_dirty);
 	}
 
-	priv->connections_loaded = TRUE;
-	_notify (self, PROP_CONNECTIONS);
+	nm_assert_storage_data_lst (&sett_conn_entry->sd_lst_head);
+
+	/* all remaining (so far unseen) dirty entries are appended to the merged list.
+	 * (append means lower priority). */
+
+	c_list_splice (&sett_conn_entry->sd_lst_head, &sett_conn_entry->dirty_sd_lst_head);
+
+	nm_assert_storage_data_lst (&sett_conn_entry->sd_lst_head);
 
-	unmanaged_specs_changed (NULL, self);
-	unrecognized_specs_changed (NULL, self);
+	/* we drop the entries that are no longer "alive" (meaning, they no longer
+	 * indicate a connection and are not a tombstone). */
+	c_list_for_each_entry_safe (sd, sd_safe, &sett_conn_entry->sd_lst_head, sd_lst) {
+		if (!_storage_data_is_alive (sd))
+			_storage_data_destroy (sd);
+	}
+
+	nm_assert_storage_data_lst (&sett_conn_entry->sd_lst_head);
+	nm_assert (c_list_is_empty (&sett_conn_entry->dirty_sd_lst_head));
+
+	/* as last, we sort the entries. Note that this is a stable-sort... */
+	c_list_sort (&sett_conn_entry->sd_lst_head,
+	             _sett_conn_entry_sds_update_cmp,
+	             NM_SETTINGS_GET_PRIVATE (self)->plugins);
+
+	nm_assert_storage_data_lst (&sett_conn_entry->sd_lst_head);
+	nm_assert (c_list_is_empty (&sett_conn_entry->dirty_sd_lst_head));
 }
 
 /*****************************************************************************/
 
-static void
-connection_updated (NMSettingsConnection *connection, gboolean by_user, gpointer user_data)
+static NMConnection *
+_connection_changed_normalize_connection (NMSettingsStorage *storage,
+                                          NMConnection *connection,
+                                          GVariant *secrets_to_merge,
+                                          NMConnection **out_connection_cloned)
 {
-	g_signal_emit (NM_SETTINGS (user_data),
-	               signals[CONNECTION_UPDATED],
-	               0,
-	               connection,
-	               by_user);
+	gs_unref_object NMConnection *connection_cloned = NULL;
+	gs_free_error GError *error = NULL;
+	const char *uuid;
+
+	nm_assert (NM_IS_SETTINGS_STORAGE (storage));
+	nm_assert (out_connection_cloned && !*out_connection_cloned);
+
+	if (!connection)
+		return NULL;
+
+	nm_assert (NM_IS_CONNECTION (connection));
+
+	uuid = nm_settings_storage_get_uuid (storage);
+
+	if (secrets_to_merge) {
+		connection_cloned = nm_simple_connection_new_clone (connection);
+		connection = connection_cloned;
+		nm_connection_update_secrets (connection,
+		                              NULL,
+		                              secrets_to_merge,
+		                              NULL);
+	}
+
+	if (!_nm_connection_ensure_normalized (connection,
+	                                       !!connection_cloned,
+	                                       uuid,
+	                                       FALSE,
+	                                       connection_cloned ? NULL : &connection_cloned,
+	                                       &error)) {
+		/* this is most likely a bug in the plugin. It provided a connection that no longer verifies.
+		 * Well, I guess it could also happen when we merge @secrets_to_merge above. In any case
+		 * somewhere is a bug. */
+		_LOGT ("storage[%s,"NM_SETTINGS_STORAGE_PRINT_FMT"]: plugin provided an invalid connection: %s",
+		       uuid,
+		       NM_SETTINGS_STORAGE_PRINT_ARG (storage),
+		       error->message);
+		return NULL;
+	}
+	if (connection_cloned)
+		connection = connection_cloned;
+
+	*out_connection_cloned = g_steal_pointer (&connection_cloned);
+	return connection;
 }
 
+/*****************************************************************************/
+
 static void
-connection_flags_changed (NMSettingsConnection *connection,
-                          gpointer user_data)
+_connection_changed_update (NMSettings *self,
+                            SettConnEntry *sett_conn_entry,
+                            NMConnection *connection,
+                            NMSettingsConnectionIntFlags sett_flags,
+                            NMSettingsConnectionIntFlags sett_mask,
+                            NMSettingsConnectionUpdateReason update_reason)
 {
-	g_signal_emit (NM_SETTINGS (user_data),
-	               signals[CONNECTION_FLAGS_CHANGED],
-	               0,
-	               connection);
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	gs_unref_object NMConnection *connection_old = NULL;
+	NMSettingsStorage *storage = sett_conn_entry->storage;
+	gs_unref_object NMSettingsConnection *sett_conn = g_object_ref (sett_conn_entry->sett_conn);
+	const char *path;
+	gboolean is_new;
+
+	nm_assert (!NM_FLAGS_ANY (sett_mask, ~_NM_SETTINGS_CONNECTION_INT_FLAGS_PERSISTENT_MASK));
+	nm_assert (!NM_FLAGS_ANY (sett_flags, ~sett_mask));
+
+	is_new = c_list_is_empty (&sett_conn->_connections_lst);
+
+	_LOGT ("update[%s]: %s connection \"%s\" ("NM_SETTINGS_STORAGE_PRINT_FMT")",
+	       nm_settings_storage_get_uuid (storage),
+	       is_new ? "adding" : "updating",
+	       nm_connection_get_id (connection),
+	       NM_SETTINGS_STORAGE_PRINT_ARG (storage));
+
+	_nm_settings_connection_set_storage (sett_conn, storage);
+
+	_nm_settings_connection_set_connection (sett_conn, connection, &connection_old, update_reason);
+
+
+	if (is_new) {
+		_nm_settings_connection_register_kf_dbs (sett_conn,
+		                                         priv->kf_db_timestamps,
+		                                         priv->kf_db_seen_bssids);
+
+		_clear_connections_cached_list (priv);
+		c_list_link_tail (&priv->connections_lst_head, &sett_conn->_connections_lst);
+		priv->connections_len++;
+		priv->connections_generation++;
+
+		g_signal_connect (sett_conn, NM_SETTINGS_CONNECTION_FLAGS_CHANGED, G_CALLBACK (connection_flags_changed), self);
+
+	}
+
+	sett_mask |= NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE;
+	if (nm_settings_connection_check_visibility (sett_conn, priv->session_monitor))
+		sett_flags |= NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE;
+	else
+		nm_assert (!NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE));
+
+	sett_mask |= NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED;
+	if (nm_settings_storage_is_keyfile_run (storage))
+		sett_flags |= NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED;
+	else {
+		nm_assert (!NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_UNSAVED));
+
+		/* Profiles that don't reside in /run, are never nm-generated
+		 * and never volatile. */
+		sett_mask |= (  NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+		              | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE);
+		sett_flags &= ~(  NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+		                | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE);
+	}
+
+	nm_settings_connection_set_flags_full (sett_conn,
+	                                       sett_mask,
+	                                       sett_flags);
+
+	if (is_new) {
+		/* FIXME(shutdown): The NMSettings instance can't be disposed
+		 * while there is any exported connection. Ideally we should
+		 * unexport all connections on NMSettings' disposal, but for now
+		 * leak @self on termination when there are connections alive. */
+		path = nm_dbus_object_export (NM_DBUS_OBJECT (sett_conn));
+	} else
+		path = nm_dbus_object_get_path (NM_DBUS_OBJECT (sett_conn));
+
+	if (   is_new
+	    || connection_old) {
+		nm_utils_log_connection_diff (nm_settings_connection_get_connection (sett_conn),
+		                              connection_old,
+		                              LOGL_DEBUG,
+		                              LOGD_CORE,
+		                              is_new ? "new connection" : "update connection",
+		                              "++ ",
+		                              path);
+	}
+
+	if (is_new) {
+		nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
+		                            &interface_info_settings,
+		                            &signal_info_new_connection,
+		                            "(o)",
+		                            path);
+		_notify (self, PROP_CONNECTIONS);
+		_emit_connection_added (self, sett_conn);
+	} else {
+		_nm_settings_connection_emit_dbus_signal_updated (sett_conn);
+		_emit_connection_updated (self, sett_conn, update_reason);
+	}
+
+	if (   !priv->started
+	    || priv->startup_complete_idx) {
+		if (nm_settings_has_connection (self, sett_conn))
+			_startup_complete_notify_connection (self, sett_conn, FALSE);
+	}
 }
 
 static void
-connection_removed (NMSettingsConnection *connection, gpointer user_data)
+_connection_changed_delete (NMSettings *self,
+                            NMSettingsStorage *storage,
+                            NMSettingsConnection *sett_conn,
+                            gboolean allow_add_to_no_auto_default)
 {
-	NMSettings *self = NM_SETTINGS (user_data);
 	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	gs_unref_object NMConnection *connection_for_agents = NULL;
 	NMDevice *device;
+	const char *uuid;
 
-	g_return_if_fail (NM_IS_SETTINGS_CONNECTION (connection));
-	g_return_if_fail (!c_list_is_empty (&connection->_connections_lst));
-	nm_assert (c_list_contains (&priv->connections_lst_head, &connection->_connections_lst));
+	nm_assert (NM_IS_SETTINGS_CONNECTION (sett_conn));
+	nm_assert (c_list_contains (&priv->connections_lst_head, &sett_conn->_connections_lst));
+	nm_assert (nm_dbus_object_is_exported (NM_DBUS_OBJECT (sett_conn)));
+
+	uuid = nm_settings_storage_get_uuid (storage);
 
-	/* When the default wired connection is removed (either deleted or saved to
-	 * a new persistent connection by a plugin), write the MAC address of the
+	_LOGT ("update[%s]: delete connection \"%s\" ("NM_SETTINGS_STORAGE_PRINT_FMT")",
+	       uuid,
+	       nm_settings_connection_get_id (sett_conn),
+	       NM_SETTINGS_STORAGE_PRINT_ARG (storage));
+
+	/* When the default wired sett_conn is removed (either deleted or saved to
+	 * a new persistent sett_conn by a plugin), write the MAC address of the
 	 * wired device to the config file and don't create a new default wired
-	 * connection for that device again.
+	 * sett_conn for that device again.
 	 */
-	device = g_object_get_qdata (G_OBJECT (connection), _default_wired_device_quark ());
+	device = nm_settings_connection_default_wired_get_device (sett_conn);
 	if (device)
-		default_wired_clear_tag (self, device, connection, TRUE);
-
-	/* Disconnect signal handlers, as plugins might still keep references
-	 * to the connection (and thus the signal handlers would still be live)
-	 * even after NMSettings has dropped all its references.
-	 */
+		default_wired_clear_tag (self, device, sett_conn, allow_add_to_no_auto_default);
 
-	g_signal_handlers_disconnect_by_func (connection, G_CALLBACK (connection_removed), self);
-	g_signal_handlers_disconnect_by_func (connection, G_CALLBACK (connection_updated), self);
-	g_signal_handlers_disconnect_by_func (connection, G_CALLBACK (connection_flags_changed), self);
-	if (!priv->startup_complete)
-		g_signal_handlers_disconnect_by_func (connection, G_CALLBACK (connection_ready_changed), self);
+	g_signal_handlers_disconnect_by_func (sett_conn, G_CALLBACK (connection_flags_changed), self);
 
-	/* Forget about the connection internally */
 	_clear_connections_cached_list (priv);
+	c_list_unlink (&sett_conn->_connections_lst);
 	priv->connections_len--;
-	c_list_unlink (&connection->_connections_lst);
+	priv->connections_generation++;
 
-	if (priv->connections_loaded) {
-		_notify (self, PROP_CONNECTIONS);
+	/* Tell agents to remove secrets for this connection */
+	connection_for_agents = nm_simple_connection_new_clone (nm_settings_connection_get_connection (sett_conn));
+	nm_connection_clear_secrets (connection_for_agents);
+	nm_agent_manager_delete_secrets (priv->agent_mgr,
+	                                 nm_dbus_object_get_path (NM_DBUS_OBJECT (self)),
+	                                 connection_for_agents);
 
-		nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
-		                            &interface_info_settings,
-		                            &signal_info_connection_removed,
-		                            "(o)",
-		                            nm_dbus_object_get_path (NM_DBUS_OBJECT (connection)));
-	}
+	_notify (self, PROP_CONNECTIONS);
+	_nm_settings_connection_emit_dbus_signal_removed (sett_conn);
+	nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
+	                            &interface_info_settings,
+	                            &signal_info_connection_removed,
+	                            "(o)",
+	                            nm_dbus_object_get_path (NM_DBUS_OBJECT (sett_conn)));
 
-	nm_dbus_object_unexport (NM_DBUS_OBJECT (connection));
+	nm_dbus_object_unexport (NM_DBUS_OBJECT (sett_conn));
 
-	if (priv->connections_loaded)
-		g_signal_emit (self, signals[CONNECTION_REMOVED], 0, connection);
+	nm_settings_connection_set_flags (sett_conn,
+	                                    NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE
+	                                  | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE,
+	                                  FALSE);
 
-	check_startup_complete (self);
+	_emit_connection_removed (self, sett_conn);
 
-	g_object_unref (connection);
+	_nm_settings_connection_cleanup_after_remove (sett_conn);
 
-	g_object_unref (self);       /* Balanced by a ref in claim_connection() */
-}
+	nm_key_file_db_remove_key (priv->kf_db_timestamps, uuid);
+	nm_key_file_db_remove_key (priv->kf_db_seen_bssids, uuid);
 
-/*****************************************************************************/
+	if (   !priv->started
+	    || priv->startup_complete_idx)
+		_startup_complete_notify_connection (self, sett_conn, TRUE);
+}
 
 static void
-claim_connection (NMSettings *self, NMSettingsConnection *sett_conn)
+_connection_changed_process_one (NMSettings *self,
+                                 SettConnEntry *sett_conn_entry,
+                                 gboolean allow_add_to_no_auto_default,
+                                 NMSettingsConnectionIntFlags sett_flags,
+                                 NMSettingsConnectionIntFlags sett_mask,
+                                 gboolean override_sett_flags,
+                                 NMSettingsConnectionUpdateReason update_reason)
 {
-	NMSettingsPrivate *priv;
-	GError *error = NULL;
-	const char *path;
-	NMSettingsConnection *existing;
+	StorageData *sd_best;
 
-	g_return_if_fail (NM_IS_SETTINGS (self));
-	g_return_if_fail (NM_IS_SETTINGS_CONNECTION (sett_conn));
-	g_return_if_fail (!nm_dbus_object_is_exported (NM_DBUS_OBJECT (sett_conn)));
+	c_list_unlink (&sett_conn_entry->sce_dirty_lst);
 
-	priv = NM_SETTINGS_GET_PRIVATE (self);
+	_sett_conn_entry_sds_update (self, sett_conn_entry);
+
+	sd_best = c_list_first_entry (&sett_conn_entry->sd_lst_head, StorageData, sd_lst);;
+
+	if (   !sd_best
+	    || !sd_best->connection) {
+		gs_unref_object NMSettingsConnection *sett_conn = NULL;
+		gs_unref_object NMSettingsStorage *storage = NULL;
+
+		if (!sett_conn_entry->sett_conn) {
+
+			if (!sd_best) {
+				_sett_conn_entries_remove_and_destroy (self, sett_conn_entry);
+				return;
+			}
+
+			if (sett_conn_entry->storage != sd_best->storage) {
+				_LOGT ("update[%s]: shadow UUID ("NM_SETTINGS_STORAGE_PRINT_FMT")",
+				       sett_conn_entry->uuid,
+				       NM_SETTINGS_STORAGE_PRINT_ARG (sd_best->storage));
+			}
 
-	/* prevent duplicates */
-	if (!c_list_is_empty (&sett_conn->_connections_lst)) {
-		nm_assert (c_list_contains (&priv->connections_lst_head, &sett_conn->_connections_lst));
+			nm_g_object_ref_set (&sett_conn_entry->storage, sd_best->storage);
+			return;
+		}
+
+		sett_conn = g_steal_pointer (&sett_conn_entry->sett_conn);
+		if (sd_best) {
+			storage = g_object_ref (sd_best->storage);
+			nm_g_object_ref_set (&sett_conn_entry->storage, storage);
+			nm_assert_valid_settings_storage (NULL, storage);
+		} else {
+			storage = g_object_ref (sett_conn_entry->storage);
+			_sett_conn_entries_remove_and_destroy (self, sett_conn_entry);
+		}
+
+		_connection_changed_delete (self, storage, sett_conn, allow_add_to_no_auto_default);
 		return;
 	}
 
-	/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-	if (!nm_connection_normalize (nm_settings_connection_get_connection (sett_conn), NULL, NULL, &error)) {
-		_LOGW ("plugin provided invalid connection: %s", error->message);
-		g_error_free (error);
-		return;
+	if (override_sett_flags) {
+		NMSettingsConnectionIntFlags s_f, s_m;
+
+		nm_settings_storage_load_sett_flags (sd_best->storage, &s_f, &s_m);
+
+		nm_assert (!NM_FLAGS_ANY (s_f, ~s_m));
+
+		sett_mask |= s_m;
+		sett_flags = (sett_flags & ~s_m) | (s_f & s_m);
 	}
 
-	existing = nm_settings_get_connection_by_uuid (self, nm_settings_connection_get_uuid (sett_conn));
-	if (existing) {
-		/* Cannot add duplicate connections per UUID. Just return without action and
-		 * log a warning.
-		 *
-		 * This means, that plugins must not provide duplicate connections (UUID).
-		 * In fact, none of the plugins currently would do that.
-		 *
-		 * But globaly, over different setting plugins, there could be duplicates
-		 * without the individual plugins being aware. Don't handle that at all, just
-		 * error out. That should not happen unless the admin misconfigured the system
-		 * to create conflicting connections. */
-		_LOGW ("plugin provided duplicate connection with UUID %s",
-		       nm_settings_connection_get_uuid (sett_conn));
-		return;
+	nm_g_object_ref_set (&sett_conn_entry->storage, sd_best->storage);
+
+	if (!sett_conn_entry->sett_conn)
+		sett_conn_entry->sett_conn = nm_settings_connection_new ();
+
+	_connection_changed_update (self,
+	                            sett_conn_entry,
+	                            sd_best->connection,
+	                            sett_flags,
+	                            sett_mask,
+	                            update_reason);
+}
+
+static void
+_connection_changed_process_all_dirty (NMSettings *self,
+                                       gboolean allow_add_to_no_auto_default,
+                                       NMSettingsConnectionIntFlags sett_flags,
+                                       NMSettingsConnectionIntFlags sett_mask,
+                                       gboolean override_sett_flags,
+                                       NMSettingsConnectionUpdateReason update_reason)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	SettConnEntry *sett_conn_entry;
+
+	while ((sett_conn_entry = c_list_first_entry (&priv->sce_dirty_lst_head, SettConnEntry, sce_dirty_lst))) {
+		_connection_changed_process_one (self,
+		                                 sett_conn_entry,
+		                                 allow_add_to_no_auto_default,
+		                                 sett_flags,
+		                                 sett_mask,
+		                                 override_sett_flags,
+		                                 update_reason);
 	}
+}
 
-	nm_settings_connection_register_kf_dbs (sett_conn,
-	                                        priv->kf_db_timestamps,
-	                                        priv->kf_db_seen_bssids);
+static SettConnEntry *
+_connection_changed_track (NMSettings *self,
+                           NMSettingsStorage *storage,
+                           NMConnection *connection,
+                           gboolean prioritize)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	SettConnEntry *sett_conn_entry;
+	StorageData *sd;
+	const char *uuid;
 
-	/* Ensure its initial visibility is up-to-date */
-	nm_settings_connection_recheck_visibility (sett_conn);
+	nm_assert_valid_settings_storage (NULL, storage);
 
-	/* This one unexports the connection, it needs to run late to give the active
-	 * connection a chance to deal with its reference to this settings connection. */
-	g_signal_connect_after (sett_conn, NM_SETTINGS_CONNECTION_REMOVED,
-	                        G_CALLBACK (connection_removed), self);
-	g_signal_connect (sett_conn, NM_SETTINGS_CONNECTION_UPDATED_INTERNAL,
-	                  G_CALLBACK (connection_updated), self);
-	g_signal_connect (sett_conn, NM_SETTINGS_CONNECTION_FLAGS_CHANGED,
-	                  G_CALLBACK (connection_flags_changed),
-	                  self);
-	if (!priv->startup_complete) {
-		g_signal_connect (sett_conn, "notify::" NM_SETTINGS_CONNECTION_READY,
-		                  G_CALLBACK (connection_ready_changed),
-		                  self);
+	uuid = nm_settings_storage_get_uuid (storage);
+
+	nm_assert (!connection || NM_IS_CONNECTION (connection));
+	nm_assert (!connection || (_nm_connection_verify (connection, NULL) == NM_SETTING_VERIFY_SUCCESS));
+	nm_assert (!connection || nm_streq0 (uuid, nm_connection_get_uuid (connection)));
+
+	nmtst_connection_assert_unchanging (connection);
+
+	sett_conn_entry =    _sett_conn_entries_get (self, uuid)
+	                  ?: _sett_conn_entries_create_and_add (self, uuid);
+
+	if (_LOGT_ENABLED ()) {
+		const char *filename;
+
+		filename = nm_settings_storage_get_filename (storage);
+		if (connection) {
+			_LOGT ("storage[%s,"NM_SETTINGS_STORAGE_PRINT_FMT"]: change event with connection \"%s\"%s%s%s",
+			       sett_conn_entry->uuid,
+			       NM_SETTINGS_STORAGE_PRINT_ARG (storage),
+			       nm_connection_get_id (connection),
+			       NM_PRINT_FMT_QUOTED (filename, " (file \"", filename, "\")", ""));
+		} else if (nm_settings_storage_is_keyfile_tombstone (storage)) {
+			_LOGT ("storage[%s,"NM_SETTINGS_STORAGE_PRINT_FMT"]: change event for hiding profile%s%s%s",
+			       sett_conn_entry->uuid,
+			       NM_SETTINGS_STORAGE_PRINT_ARG (storage),
+			       NM_PRINT_FMT_QUOTED (filename, " (file \"", filename, "\")", ""));
+		} else {
+			_LOGT ("storage[%s,"NM_SETTINGS_STORAGE_PRINT_FMT"]: change event for dropping profile%s%s%s",
+			       sett_conn_entry->uuid,
+			       NM_SETTINGS_STORAGE_PRINT_ARG (storage),
+			       NM_PRINT_FMT_QUOTED (filename, " (file \"", filename, "\")", ""));
+		}
 	}
 
-	_clear_connections_cached_list (priv);
+	/* see _sett_conn_entry_sds_update() for why we append the new events
+	 * and leave existing ones at their position. */
+	sd = _storage_data_find_in_lst (&sett_conn_entry->dirty_sd_lst_head, storage);
+	if (sd)
+		nm_g_object_ref_set (&sd->connection, connection);
+	else {
+		sd = _storage_data_new_stale (storage, connection);
+		c_list_link_tail (&sett_conn_entry->dirty_sd_lst_head, &sd->sd_lst);
+	}
 
-	g_object_ref (sett_conn);
-	/* FIXME(shutdown): The NMSettings instance can't be disposed
-	 * while there is any exported connection. Ideally we should
-	 * unexport all connections on NMSettings' disposal, but for now
-	 * leak @self on termination when there are connections alive. */
-	g_object_ref (self);
-	priv->connections_len++;
-	c_list_link_tail (&priv->connections_lst_head, &sett_conn->_connections_lst);
-
-	path = nm_dbus_object_export (NM_DBUS_OBJECT (sett_conn));
-
-	nm_utils_log_connection_diff (nm_settings_connection_get_connection (sett_conn),
-	                              NULL,
-	                              LOGL_DEBUG,
-	                              LOGD_CORE,
-	                              "new connection", "++ ",
-	                              path);
-
-	/* Only emit the individual connection-added signal after connections
-	 * have been initially loaded.
-	 */
-	if (priv->connections_loaded) {
-		nm_dbus_object_emit_signal (NM_DBUS_OBJECT (self),
-		                            &interface_info_settings,
-		                            &signal_info_new_connection,
-		                            "(o)",
-		                            nm_dbus_object_get_path (NM_DBUS_OBJECT (sett_conn)));
+	if (prioritize) {
+		StorageData *sd2;
 
-		g_signal_emit (self, signals[CONNECTION_ADDED], 0, sett_conn);
-		_notify (self, PROP_CONNECTIONS);
+		/* only one entry can be prioritized. */
+		c_list_for_each_entry (sd2, &sett_conn_entry->dirty_sd_lst_head, sd_lst)
+			sd2->prioritize = FALSE;
+		sd->prioritize = TRUE;
+	}
+
+	nm_c_list_move_tail (&priv->sce_dirty_lst_head, &sett_conn_entry->sce_dirty_lst);
+
+	return sett_conn_entry;
+}
+
+/*****************************************************************************/
+
+static void
+_plugin_connections_reload_cb (NMSettingsPlugin *plugin,
+                               NMSettingsStorage *storage,
+                               NMConnection *connection,
+                               gpointer user_data)
+{
+	_connection_changed_track (user_data, storage, connection, FALSE);
+}
+
+static void
+_plugin_connections_reload (NMSettings *self)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	GSList *iter;
+
+	for (iter = priv->plugins; iter; iter = iter->next) {
+		nm_settings_plugin_reload_connections (iter->data,
+		                                       _plugin_connections_reload_cb,
+		                                       self);
 	}
 
-	nm_settings_connection_added (sett_conn);
+	_connection_changed_process_all_dirty (self,
+	                                       FALSE,
+	                                       NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                       NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                       TRUE,
+	                                         NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS
+	                                       | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS);
+
+	for (iter = priv->plugins; iter; iter = iter->next)
+		nm_settings_plugin_load_connections_done (iter->data);
 }
 
 /*****************************************************************************/
 
+static gboolean
+_add_connection_to_first_plugin (NMSettings *self,
+                                 NMConnection *new_connection,
+                                 gboolean in_memory,
+                                 gboolean is_nm_generated,
+                                 gboolean is_volatile,
+                                 NMSettingsStorage **out_new_storage,
+                                 NMConnection **out_new_connection,
+                                 GError **error)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	GError *first_error = NULL;
+	GSList *iter;
+	const char *uuid;
+
+	uuid = nm_connection_get_uuid (new_connection);
+
+	nm_assert (nm_utils_is_uuid (uuid));
+
+	for (iter = priv->plugins; iter; iter = iter->next) {
+		NMSettingsPlugin *plugin = NM_SETTINGS_PLUGIN (iter->data);
+		gs_unref_object NMSettingsStorage *storage = NULL;
+		gs_unref_object NMConnection *connection_to_add = NULL;
+		gs_unref_object NMConnection *connection_to_add_cloned = NULL;
+		NMConnection *connection_to_add_real = NULL;
+		gs_unref_variant GVariant *agent_owned_secrets = NULL;
+		gs_free_error GError *add_error = NULL;
+		gboolean success;
+		const char *filename;
+
+		if (plugin == (NMSettingsPlugin *) priv->keyfile_plugin) {
+			success = nms_keyfile_plugin_add_connection (priv->keyfile_plugin,
+			                                             new_connection,
+			                                             is_nm_generated,
+			                                             is_volatile,
+			                                             in_memory,
+			                                             &storage,
+			                                             &connection_to_add,
+			                                             &add_error);
+		} else {
+			if (in_memory)
+				continue;
+			nm_assert (!is_nm_generated);
+			nm_assert (!is_volatile);
+			success = nm_settings_plugin_add_connection (plugin,
+			                                             new_connection,
+			                                             &storage,
+			                                             &connection_to_add,
+			                                             &add_error);
+		}
+
+		if (!success) {
+			_LOGT ("add-connection: failed to add %s/'%s': %s",
+			       nm_connection_get_uuid (new_connection),
+			       nm_connection_get_id (new_connection),
+			       add_error->message);
+			if (!first_error)
+				first_error = g_steal_pointer (&add_error);
+			continue;
+		}
+
+		if (!nm_streq0 (nm_settings_storage_get_uuid (storage), uuid)) {
+			nm_assert_not_reached ();
+			continue;
+		}
+
+		agent_owned_secrets = nm_connection_to_dbus (new_connection,
+		                                               NM_CONNECTION_SERIALIZE_ONLY_SECRETS
+		                                             | NM_CONNECTION_SERIALIZE_WITH_SECRETS_AGENT_OWNED);
+		connection_to_add_real = _connection_changed_normalize_connection (storage,
+		                                                                   connection_to_add,
+		                                                                   agent_owned_secrets,
+		                                                                   &connection_to_add_cloned);
+		if (!connection_to_add_real) {
+			nm_assert_not_reached ();
+			continue;
+		}
+
+		filename = nm_settings_storage_get_filename (storage);
+		_LOGT ("add-connection: successfully added connection %s,'%s' ("NM_SETTINGS_STORAGE_PRINT_FMT"%s%s%s",
+		       nm_settings_storage_get_uuid (storage),
+		       nm_connection_get_id (new_connection),
+		       NM_SETTINGS_STORAGE_PRINT_ARG (storage),
+		       NM_PRINT_FMT_QUOTED (filename, ", \"", filename, "\")", ")"));
+
+		*out_new_storage = g_steal_pointer (&storage);
+		*out_new_connection =    g_steal_pointer (&connection_to_add_cloned)
+		                      ?: g_steal_pointer (&connection_to_add);
+		nm_assert (NM_IS_CONNECTION (*out_new_connection));
+		return TRUE;
+	}
+
+	nm_assert (first_error);
+	g_propagate_error (error, first_error);
+	return FALSE;
+}
+
 /**
  * nm_settings_add_connection:
  * @self: the #NMSettings object
  * @connection: the source connection to create a new #NMSettingsConnection from
- * @save_to_disk: %TRUE to save the connection to disk immediately, %FALSE to
- * not save to disk
+ * @persist_mode: the persist-mode for this profile.
+ * @sett_flags: the settings flags to set.
+ * @out_sett_conn: (allow-none) (transfer none): the added settings connection on success.
  * @error: on return, a location to store any errors that may occur
  *
  * Creates a new #NMSettingsConnection for the given source @connection.
  * The returned object is owned by @self and the caller must reference
  * the object to continue using it.
  *
- * Returns: the new #NMSettingsConnection or %NULL
+ * Returns: TRUE on success.
  */
-NMSettingsConnection *
+gboolean
 nm_settings_add_connection (NMSettings *self,
                             NMConnection *connection,
-                            gboolean save_to_disk,
+                            NMSettingsConnectionPersistMode persist_mode,
+                            NMSettingsConnectionIntFlags sett_flags,
+                            NMSettingsConnection **out_sett_conn,
                             GError **error)
 {
-	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
-	GSList *iter;
-	NMSettingsConnection *added = NULL;
-	NMSettingsConnection *candidate = NULL;
+	gs_unref_object NMConnection *connection_cloned_1 = NULL;
+	gs_unref_object NMConnection *new_connection = NULL;
+	gs_unref_object NMSettingsStorage *new_storage = NULL;
+	gs_free_error GError *local = NULL;
+	SettConnEntry *sett_conn_entry;
 	const char *uuid;
+	StorageData *sd;
+
+	nm_assert (NM_IN_SET (persist_mode, NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
+	                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY));
+
+	nm_assert (!NM_FLAGS_ANY (sett_flags, ~_NM_SETTINGS_CONNECTION_INT_FLAGS_PERSISTENT_MASK));
+
+	nm_assert (   !NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED)
+	           || persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY);
+
+	nm_assert (   !NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE)
+	           || persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY);
+
+	NM_SET_OUT (out_sett_conn, NULL);
 
 	uuid = nm_connection_get_uuid (connection);
 
 	/* Make sure a connection with this UUID doesn't already exist */
-	c_list_for_each_entry (candidate, &priv->connections_lst_head, _connections_lst) {
-		if (nm_streq0 (uuid, nm_settings_connection_get_uuid (candidate))) {
-			g_set_error_literal (error,
-			                     NM_SETTINGS_ERROR,
-			                     NM_SETTINGS_ERROR_UUID_EXISTS,
-			                     "A connection with this UUID already exists.");
-			return NULL;
+	if (_sett_conn_entry_get_conn (_sett_conn_entries_get (self, uuid))) {
+		g_set_error_literal (error,
+		                     NM_SETTINGS_ERROR,
+		                     NM_SETTINGS_ERROR_UUID_EXISTS,
+		                     "a connection with this UUID already exists");
+		return FALSE;
+	}
+
+	if (!_nm_connection_ensure_normalized (connection,
+	                                       FALSE,
+	                                       NULL,
+	                                       FALSE,
+	                                       &connection_cloned_1,
+	                                       &local)) {
+		g_set_error (error,
+		             NM_SETTINGS_ERROR,
+		             NM_SETTINGS_ERROR_INVALID_CONNECTION,
+		             "connection is invalid: %s",
+		             local->message);
+		return FALSE;
+	}
+	if (connection_cloned_1)
+		connection = connection_cloned_1;
+
+	if (!_add_connection_to_first_plugin (self,
+	                                      connection,
+	                                      (   persist_mode != NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK
+	                                       || NM_FLAGS_ANY (sett_flags,   NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE
+	                                                                    | NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED)),
+	                                      NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED),
+	                                      NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE),
+	                                      &new_storage,
+	                                      &new_connection,
+	                                      &local)) {
+		g_set_error (error,
+		             NM_SETTINGS_ERROR,
+		             NM_SETTINGS_ERROR_FAILED,
+		             "failure adding connection: %s",
+		             local->message);
+		return FALSE;
+	}
+
+	sett_conn_entry = _connection_changed_track (self, new_storage, new_connection, TRUE);
+
+	c_list_for_each_entry (sd, &sett_conn_entry->sd_lst_head, sd_lst) {
+
+		if (!nm_settings_storage_is_keyfile_tombstone (sd->storage))
+			continue;
+
+		if (nm_settings_storage_is_keyfile_run (sd->storage)) {
+			/* We remove this file from /run. */
+		} else {
+			if (nm_settings_storage_is_keyfile_run (new_storage)) {
+				/* Don't remove the file from /etc if we just wrote an in-memory connection */
+				continue;
+			}
 		}
+
+		nm_settings_plugin_delete_connection (nm_settings_storage_get_plugin (sd->storage),
+		                                      sd->storage,
+		                                      NULL);
+
+		nm_assert (!nm_settings_storage_is_keyfile_tombstone (sd->storage));
+
+		_connection_changed_track (self, sd->storage, NULL, FALSE);
 	}
 
-	/* 1) plugin writes the NMConnection to disk
-	 * 2) plugin creates a new NMSettingsConnection subclass with the settings
-	 *     from the NMConnection and returns it to the settings service
-	 * 3) settings service exports the new NMSettingsConnection subclass
-	 * 4) plugin notices that something on the filesystem has changed
-	 * 5) plugin reads the changes and ignores them because they will
-	 *     contain the same data as the connection it already knows about
-	 */
-	for (iter = priv->plugins; iter; iter = g_slist_next (iter)) {
-		NMSettingsPlugin *plugin = NM_SETTINGS_PLUGIN (iter->data);
-		GError *add_error = NULL;
-		gs_unref_variant GVariant *secrets = NULL;
-
-		/* Make a copy of agent-owned secrets because they won't be present in
-		 * the connection returned by plugins, as plugins return only what was
-		 * reread from the file. */
-		secrets = nm_connection_to_dbus (connection,
-		                                   NM_CONNECTION_SERIALIZE_ONLY_SECRETS
-		                                 | NM_CONNECTION_SERIALIZE_WITH_SECRETS_AGENT_OWNED);
-
-		added = nm_settings_plugin_add_connection (plugin, connection, save_to_disk, &add_error);
-		if (added) {
-			if (secrets) {
-				/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-				nm_connection_update_secrets (nm_settings_connection_get_connection (added),
-				                              NULL,
-				                              secrets,
-				                              NULL);
+	_connection_changed_process_all_dirty (self,
+	                                       FALSE,
+	                                       sett_flags,
+	                                       _NM_SETTINGS_CONNECTION_INT_FLAGS_PERSISTENT_MASK,
+	                                       FALSE,
+	                                         NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS
+	                                       | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS);
+
+	nm_assert (sett_conn_entry == _sett_conn_entries_get (self, sett_conn_entry->uuid));
+	nm_assert (NM_IS_SETTINGS_CONNECTION (sett_conn_entry->sett_conn));
+
+	NM_SET_OUT (out_sett_conn, _sett_conn_entry_get_conn (sett_conn_entry));
+	return TRUE;
+}
+
+/*****************************************************************************/
+
+gboolean
+nm_settings_update_connection (NMSettings *self,
+                               NMSettingsConnection *sett_conn,
+                               NMConnection *connection,
+                               NMSettingsConnectionPersistMode persist_mode,
+                               NMSettingsConnectionIntFlags sett_flags,
+                               NMSettingsConnectionIntFlags sett_mask,
+                               NMSettingsConnectionUpdateReason update_reason,
+                               const char *log_context_name,
+                               GError **error)
+{
+	NMSettingsPrivate *priv;
+	gs_unref_object NMConnection *connection_cloned_1 = NULL;
+	gs_unref_object NMConnection *new_connection_cloned = NULL;
+	gs_unref_object NMConnection *new_connection = NULL;
+	NMConnection *new_connection_real;
+	gs_unref_object NMSettingsStorage *cur_storage = NULL;
+	gs_unref_object NMSettingsStorage *new_storage = NULL;
+	gboolean cur_in_memory;
+	gboolean new_in_memory;
+	const char *uuid;
+
+	g_return_val_if_fail (NM_IS_SETTINGS (self), FALSE);
+	g_return_val_if_fail (NM_IS_SETTINGS_CONNECTION (sett_conn), FALSE);
+	g_return_val_if_fail (!connection || NM_IS_CONNECTION (connection), FALSE);
+
+	nm_assert (!NM_FLAGS_ANY (sett_mask, ~_NM_SETTINGS_CONNECTION_INT_FLAGS_PERSISTENT_MASK));
+	nm_assert (!NM_FLAGS_ANY (sett_flags, ~sett_mask));
+	nm_assert (NM_IN_SET (persist_mode, NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP,
+	                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST,
+	                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK,
+	                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED,
+	                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY));
+
+	priv = NM_SETTINGS_GET_PRIVATE (self);
+
+	cur_storage = g_object_ref (nm_settings_connection_get_storage (sett_conn));
+
+	uuid = nm_settings_storage_get_uuid (cur_storage);
+
+	nm_assert (NM_IS_SETTINGS_STORAGE (cur_storage));
+	nm_assert (_sett_conn_entry_get_conn (_sett_conn_entries_get (self, uuid)) == sett_conn);
+
+	if (connection) {
+		gs_free_error GError *local = NULL;
+
+		if (!_nm_connection_ensure_normalized (connection,
+		                                       FALSE,
+		                                       uuid,
+		                                       TRUE,
+		                                       &connection_cloned_1,
+		                                       &local)) {
+			_LOGT ("update[%s]: %s: failed because profile is invalid: %s",
+			       nm_settings_storage_get_uuid (cur_storage),
+			       log_context_name,
+			       local->message);
+			g_set_error (error,
+			             NM_SETTINGS_ERROR,
+			             NM_SETTINGS_ERROR_INVALID_CONNECTION,
+			             "connection is invalid: %s",
+			             local->message);
+			return FALSE;
+		}
+		if (connection_cloned_1)
+			connection = connection_cloned_1;
+	} else
+		connection = nm_settings_connection_get_connection (sett_conn);
+
+	cur_in_memory = nm_settings_storage_is_keyfile_run (cur_storage);
+
+	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP) {
+		persist_mode =   cur_in_memory
+		               ? NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED
+		               : NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
+	}
+
+	if (   NM_FLAGS_HAS (sett_mask, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED)
+	    && !NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED)) {
+		NMDevice *device;
+
+		/* The connection has been changed by the user, it should no longer be
+		 * considered a default wired connection, and should no longer affect
+		 * the no-auto-default configuration option.
+		 */
+		device = nm_settings_connection_default_wired_get_device (sett_conn);
+		if (device) {
+			nm_assert (cur_in_memory);
+			nm_assert (!NM_FLAGS_ANY (nm_settings_connection_get_flags (sett_conn),
+			                            NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+			                          | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE));
+
+			default_wired_clear_tag (self, device, sett_conn, FALSE);
+
+			if (NM_IN_SET (persist_mode, NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP,
+			                             NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST)) {
+				/* making a default-wired-connection a regulard connection implies persisting
+				 * it to disk (unless specified differently). */
+				persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK;
 			}
-			claim_connection (self, added);
-			return added;
 		}
-		_LOGD ("Failed to add %s/'%s': %s",
-		       nm_connection_get_uuid (connection),
-		       nm_connection_get_id (connection),
-		       add_error->message);
-		g_clear_error (&add_error);
 	}
 
-	g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-	                     "No plugin supported adding this connection");
-	return NULL;
+	if (   persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST
+	    && NM_FLAGS_ANY (sett_mask,   NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+	                                | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE)
+	    && NM_FLAGS_ANY ((sett_flags ^ nm_settings_connection_get_flags (sett_conn)) & sett_mask,
+	                       NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+	                     | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE)) {
+		/* we update the nm-generated/volatile setting of a profile (which is inherrently
+		 * in-memory. The caller did not request to persist this to disk, however we need
+		 * to store the flags in run. */
+		nm_assert (cur_in_memory);
+		persist_mode = NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED;
+	}
+
+	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK)
+		new_in_memory = FALSE;
+	else if (NM_IN_SET (persist_mode, NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED,
+	                                  NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY))
+		new_in_memory = TRUE;
+	else {
+		nm_assert (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST);
+		new_in_memory = cur_in_memory;
+	}
+
+	if (!new_in_memory) {
+		/* Persistent connections cannot be volatile nor nm-generated.
+		 *
+		 * That is obviously true for volatile, as it is enforced by Update2() API.
+		 *
+		 * For nm-generated profiles also, because the nm-generated flag is only stored
+		 * for in-memory profiles. If we would persist the profile to /etc it would loose
+		 * the nm-generated flag after restart/reload, and that cannot be right. If a profile
+		 * ends up on disk, the information who created it gets lost. */
+		nm_assert (!NM_FLAGS_ANY (sett_flags,   NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+		                                      | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE));
+		sett_mask |=   NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+		             | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE;
+		sett_flags &= ~(  NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+		                | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE);
+	}
+
+
+	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST) {
+		new_storage = g_object_ref (cur_storage);
+		new_connection = g_object_ref (connection);
+		_LOGT ("update[%s]: %s: update profile \"%s\" (not persisted)",
+		       nm_settings_storage_get_uuid (cur_storage),
+		       log_context_name,
+		       nm_connection_get_id (connection));
+	} else {
+		gboolean success;
+		gboolean migrate_storage;
+		gs_free_error GError *local = NULL;
+
+		if (new_in_memory != cur_in_memory)
+			migrate_storage = TRUE;
+		else if  (   !new_in_memory
+		          && nm_settings_storage_is_keyfile_lib (cur_storage)) {
+			/* the profile is a keyfile in /usr/lib. It cannot be overwritten, we must migrate it
+			 * from /usr/lib to /etc. */
+			migrate_storage = TRUE;
+		} else
+			migrate_storage = FALSE;
+
+		if (migrate_storage) {
+			success = _add_connection_to_first_plugin (self,
+			                                           connection,
+			                                           new_in_memory,
+			                                           NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED),
+			                                           NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE),
+			                                           &new_storage,
+			                                           &new_connection,
+			                                           &local);
+		} else {
+			NMSettingsPlugin *plugin;
+
+			plugin = nm_settings_storage_get_plugin (cur_storage);
+			if (plugin == (NMSettingsPlugin *) priv->keyfile_plugin) {
+				success = nms_keyfile_plugin_update_connection (priv->keyfile_plugin,
+				                                                cur_storage,
+				                                                connection,
+				                                                NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED),
+				                                                NM_FLAGS_HAS (sett_flags, NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE),
+				                                                NM_FLAGS_HAS (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_FORCE_RENAME),
+				                                                &new_storage,
+				                                                &new_connection,
+				                                                &local);
+			} else {
+				success = nm_settings_plugin_update_connection (nm_settings_storage_get_plugin (cur_storage),
+				                                                cur_storage,
+				                                                connection,
+				                                                &new_storage,
+				                                                &new_connection,
+				                                                &local);
+			}
+		}
+		if (!success) {
+			gboolean ignore_failure;
+
+			ignore_failure = NM_FLAGS_ANY (update_reason, NM_SETTINGS_CONNECTION_UPDATE_REASON_IGNORE_PERSIST_FAILURE);
+
+			_LOGT ("update[%s]: %s: %sfailure to %s connection \"%s\" on storage: %s",
+			       nm_settings_storage_get_uuid (cur_storage),
+			       log_context_name,
+			       ignore_failure ? "ignore " : "",
+			       migrate_storage ? "write" : "update",
+			       nm_connection_get_id (connection),
+			       local->message);
+			if (!ignore_failure) {
+				g_set_error (error,
+				             NM_SETTINGS_ERROR,
+				             NM_SETTINGS_ERROR_INVALID_CONNECTION,
+				             "failed to %s connection: %s",
+				             migrate_storage ? "write" : "update",
+				             local->message);
+				return FALSE;
+			}
+			new_storage = g_object_ref (cur_storage);
+			new_connection = g_object_ref (connection);
+		} else {
+			_LOGT ("update[%s]: %s: %s profile \"%s\"",
+			       nm_settings_storage_get_uuid (cur_storage),
+			       log_context_name,
+			       migrate_storage ? "write" : "update",
+			       nm_connection_get_id (connection));
+		}
+	}
+
+	nm_assert_valid_settings_storage (NULL, new_storage);
+	nm_assert (NM_IS_CONNECTION (new_connection));
+	nm_assert (nm_streq (uuid, nm_settings_storage_get_uuid (new_storage)));
+
+	if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_NO_PERSIST)
+		new_connection_real = new_connection;
+	else {
+		gs_unref_variant GVariant *agent_owned_secrets = NULL;
+
+		agent_owned_secrets = nm_connection_to_dbus (connection,
+		                                               NM_CONNECTION_SERIALIZE_ONLY_SECRETS
+		                                             | NM_CONNECTION_SERIALIZE_WITH_SECRETS_AGENT_OWNED);
+		new_connection_real = _connection_changed_normalize_connection (new_storage,
+		                                                                new_connection,
+		                                                                agent_owned_secrets,
+		                                                                &new_connection_cloned);
+		if (!new_connection_real) {
+			nm_assert_not_reached ();
+			new_connection_real = new_connection;
+		}
+	}
+
+	nm_assert (NM_IS_CONNECTION (new_connection_real));
+
+	_connection_changed_track (self, new_storage, new_connection_real, TRUE);
+
+	if (new_storage != cur_storage) {
+		gs_free_error GError *local = NULL;
+		gboolean remove_from_disk;
+
+		if (persist_mode == NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_DETACHED)
+			remove_from_disk = FALSE;
+		else if (nm_settings_storage_is_keyfile_lib (cur_storage))
+			remove_from_disk = FALSE;
+		else
+			remove_from_disk = TRUE;
+
+		if (remove_from_disk) {
+			if (!nm_settings_plugin_delete_connection (nm_settings_storage_get_plugin (cur_storage),
+			                                           cur_storage,
+			                                           &local)) {
+				const char *filename;
+
+				filename = nm_settings_storage_get_filename (cur_storage);
+				_LOGW ("update[%s]: failed to delete moved storage "NM_SETTINGS_STORAGE_PRINT_FMT"%s%s%s: %s",
+				       nm_settings_storage_get_uuid (cur_storage),
+				       NM_SETTINGS_STORAGE_PRINT_ARG (cur_storage),
+				       local->message,
+				       NM_PRINT_FMT_QUOTED (filename, " (file \"", filename, "\")", ""));
+			}
+
+			_connection_changed_track (self, cur_storage, NULL, FALSE);
+		}
+	}
+
+	_connection_changed_process_all_dirty (self,
+	                                       FALSE,
+	                                       sett_flags,
+	                                       sett_mask,
+	                                       FALSE,
+	                                       update_reason);
+
+	return TRUE;
 }
 
+void
+nm_settings_delete_connection (NMSettings *self,
+                               NMSettingsConnection *sett_conn,
+                               gboolean allow_add_to_no_auto_default)
+{
+	NMSettingsPrivate *priv;
+	NMSettingsStorage *cur_storage;
+	gs_free_error GError *local = NULL;
+	SettConnEntry *sett_conn_entry = NULL;
+	const char *uuid;
+	gboolean delete;
+	gboolean tombstone_in_memory = FALSE;
+	gboolean tombstone_on_disk = FALSE;
+	gs_unref_object NMSettingsStorage *tombstone_1_storage = NULL;
+	gs_unref_object NMSettingsStorage *tombstone_2_storage = NULL;
+
+	g_return_if_fail (NM_IS_SETTINGS (self));
+	g_return_if_fail (NM_IS_SETTINGS_CONNECTION (sett_conn));
+	g_return_if_fail (nm_settings_has_connection (self, sett_conn));
+
+	priv = NM_SETTINGS_GET_PRIVATE (self);
+
+	cur_storage = nm_settings_connection_get_storage (sett_conn);
+
+	nm_assert (NM_IS_SETTINGS_STORAGE (cur_storage));
+
+	uuid = nm_settings_storage_get_uuid (cur_storage);
+	nm_assert (nm_utils_is_uuid (uuid));
+
+	sett_conn_entry = _sett_conn_entries_get (self, uuid);
+
+	g_return_if_fail (sett_conn_entry);
+	nm_assert (sett_conn_entry->sett_conn == sett_conn);
+	nm_assert (sett_conn_entry->storage == cur_storage);
+
+	if (NMS_IS_KEYFILE_STORAGE (cur_storage)) {
+		NMSKeyfileStorage *s = NMS_KEYFILE_STORAGE (cur_storage);
+
+		if (NM_IN_SET (s->storage_type, NMS_KEYFILE_STORAGE_TYPE_RUN,
+		                                NMS_KEYFILE_STORAGE_TYPE_ETC))
+			delete = TRUE;
+		else {
+			tombstone_on_disk = TRUE;
+			delete = FALSE;
+		}
+	} else
+		delete = TRUE;
+
+	if (delete) {
+		if (!nm_settings_plugin_delete_connection (nm_settings_storage_get_plugin (cur_storage),
+		                                           cur_storage,
+		                                           &local)) {
+			_LOGW ("delete-connection: failed to delete storage "NM_SETTINGS_STORAGE_PRINT_FMT": %s",
+			       NM_SETTINGS_STORAGE_PRINT_ARG (cur_storage),
+			       local->message);
+			g_clear_error (&local);
+			/* there is no aborting back form this. We must get rid of the connection and
+			 * cannot do better than warn. Proceed... */
+			tombstone_in_memory = TRUE;
+		}
+		_connection_changed_track (self, cur_storage, NULL, FALSE);
+	}
+
+	if (tombstone_on_disk) {
+		if (!nms_keyfile_plugin_set_nmmeta_tombstone (priv->keyfile_plugin,
+		                                              FALSE,
+		                                              uuid,
+		                                              FALSE,
+		                                              TRUE,
+		                                              &tombstone_1_storage,
+		                                              NULL))
+			tombstone_in_memory = TRUE;
+		if (tombstone_1_storage)
+			_connection_changed_track (self, tombstone_1_storage, NULL, FALSE);
+	}
+
+	if (tombstone_in_memory) {
+		if (!nms_keyfile_plugin_set_nmmeta_tombstone (priv->keyfile_plugin,
+		                                              FALSE,
+		                                              uuid,
+		                                              TRUE,
+		                                              TRUE,
+		                                              &tombstone_2_storage,
+		                                              NULL)) {
+			nms_keyfile_plugin_set_nmmeta_tombstone (priv->keyfile_plugin,
+			                                         TRUE,
+			                                         uuid,
+			                                         TRUE,
+			                                         TRUE,
+			                                         &tombstone_2_storage,
+			                                         NULL);
+		}
+		_connection_changed_track (self, tombstone_2_storage, NULL, FALSE);
+	}
+
+	_connection_changed_process_all_dirty (self,
+	                                       allow_add_to_no_auto_default,
+	                                       NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                       NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+	                                       FALSE,
+	                                       NM_SETTINGS_CONNECTION_UPDATE_REASON_NONE);
+}
+
+/*****************************************************************************/
+
 static void
 send_agent_owned_secrets (NMSettings *self,
                           NMSettingsConnection *sett_conn,
@@ -657,7 +1930,6 @@ pk_add_cb (NMAuthChain *chain,
 	gpointer callback_data;
 	NMAuthSubject *subject;
 	const char *perm;
-	gboolean save_to_disk;
 
 	nm_assert (G_IS_DBUS_METHOD_INVOCATION (context));
 
@@ -675,10 +1947,14 @@ pk_add_cb (NMAuthChain *chain,
 	} else {
 		/* Authorized */
 		connection = nm_auth_chain_get_data (chain, "connection");
-		nm_assert (connection);
+		nm_assert (NM_IS_CONNECTION (connection));
 
-		save_to_disk = GPOINTER_TO_UINT (nm_auth_chain_get_data (chain, "save-to-disk"));
-		added = nm_settings_add_connection (self, connection, save_to_disk, &error);
+		nm_settings_add_connection (self,
+		                            connection,
+		                            GPOINTER_TO_UINT (nm_auth_chain_get_data (chain, "persist-mode")),
+		                            GPOINTER_TO_UINT (nm_auth_chain_get_data (chain, "sett-flags")),
+		                            &added,
+		                            &error);
 
 		/* The callback may remove the connection from the settings manager (e.g.
 		 * because it's found to be incompatible with the device on AddAndActivate).
@@ -694,8 +1970,7 @@ pk_add_cb (NMAuthChain *chain,
 	callback (self, added, error, context, subject, callback_data);
 
 	/* Send agent-owned secrets to the agents */
-	if (   !error
-	    && added
+	if (   added
 	    && nm_settings_has_connection (self, added))
 		send_agent_owned_secrets (self, added, subject);
 }
@@ -703,7 +1978,8 @@ pk_add_cb (NMAuthChain *chain,
 void
 nm_settings_add_connection_dbus (NMSettings *self,
                                  NMConnection *connection,
-                                 gboolean save_to_disk,
+                                 NMSettingsConnectionPersistMode persist_mode,
+                                 NMSettingsConnectionIntFlags sett_flags,
                                  NMAuthSubject *subject,
                                  GDBusMethodInvocation *context,
                                  NMSettingsAddCallback callback,
@@ -719,6 +1995,8 @@ nm_settings_add_connection_dbus (NMSettings *self,
 	g_return_if_fail (NM_IS_AUTH_SUBJECT (subject));
 	g_return_if_fail (G_IS_DBUS_METHOD_INVOCATION (context));
 
+	nm_assert (!NM_FLAGS_ANY (sett_flags, ~_NM_SETTINGS_CONNECTION_INT_FLAGS_PERSISTENT_MASK));
+
 	/* Connection must be valid, of course */
 	if (_nm_connection_verify (connection, &tmp_error) != NM_SETTING_VERIFY_SUCCESS) {
 		error = g_error_new (NM_SETTINGS_ERROR,
@@ -774,7 +2052,8 @@ nm_settings_add_connection_dbus (NMSettings *self,
 	nm_auth_chain_set_data (chain, "callback", callback, NULL);
 	nm_auth_chain_set_data (chain, "callback-data", user_data, NULL);
 	nm_auth_chain_set_data (chain, "subject", g_object_ref (subject), g_object_unref);
-	nm_auth_chain_set_data (chain, "save-to-disk", GUINT_TO_POINTER (save_to_disk), NULL);
+	nm_auth_chain_set_data (chain, "persist-mode", GUINT_TO_POINTER (persist_mode), NULL);
+	nm_auth_chain_set_data (chain, "sett-flags", GUINT_TO_POINTER (sett_flags), NULL);
 	nm_auth_chain_add_call_unsafe (chain, perm, TRUE);
 	return;
 
@@ -808,7 +2087,7 @@ static void
 settings_add_connection_helper (NMSettings *self,
                                 GDBusMethodInvocation *context,
                                 GVariant *settings,
-                                gboolean save_to_disk)
+                                NMSettingsConnectionPersistMode persist_mode)
 {
 	gs_unref_object NMConnection *connection = NULL;
 	GError *error = NULL;
@@ -836,7 +2115,8 @@ settings_add_connection_helper (NMSettings *self,
 
 	nm_settings_add_connection_dbus (self,
 	                                 connection,
-	                                 save_to_disk,
+	                                 persist_mode,
+	                                 NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
 	                                 subject,
 	                                 context,
 	                                 settings_add_connection_add_cb,
@@ -856,7 +2136,7 @@ impl_settings_add_connection (NMDBusObject *obj,
 	gs_unref_variant GVariant *settings = NULL;
 
 	g_variant_get (parameters, "(@a{sa{sv}})", &settings);
-	settings_add_connection_helper (self, invocation, settings, TRUE);
+	settings_add_connection_helper (self, invocation, settings, NM_SETTINGS_CONNECTION_PERSIST_MODE_DISK);
 }
 
 static void
@@ -872,14 +2152,16 @@ impl_settings_add_connection_unsaved (NMDBusObject *obj,
 	gs_unref_variant GVariant *settings = NULL;
 
 	g_variant_get (parameters, "(@a{sa{sv}})", &settings);
-	settings_add_connection_helper (self, invocation, settings, FALSE);
+	settings_add_connection_helper (self, invocation, settings, NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY);
 }
 
+/*****************************************************************************/
+
 static void
 impl_settings_load_connections (NMDBusObject *obj,
                                 const NMDBusInterfaceInfoExtended *interface_info,
                                 const NMDBusMethodInfoExtended *method_info,
-                                GDBusConnection *connection,
+                                GDBusConnection *dbus_connection,
                                 const char *sender,
                                 GDBusMethodInvocation *invocation,
                                 GVariant *parameters)
@@ -903,30 +2185,53 @@ impl_settings_load_connections (NMDBusObject *obj,
 	                                 NM_SETTINGS_ERROR_PERMISSION_DENIED))
 		return;
 
-	if (filenames) {
+	if (   filenames
+	    && filenames[0]) {
+		NMSettingsPluginConnectionLoadEntry *entries;
+		gsize n_entries;
 		gsize i;
+		GSList *iter;
 
-		for (i = 0; filenames[i]; i++) {
-			GSList *iter;
+		entries = nm_settings_plugin_create_connection_load_entries (filenames, &n_entries);
 
-			if (filenames[i][0] != '/')
-				_LOGW ("load: connection filename '%s' is not an absolute path", filenames[i]);
-			else {
-				for (iter = priv->plugins; iter; iter = iter->next) {
-					NMSettingsPlugin *plugin = NM_SETTINGS_PLUGIN (iter->data);
+		for (iter = priv->plugins; iter; iter = iter->next) {
+			NMSettingsPlugin *plugin = iter->data;
 
-					if (nm_settings_plugin_load_connection (plugin, filenames[i]))
-						goto next_filename;
-				}
-			}
+			nm_settings_plugin_load_connections (plugin,
+			                                     entries,
+			                                     n_entries,
+			                                     _plugin_connections_reload_cb,
+			                                     self);
+		}
+
+		for (i = 0; i < n_entries; i++) {
+			NMSettingsPluginConnectionLoadEntry *entry = &entries[i];
+
+			if (!entry->handled)
+				_LOGW ("load: no settings plugin could load \"%s\"", entry->filename);
+			else if (entry->error) {
+				_LOGW ("load: failure to load \"%s\": %s", entry->filename, entry->error->message);
+				g_clear_error (&entry->error);
+			} else
+				continue;
 
 			if (!failures)
 				failures = g_ptr_array_new ();
-			g_ptr_array_add (failures, (char *) filenames[i]);
-
-next_filename:
-			;
+			g_ptr_array_add (failures, (char *) entry->filename);
 		}
+
+		nm_clear_g_free (&entries);
+
+		_connection_changed_process_all_dirty (self,
+		                                       TRUE,
+		                                       NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+		                                       NM_SETTINGS_CONNECTION_INT_FLAGS_NONE,
+		                                       TRUE,
+		                                         NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_SYSTEM_SECRETS
+		                                       | NM_SETTINGS_CONNECTION_UPDATE_REASON_RESET_AGENT_SECRETS);
+
+		for (iter = priv->plugins; iter; iter = iter->next)
+			nm_settings_plugin_load_connections_done (iter->data);
 	}
 
 	if (failures)
@@ -957,8 +2262,6 @@ impl_settings_reload_connections (NMDBusObject *obj,
                                   GVariant *parameters)
 {
 	NMSettings *self = NM_SETTINGS (obj);
-	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
-	GSList *iter;
 
 	/* The permission is already enforced by the D-Bus daemon, but we ensure
 	 * that the caller is still alive so that clients are forced to wait and
@@ -971,11 +2274,7 @@ impl_settings_reload_connections (NMDBusObject *obj,
 	                                 NM_SETTINGS_ERROR_PERMISSION_DENIED))
 		return;
 
-	for (iter = priv->plugins; iter; iter = g_slist_next (iter)) {
-		NMSettingsPlugin *plugin = NM_SETTINGS_PLUGIN (iter->data);
-
-		nm_settings_plugin_reload_connections (plugin);
-	}
+	_plugin_connections_reload (self);
 
 	nm_audit_log_connection_op (NM_AUDIT_OP_CONNS_RELOAD, NULL, TRUE, NULL, invocation, NULL);
 
@@ -1029,20 +2328,24 @@ impl_settings_list_connections (NMDBusObject *obj,
 NMSettingsConnection *
 nm_settings_get_connection_by_uuid (NMSettings *self, const char *uuid)
 {
-	NMSettingsPrivate *priv;
-	NMSettingsConnection *candidate;
-
 	g_return_val_if_fail (NM_IS_SETTINGS (self), NULL);
 	g_return_val_if_fail (uuid != NULL, NULL);
 
-	priv = NM_SETTINGS_GET_PRIVATE (self);
+	return _sett_conn_entry_get_conn (_sett_conn_entries_get (self, uuid));
+}
 
-	c_list_for_each_entry (candidate, &priv->connections_lst_head, _connections_lst) {
-		if (nm_streq (uuid, nm_settings_connection_get_uuid (candidate)))
-			return candidate;
-	}
+const char *
+nm_settings_get_dbus_path_for_uuid (NMSettings *self,
+                                    const char *uuid)
+{
+	NMSettingsConnection *sett_conn;
 
-	return NULL;
+	sett_conn = nm_settings_get_connection_by_uuid (self, uuid);
+
+	if (!sett_conn)
+		return NULL;
+
+	return nm_dbus_object_get_path (NM_DBUS_OBJECT (sett_conn));
 }
 
 static void
@@ -1259,6 +2562,9 @@ add_plugin (NMSettings *self,
 	nm_assert (NM_IS_SETTINGS (self));
 	nm_assert (NM_IS_SETTINGS_PLUGIN (plugin));
 
+	nm_assert (pname);
+	nm_assert (nm_streq0 (pname, nm_settings_plugin_get_plugin_name (plugin)));
+
 	priv = NM_SETTINGS_GET_PRIVATE (self);
 
 	nm_assert (!g_slist_find (priv->plugins, plugin));
@@ -1514,6 +2820,13 @@ have_connection_for_device (NMSettings *self, NMDevice *device)
 		if (!nm_device_check_connection_compatible (device, connection, NULL))
 			continue;
 
+		if (nm_settings_connection_default_wired_get_device (sett_conn))
+			continue;
+
+		if (NM_FLAGS_ANY (nm_settings_connection_get_flags (sett_conn),
+		                  NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE))
+			continue;
+
 		iface = nm_setting_connection_get_interface_name (s_con);
 		if (!nm_streq0 (iface, nm_device_get_iface (device)))
 			continue;
@@ -1545,38 +2858,20 @@ have_connection_for_device (NMSettings *self, NMDevice *device)
 }
 
 static void
-default_wired_connection_updated_by_user_cb (NMSettingsConnection *connection, gboolean by_user, NMSettings *self)
-{
-	NMDevice *device;
-
-	if (!by_user)
-		return;
-
-	/* The connection has been changed by the user, it should no longer be
-	 * considered a default wired connection, and should no longer affect
-	 * the no-auto-default configuration option.
-	 */
-	device = g_object_get_qdata (G_OBJECT (connection), _default_wired_device_quark ());
-	if (device)
-		default_wired_clear_tag (self, device, connection, FALSE);
-}
-
-static void
 default_wired_clear_tag (NMSettings *self,
                          NMDevice *device,
-                         NMSettingsConnection *connection,
+                         NMSettingsConnection *sett_conn,
                          gboolean add_to_no_auto_default)
 {
 	nm_assert (NM_IS_SETTINGS (self));
 	nm_assert (NM_IS_DEVICE (device));
-	nm_assert (NM_IS_SETTINGS_CONNECTION (connection));
-	nm_assert (device == g_object_get_qdata (G_OBJECT (connection), _default_wired_device_quark ()));
-	nm_assert (connection == g_object_get_qdata (G_OBJECT (device), _default_wired_connection_quark ()));
+	nm_assert (NM_IS_SETTINGS_CONNECTION (sett_conn));
+	nm_assert (device == nm_settings_connection_default_wired_get_device (sett_conn));
+	nm_assert (sett_conn == g_object_get_qdata (G_OBJECT (device), _default_wired_connection_quark ()));
 
-	g_object_set_qdata (G_OBJECT (connection), _default_wired_device_quark (), NULL);
-	g_object_set_qdata (G_OBJECT (device), _default_wired_connection_quark (), NULL);
+	nm_settings_connection_default_wired_set_device (sett_conn, NULL);
 
-	g_signal_handlers_disconnect_by_func (connection, G_CALLBACK (default_wired_connection_updated_by_user_cb), self);
+	g_object_set_qdata (G_OBJECT (device), _default_wired_connection_quark (), NULL);
 
 	if (add_to_no_auto_default)
 		nm_config_set_no_auto_default_for_device (NM_SETTINGS_GET_PRIVATE (self)->config, device);
@@ -1585,7 +2880,7 @@ default_wired_clear_tag (NMSettings *self,
 static void
 device_realized (NMDevice *device, GParamSpec *pspec, NMSettings *self)
 {
-	NMConnection *connection;
+	gs_unref_object NMConnection *connection = NULL;
 	NMSettingsConnection *added;
 	GError *error = NULL;
 
@@ -1608,10 +2903,12 @@ device_realized (NMDevice *device, GParamSpec *pspec, NMSettings *self)
 	if (!connection)
 		return;
 
-	/* Add the connection */
-	added = nm_settings_add_connection (self, connection, FALSE, &error);
-	g_object_unref (connection);
-
+	nm_settings_add_connection (self,
+	                            connection,
+	                            NM_SETTINGS_CONNECTION_PERSIST_MODE_IN_MEMORY_ONLY,
+	                            NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED,
+	                            &added,
+	                            &error);
 	if (!added) {
 		if (!g_error_matches (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_UUID_EXISTS)) {
 			_LOGW ("(%s) couldn't create default wired connection: %s",
@@ -1622,11 +2919,9 @@ device_realized (NMDevice *device, GParamSpec *pspec, NMSettings *self)
 		return;
 	}
 
-	g_object_set_qdata (G_OBJECT (added), _default_wired_device_quark (), device);
-	g_object_set_qdata (G_OBJECT (device), _default_wired_connection_quark (), added);
+	nm_settings_connection_default_wired_set_device (added, device);
 
-	g_signal_connect (added, NM_SETTINGS_CONNECTION_UPDATED_INTERNAL,
-	                  G_CALLBACK (default_wired_connection_updated_by_user_cb), self);
+	g_object_set_qdata (G_OBJECT (device), _default_wired_connection_quark (), added);
 
 	_LOGI ("(%s): created default wired connection '%s'",
 	       nm_device_get_iface (device),
@@ -1639,6 +2934,8 @@ nm_settings_device_added (NMSettings *self, NMDevice *device)
 	if (nm_device_is_real (device))
 		device_realized (device, NULL, self);
 	else {
+		/* FIXME(shutdown): we need to disconnect this signal handler during
+		 *   shutdown. */
 		g_signal_connect_after (device, "notify::" NM_DEVICE_REAL,
 		                        G_CALLBACK (device_realized),
 		                        self);
@@ -1662,7 +2959,43 @@ nm_settings_device_removed (NMSettings *self, NMDevice *device, gboolean quittin
 		 * remains up and can be assumed if NM starts again.
 		 */
 		if (quitting == FALSE)
-			nm_settings_connection_delete (connection, NULL);
+			nm_settings_connection_delete (connection, TRUE);
+	}
+}
+
+/*****************************************************************************/
+
+static void
+session_monitor_changed_cb (NMSessionMonitor *session_monitor,
+                            NMSettings *self)
+{
+	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
+	NMSettingsConnection *const*list;
+	guint i, len;
+	guint generation;
+
+again:
+	list = nm_settings_get_connections (self, &len);
+	generation = priv->connections_generation;
+	for (i = 0; i < len; i++) {
+		gboolean is_visible;
+
+		is_visible = nm_settings_connection_check_visibility (list[i],
+		                                                      session_monitor);
+		nm_settings_connection_set_flags (list[i],
+		                                  NM_SETTINGS_CONNECTION_INT_FLAGS_VISIBLE,
+		                                  is_visible);
+		if (generation != priv->connections_generation) {
+			/* the cached list was invalidated. Start again.
+			 *
+			 * Note that nm_settings_connection_recheck_visibility() will do nothing
+			 * if the visibility didn't change (including emitting no signals,
+			 * and not invalidating the list).
+			 *
+			 * Hence, for this to be an endless loop, the settings would have
+			 * to constantly change the visibility flag and also invalidate the list. */
+			goto again;
+		}
 	}
 }
 
@@ -1795,9 +3128,14 @@ nm_settings_start (NMSettings *self, GError **error)
 {
 	NMSettingsPrivate *priv;
 	gs_strfreev char **plugins = NULL;
+	GSList *iter;
 
 	priv = NM_SETTINGS_GET_PRIVATE (self);
 
+	nm_assert (!priv->started);
+
+	priv->hostname_manager = g_object_ref (nm_hostname_manager_get ());
+
 	priv->kf_db_timestamps = nm_key_file_db_new (NMSTATEDIR "/timestamps",
 	                                             "timestamps",
 	                                             _kf_db_log_fcn,
@@ -1817,11 +3155,20 @@ nm_settings_start (NMSettings *self, GError **error)
 	if (!load_plugins (self, (const char *const*) plugins, error))
 		return FALSE;
 
-	load_connections (self);
+	for (iter = priv->plugins; iter; iter = iter->next) {
+		NMSettingsPlugin *plugin = NM_SETTINGS_PLUGIN (iter->data);
 
-	check_startup_complete (self);
+		g_signal_connect (plugin, NM_SETTINGS_PLUGIN_UNMANAGED_SPECS_CHANGED,
+		                  G_CALLBACK (_plugin_unmanaged_specs_changed), self);
+		g_signal_connect (plugin, NM_SETTINGS_PLUGIN_UNRECOGNIZED_SPECS_CHANGED,
+		                  G_CALLBACK (_plugin_unrecognized_specs_changed), self);
+	}
+
+	_plugin_unmanaged_specs_changed (NULL, self);
+	_plugin_unrecognized_specs_changed (NULL, self);
+
+	_plugin_connections_reload (self);
 
-	priv->hostname_manager = g_object_ref (nm_hostname_manager_get ());
 	g_signal_connect (priv->hostname_manager,
 	                  "notify::"NM_HOSTNAME_MANAGER_HOSTNAME,
 	                  G_CALLBACK (_hostname_changed_cb),
@@ -1829,6 +3176,14 @@ nm_settings_start (NMSettings *self, GError **error)
 	if (nm_hostname_manager_get_hostname (priv->hostname_manager))
 		_notify (self, PROP_HOSTNAME);
 
+	priv->started = TRUE;
+	_startup_complete_check (self, 0);
+
+	/* FIXME(shutdown): we also need a nm_settings_stop() during shutdown.
+	 *
+	 * In particular, we need to remove all in-memory keyfiles from /run that are nm-generated.
+	 * alternatively, the nm-generated flag must also be persisted and loaded to /run. */
+
 	return TRUE;
 }
 
@@ -1858,14 +3213,11 @@ get_property (GObject *object, guint prop_id,
 		g_value_set_boolean (value, TRUE);
 		break;
 	case PROP_CONNECTIONS:
-		if (priv->connections_loaded) {
-			strv = nm_dbus_utils_get_paths_for_clist (&priv->connections_lst_head,
-			                                          priv->connections_len,
-			                                          G_STRUCT_OFFSET (NMSettingsConnection, _connections_lst),
-			                                          TRUE);
-			g_value_take_boxed (value, nm_utils_strv_make_deep_copied (strv));
-		} else
-			g_value_set_boxed (value, NULL);
+		strv = nm_dbus_utils_get_paths_for_clist (&priv->connections_lst_head,
+		                                          priv->connections_len,
+		                                          G_STRUCT_OFFSET (NMSettingsConnection, _connections_lst),
+		                                          TRUE);
+		g_value_take_boxed (value, nm_utils_strv_make_deep_copied (strv));
 		break;
 	case PROP_STARTUP_COMPLETE:
 		g_value_set_boolean (value, !nm_settings_get_startup_complete_blocked_reason (self));
@@ -1886,8 +3238,21 @@ nm_settings_init (NMSettings *self)
 	c_list_init (&priv->auth_lst_head);
 	c_list_init (&priv->connections_lst_head);
 
-	priv->agent_mgr = g_object_ref (nm_agent_manager_get ());
+	c_list_init (&priv->sce_dirty_lst_head);
+	priv->sce_idx = g_hash_table_new_full (nm_pstr_hash, nm_pstr_equal,
+	                                       NULL, (GDestroyNotify) _sett_conn_entry_free);
+
 	priv->config = g_object_ref (nm_config_get ());
+
+	priv->agent_mgr = g_object_ref (nm_agent_manager_get ());
+
+	priv->platform = g_object_ref (NM_PLATFORM_GET);
+
+	priv->session_monitor = g_object_ref (nm_session_monitor_get ());
+	g_signal_connect (priv->session_monitor,
+	                  NM_SESSION_MONITOR_CHANGED,
+	                  G_CALLBACK (session_monitor_changed_cb),
+	                  self);
 }
 
 NMSettings *
@@ -1903,6 +3268,12 @@ dispose (GObject *object)
 	NMSettingsPrivate *priv = NM_SETTINGS_GET_PRIVATE (self);
 	CList *iter;
 
+	nm_assert (c_list_is_empty (&priv->sce_dirty_lst_head));
+	nm_assert (g_hash_table_size (priv->sce_idx) == 0);
+
+	nm_clear_g_source (&priv->startup_complete_timeout_id);
+	nm_clear_g_signal_handler (priv->platform, &priv->startup_complete_platform_change_id);
+	nm_clear_pointer (&priv->startup_complete_idx, g_hash_table_destroy);
 	g_clear_object (&priv->startup_complete_blocked_by);
 
 	while ((iter = c_list_first (&priv->auth_lst_head)))
@@ -1915,6 +3286,13 @@ dispose (GObject *object)
 		g_clear_object (&priv->hostname_manager);
 	}
 
+	if (priv->session_monitor) {
+		g_signal_handlers_disconnect_by_func (priv->session_monitor,
+		                                      G_CALLBACK (session_monitor_changed_cb),
+		                                      self);
+		g_clear_object (&priv->session_monitor);
+	}
+
 	G_OBJECT_CLASS (nm_settings_parent_class)->dispose (object);
 }
 
@@ -1929,6 +3307,11 @@ finalize (GObject *object)
 
 	nm_assert (c_list_is_empty (&priv->connections_lst_head));
 
+	nm_assert (c_list_is_empty (&priv->sce_dirty_lst_head));
+	nm_assert (g_hash_table_size (priv->sce_idx) == 0);
+
+	nm_clear_pointer (&priv->sce_idx, g_hash_table_destroy);
+
 	g_slist_free_full (priv->unmanaged_specs, g_free);
 	g_slist_free_full (priv->unrecognized_specs, g_free);
 
@@ -1943,8 +3326,6 @@ finalize (GObject *object)
 
 	g_clear_object (&priv->agent_mgr);
 
-	g_clear_object (&priv->config);
-
 	nm_clear_g_source (&priv->kf_db_flush_idle_id_timestamps);
 	nm_clear_g_source (&priv->kf_db_flush_idle_id_seen_bssids);
 	nm_key_file_db_to_file (priv->kf_db_timestamps, FALSE);
@@ -1953,6 +3334,10 @@ finalize (GObject *object)
 	nm_key_file_db_destroy (priv->kf_db_seen_bssids);
 
 	G_OBJECT_CLASS (nm_settings_parent_class)->finalize (object);
+
+	g_clear_object (&priv->config);
+
+	g_clear_object (&priv->platform);
 }
 
 static const GDBusSignalInfo signal_info_new_connection = NM_DEFINE_GDBUS_SIGNAL_INFO_INIT (
@@ -2123,7 +3508,7 @@ nm_settings_class_init (NMSettingsClass *class)
 	                  G_SIGNAL_RUN_FIRST,
 	                  0, NULL, NULL,
 	                  NULL,
-	                  G_TYPE_NONE, 2, NM_TYPE_SETTINGS_CONNECTION, G_TYPE_BOOLEAN);
+	                  G_TYPE_NONE, 2, NM_TYPE_SETTINGS_CONNECTION, G_TYPE_UINT);
 
 	signals[CONNECTION_REMOVED] =
 	    g_signal_new (NM_SETTINGS_SIGNAL_CONNECTION_REMOVED,
diff --git a/src/settings/nm-settings.h b/src/settings/nm-settings.h
index 02d65377e0..bcb30dff2a 100644
--- a/src/settings/nm-settings.h
+++ b/src/settings/nm-settings.h
@@ -27,6 +27,8 @@
 
 #include "nm-connection.h"
 
+#include "nm-settings-connection.h"
+
 #define NM_TYPE_SETTINGS            (nm_settings_get_type ())
 #define NM_SETTINGS(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NM_TYPE_SETTINGS, NMSettings))
 #define NM_SETTINGS_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass),  NM_TYPE_SETTINGS, NMSettingsClass))
@@ -67,6 +69,7 @@ NMSettings *nm_settings_get (void);
 #define NM_SETTINGS_GET (nm_settings_get ())
 
 NMSettings *nm_settings_new (void);
+
 gboolean nm_settings_start (NMSettings *self, GError **error);
 
 typedef void (*NMSettingsAddCallback) (NMSettings *settings,
@@ -78,7 +81,8 @@ typedef void (*NMSettingsAddCallback) (NMSettings *settings,
 
 void nm_settings_add_connection_dbus (NMSettings *self,
                                       NMConnection *connection,
-                                      gboolean save_to_disk,
+                                      NMSettingsConnectionPersistMode persist_mode,
+                                      NMSettingsConnectionIntFlags sett_flags,
                                       NMAuthSubject *subject,
                                       GDBusMethodInvocation *context,
                                       NMSettingsAddCallback callback,
@@ -93,16 +97,36 @@ NMSettingsConnection **nm_settings_get_connections_clone (NMSettings *self,
                                                           GCompareDataFunc sort_compare_func,
                                                           gpointer sort_data);
 
-NMSettingsConnection *nm_settings_add_connection (NMSettings *settings,
-                                                  NMConnection *connection,
-                                                  gboolean save_to_disk,
-                                                  GError **error);
+gboolean nm_settings_add_connection (NMSettings *settings,
+                                     NMConnection *connection,
+                                     NMSettingsConnectionPersistMode persist_mode,
+                                     NMSettingsConnectionIntFlags sett_flags,
+                                     NMSettingsConnection **out_sett_conn,
+                                     GError **error);
+
+gboolean nm_settings_update_connection (NMSettings *self,
+                                        NMSettingsConnection *sett_conn,
+                                        NMConnection *new_connection,
+                                        NMSettingsConnectionPersistMode persist_mode,
+                                        NMSettingsConnectionIntFlags sett_flags,
+                                        NMSettingsConnectionIntFlags sett_mask,
+                                        NMSettingsConnectionUpdateReason update_reason,
+                                        const char *log_context_name,
+                                        GError **error);
+
+void nm_settings_delete_connection (NMSettings *self,
+                                    NMSettingsConnection *sett_conn,
+                                    gboolean allow_add_to_no_auto_default);
+
 NMSettingsConnection *nm_settings_get_connection_by_path (NMSettings *settings,
                                                           const char *path);
 
 NMSettingsConnection *nm_settings_get_connection_by_uuid (NMSettings *settings,
                                                           const char *uuid);
 
+const char *nm_settings_get_dbus_path_for_uuid (NMSettings *self,
+                                                const char *uuid);
+
 gboolean nm_settings_has_connection (NMSettings *self, NMSettingsConnection *connection);
 
 const GSList *nm_settings_get_unmanaged_specs (NMSettings *self);
diff --git a/src/settings/plugins/ifcfg-rh/meson.build b/src/settings/plugins/ifcfg-rh/meson.build
index a238db940e..58acdcfcb1 100644
--- a/src/settings/plugins/ifcfg-rh/meson.build
+++ b/src/settings/plugins/ifcfg-rh/meson.build
@@ -35,7 +35,7 @@ libnms_ifcfg_rh_core = static_library(
   dependencies: deps,
 )
 
-sources = [dbus_sources] + core_sources + files('nms-ifcfg-rh-connection.c', 'nms-ifcfg-rh-plugin.c')
+sources = [dbus_sources] + core_sources + files('nms-ifcfg-rh-storage.c', 'nms-ifcfg-rh-plugin.c')
 
 libnm_settings_plugin_ifcfg_rh = shared_module(
   'nm-settings-plugin-ifcfg-rh',
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.c b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.c
deleted file mode 100644
index 5caa861de9..0000000000
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.c
+++ /dev/null
@@ -1,400 +0,0 @@
-/* NetworkManager system settings service
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Copyright (C) 2008 - 2011 Red Hat, Inc.
- */
-
-#include "nm-default.h"
-
-#include "nms-ifcfg-rh-connection.h"
-
-#include <sys/inotify.h>
-#include <glib/gstdio.h>
-
-#include "nm-dbus-interface.h"
-#include "nm-setting-connection.h"
-#include "nm-setting-wired.h"
-#include "nm-setting-wireless.h"
-#include "nm-setting-gsm.h"
-#include "nm-setting-cdma.h"
-#include "nm-setting-pppoe.h"
-#include "nm-setting-wireless-security.h"
-#include "nm-setting-8021x.h"
-#include "platform/nm-platform.h"
-#include "nm-config.h"
-
-#include "nms-ifcfg-rh-common.h"
-#include "nms-ifcfg-rh-reader.h"
-#include "nms-ifcfg-rh-writer.h"
-#include "nms-ifcfg-rh-utils.h"
-
-/*****************************************************************************/
-
-NM_GOBJECT_PROPERTIES_DEFINE_BASE (
-	PROP_UNMANAGED_SPEC,
-	PROP_UNRECOGNIZED_SPEC,
-);
-
-typedef struct {
-	char *unmanaged_spec;
-	char *unrecognized_spec;
-
-	gulong devtimeout_link_changed_handler;
-	guint devtimeout_timeout_id;
-} NMIfcfgConnectionPrivate;
-
-struct _NMIfcfgConnection {
-	NMSettingsConnection parent;
-	NMIfcfgConnectionPrivate _priv;
-};
-
-struct _NMIfcfgConnectionClass {
-	NMSettingsConnectionClass parent;
-};
-
-G_DEFINE_TYPE (NMIfcfgConnection, nm_ifcfg_connection, NM_TYPE_SETTINGS_CONNECTION)
-
-#define NM_IFCFG_CONNECTION_GET_PRIVATE(self) _NM_GET_PRIVATE (self, NMIfcfgConnection, NM_IS_IFCFG_CONNECTION)
-
-/*****************************************************************************/
-
-static gboolean
-devtimeout_ready (gpointer user_data)
-{
-	NMIfcfgConnection *self = user_data;
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE (self);
-
-	priv->devtimeout_timeout_id = 0;
-	nm_settings_connection_set_ready (NM_SETTINGS_CONNECTION (self), TRUE);
-	return FALSE;
-}
-
-static void
-link_changed (NMPlatform *platform, int obj_type_i, int ifindex, const NMPlatformLink *link,
-              int change_type_i,
-              NMConnection *self)
-{
-	const NMPlatformSignalChangeType change_type = change_type_i;
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE ((NMIfcfgConnection *) self);
-	const char *ifname;
-
-	ifname = nm_connection_get_interface_name (self);
-	if (g_strcmp0 (link->name, ifname) != 0)
-		return;
-
-	if (change_type == NM_PLATFORM_SIGNAL_REMOVED)
-		return;
-
-	nm_log_info (LOGD_SETTINGS, "Device %s appeared; connection '%s' now ready",
-	             ifname, nm_connection_get_id (self));
-
-	g_signal_handler_disconnect (platform, priv->devtimeout_link_changed_handler);
-	priv->devtimeout_link_changed_handler = 0;
-	g_source_remove (priv->devtimeout_timeout_id);
-
-	/* Don't declare the connection ready right away, since NMManager may not have
-	 * started processing the device yet.
-	 */
-	priv->devtimeout_timeout_id = g_idle_add (devtimeout_ready, self);
-}
-
-static gboolean
-devtimeout_expired (gpointer user_data)
-{
-	NMIfcfgConnection *self = user_data;
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE (self);
-
-	nm_log_info (LOGD_SETTINGS, "Device for connection '%s' did not appear before timeout",
-	             nm_settings_connection_get_id (NM_SETTINGS_CONNECTION (self)));
-
-	g_signal_handler_disconnect (NM_PLATFORM_GET, priv->devtimeout_link_changed_handler);
-	priv->devtimeout_link_changed_handler = 0;
-	priv->devtimeout_timeout_id = 0;
-
-	nm_settings_connection_set_ready (NM_SETTINGS_CONNECTION (self), TRUE);
-	return FALSE;
-}
-
-static void
-nm_ifcfg_connection_check_devtimeout (NMIfcfgConnection *self)
-{
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE (self);
-	NMSettingConnection *s_con;
-	const char *ifname;
-	const char *filename;
-	guint devtimeout;
-	const NMPlatformLink *pllink;
-
-	s_con = nm_connection_get_setting_connection (nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (self)));
-
-	if (!nm_setting_connection_get_autoconnect (s_con))
-		return;
-	ifname = nm_setting_connection_get_interface_name (s_con);
-	if (!ifname)
-		return;
-	filename = nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (self));
-	if (!filename)
-		return;
-
-	pllink = nm_platform_link_get_by_ifname (NM_PLATFORM_GET, ifname);
-	if (pllink && pllink->initialized)
-		return;
-
-	devtimeout = devtimeout_from_file (filename);
-	if (!devtimeout)
-		return;
-
-	/* ONBOOT=yes, DEVICE and DEVTIMEOUT are set, but device is not present */
-	nm_settings_connection_set_ready (NM_SETTINGS_CONNECTION (self), FALSE);
-
-	nm_log_info (LOGD_SETTINGS, "Waiting %u seconds for %s to appear for connection '%s'",
-	             devtimeout, ifname, nm_settings_connection_get_id (NM_SETTINGS_CONNECTION (self)));
-
-	priv->devtimeout_link_changed_handler =
-	    g_signal_connect (NM_PLATFORM_GET, NM_PLATFORM_SIGNAL_LINK_CHANGED,
-	                      G_CALLBACK (link_changed), self);
-	priv->devtimeout_timeout_id = g_timeout_add_seconds (devtimeout, devtimeout_expired, self);
-}
-
-const char *
-nm_ifcfg_connection_get_unmanaged_spec (NMIfcfgConnection *self)
-{
-	g_return_val_if_fail (NM_IS_IFCFG_CONNECTION (self), NULL);
-
-	return NM_IFCFG_CONNECTION_GET_PRIVATE (self)->unmanaged_spec;
-}
-
-const char *
-nm_ifcfg_connection_get_unrecognized_spec (NMIfcfgConnection *self)
-{
-	g_return_val_if_fail (NM_IS_IFCFG_CONNECTION (self), NULL);
-
-	return NM_IFCFG_CONNECTION_GET_PRIVATE (self)->unrecognized_spec;
-}
-
-static gboolean
-commit_changes (NMSettingsConnection *connection,
-                NMConnection *new_connection,
-                NMSettingsConnectionCommitReason commit_reason,
-                NMConnection **out_reread_connection,
-                char **out_logmsg_change,
-                GError **error)
-{
-	const char *filename;
-	gs_unref_object NMConnection *reread = NULL;
-	gboolean reread_same = TRUE;
-	const char *operation_message;
-	gs_free char *ifcfg_path = NULL;
-
-	nm_assert (out_reread_connection && !*out_reread_connection);
-	nm_assert (!out_logmsg_change || !*out_logmsg_change);
-
-	filename = nm_settings_connection_get_filename (connection);
-	if (!nms_ifcfg_rh_writer_write_connection (new_connection,
-	                                           IFCFG_DIR,
-	                                           filename,
-	                                           NULL,
-	                                           NULL,
-	                                           &ifcfg_path,
-	                                           &reread,
-	                                           &reread_same,
-	                                           error))
-		return FALSE;
-
-	nm_assert ((!filename && ifcfg_path) || (filename && !ifcfg_path));
-	if (ifcfg_path) {
-		nm_settings_connection_set_filename (connection, ifcfg_path);
-		operation_message = "persist";
-	} else
-		operation_message = "update";
-
-	if (reread && !reread_same)
-		*out_reread_connection = g_steal_pointer (&reread);
-
-	NM_SET_OUT (out_logmsg_change,
-	            g_strdup_printf ("ifcfg-rh: %s %s",
-	                             operation_message, filename));
-	return TRUE;
-}
-
-static gboolean
-delete (NMSettingsConnection *connection,
-        GError **error)
-{
-	const char *filename;
-
-	filename = nm_settings_connection_get_filename (connection);
-	if (filename) {
-		gs_free char *keyfile = utils_get_keys_path (filename);
-		gs_free char *routefile = utils_get_route_path (filename);
-		gs_free char *route6file = utils_get_route6_path (filename);
-
-		g_unlink (filename);
-		if (keyfile)
-			g_unlink (keyfile);
-		if (routefile)
-			g_unlink (routefile);
-		if (route6file)
-			g_unlink (route6file);
-	}
-
-	return TRUE;
-}
-
-/*****************************************************************************/
-
-static void
-get_property (GObject *object, guint prop_id,
-              GValue *value, GParamSpec *pspec)
-{
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE ((NMIfcfgConnection *) object);
-
-	switch (prop_id) {
-	case PROP_UNMANAGED_SPEC:
-		g_value_set_string (value, priv->unmanaged_spec);
-		break;
-	case PROP_UNRECOGNIZED_SPEC:
-		g_value_set_string (value, priv->unrecognized_spec);
-		break;
-	default:
-		G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
-		break;
-	}
-}
-
-static void
-set_property (GObject *object, guint prop_id,
-              const GValue *value, GParamSpec *pspec)
-{
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE ((NMIfcfgConnection *) object);
-
-	switch (prop_id) {
-	case PROP_UNMANAGED_SPEC:
-		priv->unmanaged_spec = g_value_dup_string (value);
-		break;
-	case PROP_UNRECOGNIZED_SPEC:
-		priv->unrecognized_spec = g_value_dup_string (value);
-		break;
-	default:
-		G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
-		break;
-	}
-}
-
-/*****************************************************************************/
-
-static void
-nm_ifcfg_connection_init (NMIfcfgConnection *self)
-{
-}
-
-NMIfcfgConnection *
-nm_ifcfg_connection_new (NMConnection *source,
-                         const char *full_path,
-                         GError **error,
-                         gboolean *out_ignore_error)
-{
-	GObject *object;
-	NMConnection *tmp;
-	char *unhandled_spec = NULL;
-	const char *unmanaged_spec = NULL, *unrecognized_spec = NULL;
-
-	g_assert (source || full_path);
-
-	if (out_ignore_error)
-		*out_ignore_error = FALSE;
-
-	/* If we're given a connection already, prefer that instead of re-reading */
-	if (source)
-		tmp = g_object_ref (source);
-	else {
-		tmp = connection_from_file (full_path,
-		                            &unhandled_spec,
-		                            error,
-		                            out_ignore_error);
-		if (!tmp)
-			return NULL;
-	}
-
-	if (unhandled_spec && g_str_has_prefix (unhandled_spec, "unmanaged:"))
-		unmanaged_spec = unhandled_spec + strlen ("unmanaged:");
-	else if (unhandled_spec && g_str_has_prefix (unhandled_spec, "unrecognized:"))
-		unrecognized_spec = unhandled_spec + strlen ("unrecognized:");
-
-	object = (GObject *) g_object_new (NM_TYPE_IFCFG_CONNECTION,
-	                                   NM_SETTINGS_CONNECTION_FILENAME, full_path,
-	                                   NM_IFCFG_CONNECTION_UNMANAGED_SPEC, unmanaged_spec,
-	                                   NM_IFCFG_CONNECTION_UNRECOGNIZED_SPEC, unrecognized_spec,
-	                                   NULL);
-	/* Update our settings with what was read from the file */
-	if (nm_settings_connection_update (NM_SETTINGS_CONNECTION (object),
-	                                   tmp,
-	                                   full_path
-	                                     ? NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP_SAVED
-	                                     : NM_SETTINGS_CONNECTION_PERSIST_MODE_UNSAVED,
-	                                   NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-	                                   NULL,
-	                                   error))
-		nm_ifcfg_connection_check_devtimeout (NM_IFCFG_CONNECTION (object));
-	else
-		g_clear_object (&object);
-
-	g_object_unref (tmp);
-	g_free (unhandled_spec);
-	return (NMIfcfgConnection *) object;
-}
-
-static void
-dispose (GObject *object)
-{
-	NMIfcfgConnectionPrivate *priv = NM_IFCFG_CONNECTION_GET_PRIVATE ((NMIfcfgConnection *) object);
-
-	nm_clear_g_signal_handler (NM_PLATFORM_GET, &priv->devtimeout_link_changed_handler);
-	nm_clear_g_source (&priv->devtimeout_timeout_id);
-
-	g_clear_pointer (&priv->unmanaged_spec, g_free);
-	g_clear_pointer (&priv->unrecognized_spec, g_free);
-
-	G_OBJECT_CLASS (nm_ifcfg_connection_parent_class)->dispose (object);
-}
-
-static void
-nm_ifcfg_connection_class_init (NMIfcfgConnectionClass *ifcfg_connection_class)
-{
-	GObjectClass *object_class = G_OBJECT_CLASS (ifcfg_connection_class);
-	NMSettingsConnectionClass *settings_class = NM_SETTINGS_CONNECTION_CLASS (ifcfg_connection_class);
-
-	object_class->set_property = set_property;
-	object_class->get_property = get_property;
-	object_class->dispose      = dispose;
-
-	settings_class->delete = delete;
-	settings_class->commit_changes = commit_changes;
-
-	obj_properties[PROP_UNMANAGED_SPEC] =
-	     g_param_spec_string (NM_IFCFG_CONNECTION_UNMANAGED_SPEC, "", "",
-	                          NULL,
-	                          G_PARAM_READWRITE |
-	                          G_PARAM_STATIC_STRINGS);
-
-	obj_properties[PROP_UNRECOGNIZED_SPEC] =
-	     g_param_spec_string (NM_IFCFG_CONNECTION_UNRECOGNIZED_SPEC, "", "",
-	                          NULL,
-	                          G_PARAM_READWRITE |
-	                          G_PARAM_STATIC_STRINGS);
-
-	g_object_class_install_properties (object_class, _PROPERTY_ENUMS_LAST, obj_properties);
-}
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.h b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.h
deleted file mode 100644
index 8f5e77ef33..0000000000
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-connection.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/* NetworkManager system settings service
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Copyright (C) 2008 - 2011 Red Hat, Inc.
- */
-
-#ifndef __NETWORKMANAGER_IFCFG_CONNECTION_H__
-#define __NETWORKMANAGER_IFCFG_CONNECTION_H__
-
-#include "nm-dbus-interface.h"
-#include "settings/nm-settings-connection.h"
-
-#define NM_TYPE_IFCFG_CONNECTION            (nm_ifcfg_connection_get_type ())
-#define NM_IFCFG_CONNECTION(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NM_TYPE_IFCFG_CONNECTION, NMIfcfgConnection))
-#define NM_IFCFG_CONNECTION_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NM_TYPE_IFCFG_CONNECTION, NMIfcfgConnectionClass))
-#define NM_IS_IFCFG_CONNECTION(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NM_TYPE_IFCFG_CONNECTION))
-#define NM_IS_IFCFG_CONNECTION_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NM_TYPE_IFCFG_CONNECTION))
-#define NM_IFCFG_CONNECTION_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NM_TYPE_IFCFG_CONNECTION, NMIfcfgConnectionClass))
-
-#define NM_IFCFG_CONNECTION_UNMANAGED_SPEC    "unmanaged-spec"
-#define NM_IFCFG_CONNECTION_UNRECOGNIZED_SPEC "unrecognized-spec"
-
-typedef struct _NMIfcfgConnection NMIfcfgConnection;
-typedef struct _NMIfcfgConnectionClass NMIfcfgConnectionClass;
-
-GType nm_ifcfg_connection_get_type (void);
-
-NMIfcfgConnection *nm_ifcfg_connection_new (NMConnection *source,
-                                            const char *full_path,
-                                            GError **error,
-                                            gboolean *out_ignore_error);
-
-const char *nm_ifcfg_connection_get_unmanaged_spec (NMIfcfgConnection *self);
-const char *nm_ifcfg_connection_get_unrecognized_spec (NMIfcfgConnection *self);
-
-gboolean nm_ifcfg_connection_update (NMIfcfgConnection *self,
-                                     GHashTable *new_settings,
-                                     GError **error);
-
-#endif /* __NETWORKMANAGER_IFCFG_CONNECTION_H__ */
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.c b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.c
index 6375b6cc14..cc4fe4ceca 100644
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.c
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.c
@@ -24,23 +24,26 @@
 
 #include "nms-ifcfg-rh-plugin.h"
 
-#include <unistd.h>
 #include <sys/types.h>
 #include <sys/stat.h>
-#include <gmodule.h>
+#include <unistd.h>
 
+#include "nm-std-aux/c-list-util.h"
+#include "nm-glib-aux/nm-c-list.h"
+#include "nm-glib-aux/nm-io-utils.h"
 #include "nm-std-aux/nm-dbus-compat.h"
-#include "nm-setting-connection.h"
-#include "settings/nm-settings-plugin.h"
+#include "nm-utils.h"
+#include "nm-core-internal.h"
 #include "nm-config.h"
+#include "settings/nm-settings-plugin.h"
+#include "settings/nm-settings-utils.h"
 #include "NetworkManagerUtils.h"
 
-#include "nms-ifcfg-rh-connection.h"
+#include "nms-ifcfg-rh-storage.h"
 #include "nms-ifcfg-rh-common.h"
+#include "nms-ifcfg-rh-utils.h"
 #include "nms-ifcfg-rh-reader.h"
 #include "nms-ifcfg-rh-writer.h"
-#include "nms-ifcfg-rh-utils.h"
-#include "shvar.h"
 
 #define IFCFGRH1_BUS_NAME                               "com.redhat.ifcfgrh1"
 #define IFCFGRH1_OBJECT_PATH                            "/com/redhat/ifcfgrh1"
@@ -59,23 +62,25 @@ typedef struct {
 		guint regist_id;
 	} dbus;
 
-	GHashTable *connections;  /* uuid::connection */
+	NMSettUtilStorages storages;
 
-	bool initialized:1;
-} SettingsPluginIfcfgPrivate;
+	GHashTable *unmanaged_specs;
+	GHashTable *unrecognized_specs;
 
-struct _SettingsPluginIfcfg {
+} NMSIfcfgRHPluginPrivate;
+
+struct _NMSIfcfgRHPlugin {
 	NMSettingsPlugin parent;
-	SettingsPluginIfcfgPrivate _priv;
+	NMSIfcfgRHPluginPrivate _priv;
 };
 
-struct _SettingsPluginIfcfgClass {
+struct _NMSIfcfgRHPluginClass {
 	NMSettingsPluginClass parent;
 };
 
-G_DEFINE_TYPE (SettingsPluginIfcfg, settings_plugin_ifcfg, NM_TYPE_SETTINGS_PLUGIN)
+G_DEFINE_TYPE (NMSIfcfgRHPlugin, nms_ifcfg_rh_plugin, NM_TYPE_SETTINGS_PLUGIN)
 
-#define SETTINGS_PLUGIN_IFCFG_GET_PRIVATE(self) _NM_GET_PRIVATE (self, SettingsPluginIfcfg, SETTINGS_IS_PLUGIN_IFCFG)
+#define NMS_IFCFG_RH_PLUGIN_GET_PRIVATE(self) _NM_GET_PRIVATE (self, NMSIfcfgRHPlugin, NMS_IS_IFCFG_RH_PLUGIN, NMSettingsPlugin)
 
 /*****************************************************************************/
 
@@ -90,531 +95,811 @@ G_DEFINE_TYPE (SettingsPluginIfcfg, settings_plugin_ifcfg, NM_TYPE_SETTINGS_PLUG
 
 /*****************************************************************************/
 
-static NMIfcfgConnection *update_connection (SettingsPluginIfcfg *plugin,
-                                             NMConnection *source,
-                                             const char *full_path,
-                                             NMIfcfgConnection *connection,
-                                             gboolean protect_existing_connection,
-                                             GHashTable *protected_connections,
-                                             GError **error);
+static void _unhandled_specs_reset (NMSIfcfgRHPlugin *self);
+
+static void _unhandled_specs_merge_storages (NMSIfcfgRHPlugin *self,
+                                             NMSettUtilStorages *storages);
 
 /*****************************************************************************/
 
 static void
-connection_removed_cb (NMSettingsConnection *obj, gpointer user_data)
+nm_assert_self (NMSIfcfgRHPlugin *self, gboolean unhandled_specs_consistent)
 {
-	g_hash_table_remove (SETTINGS_PLUGIN_IFCFG_GET_PRIVATE ((SettingsPluginIfcfg *) user_data)->connections,
-	                     nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (obj)));
+	nm_assert (NMS_IS_IFCFG_RH_PLUGIN (self));
+
+#if NM_MORE_ASSERTS > 5
+	{
+		NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+		NMSIfcfgRHStorage *storage;
+		gsize n_uuid;
+		gs_unref_hashtable GHashTable *h_unmanaged = NULL;
+		gs_unref_hashtable GHashTable *h_unrecognized = NULL;
+
+		nm_assert (g_hash_table_size (priv->storages.idx_by_filename) == c_list_length (&priv->storages._storage_lst_head));
+
+		h_unmanaged = g_hash_table_new (nm_str_hash, g_str_equal);
+		h_unrecognized = g_hash_table_new (nm_str_hash, g_str_equal);
+
+		n_uuid = 0;
+
+		c_list_for_each_entry (storage, &priv->storages._storage_lst_head, parent._storage_lst) {
+			const char *uuid;
+			const char *filename;
+
+			filename = nms_ifcfg_rh_storage_get_filename (storage);
+
+			nm_assert (filename && NM_STR_HAS_PREFIX (filename, IFCFG_DIR"/"));
+
+			uuid = nms_ifcfg_rh_storage_get_uuid_opt (storage);
+
+			nm_assert ((!!uuid) + (!!storage->unmanaged_spec) + (!!storage->unrecognized_spec) == 1);
+
+			nm_assert (storage == nm_sett_util_storages_lookup_by_filename (&priv->storages, filename));
+
+			if (uuid) {
+				NMSettUtilStorageByUuidHead *sbuh;
+				NMSettUtilStorageByUuidHead *sbuh2;
+
+				if (storage->connection)
+					nm_assert (nm_streq0 (nm_connection_get_uuid (storage->connection), uuid));
+
+				if (!g_hash_table_lookup_extended (priv->storages.idx_by_uuid, &uuid, (gpointer *) &sbuh, (gpointer *) &sbuh2))
+					nm_assert_not_reached ();
+
+				nm_assert (sbuh);
+				nm_assert (nm_streq (uuid, sbuh->uuid));
+				nm_assert (sbuh == sbuh2);
+				nm_assert (c_list_contains (&sbuh->_storage_by_uuid_lst_head, &storage->parent._storage_by_uuid_lst));
+
+				if (c_list_first (&sbuh->_storage_by_uuid_lst_head) == &storage->parent._storage_by_uuid_lst)
+					n_uuid++;
+			} else if (storage->unmanaged_spec) {
+				nm_assert (strlen (storage->unmanaged_spec) > 0);
+				g_hash_table_add (h_unmanaged, storage->unmanaged_spec);
+			} else if (storage->unrecognized_spec) {
+				nm_assert (strlen (storage->unrecognized_spec) > 0);
+				g_hash_table_add (h_unrecognized, storage->unrecognized_spec);
+			} else
+				nm_assert_not_reached ();
+
+			nm_assert (!storage->connection);
+		}
+
+		nm_assert (g_hash_table_size (priv->storages.idx_by_uuid) == n_uuid);
+
+		if (unhandled_specs_consistent) {
+			nm_assert (nm_utils_hashtable_same_keys (h_unmanaged, priv->unmanaged_specs));
+			nm_assert (nm_utils_hashtable_same_keys (h_unrecognized, priv->unrecognized_specs));
+		}
+	}
+#endif
 }
 
-static void
-remove_connection (SettingsPluginIfcfg *self, NMIfcfgConnection *connection)
+/*****************************************************************************/
+
+static NMSIfcfgRHStorage *
+_load_file (NMSIfcfgRHPlugin *self,
+            const char *filename,
+            GError **error)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
-	gboolean unmanaged, unrecognized;
+	gs_unref_object NMConnection *connection = NULL;
+	gs_free_error GError *load_error = NULL;
+	gs_free char *unhandled_spec = NULL;
+	gboolean load_error_ignore;
+	struct stat st;
 
-	g_return_if_fail (self != NULL);
-	g_return_if_fail (connection != NULL);
+	if (stat (filename, &st) != 0) {
+		int errsv = errno;
 
-	_LOGI ("remove "NM_IFCFG_CONNECTION_LOG_FMT, NM_IFCFG_CONNECTION_LOG_ARG (connection));
+		if (error) {
+			nm_utils_error_set_errno (error, errsv,
+			                          "failure to stat file \%s\": %s",
+			                          filename);
+		} else
+			_LOGT ("load[%s]: failure to stat file: %s", filename, nm_strerror_native (errsv));
+		return NULL;
+	}
 
-	unmanaged = !!nm_ifcfg_connection_get_unmanaged_spec (connection);
-	unrecognized = !!nm_ifcfg_connection_get_unrecognized_spec (connection);
+	connection = connection_from_file (filename,
+	                                   &unhandled_spec,
+	                                   &load_error,
+	                                   &load_error_ignore);
+	if (load_error) {
+		if (error) {
+			nm_utils_error_set (error, NM_UTILS_ERROR_UNKNOWN,
+			                    "failure to read file \"%s\": %s",
+			                    filename, load_error->message);
+		} else {
+			_NMLOG (load_error_ignore ? LOGL_TRACE : LOGL_WARN,
+			        "load[%s]: failure to read file: %s", filename, load_error->message);
+		}
+		return NULL;
+	}
 
-	g_object_ref (connection);
-	g_hash_table_remove (priv->connections, nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (connection)));
-	if (!unmanaged && !unrecognized)
-		nm_settings_connection_signal_remove (NM_SETTINGS_CONNECTION (connection));
-	g_object_unref (connection);
+	if (unhandled_spec) {
+		const char *unmanaged_spec;
+		const char *unrecognized_spec;
+
+		if (!nms_ifcfg_rh_util_parse_unhandled_spec (unhandled_spec,
+		                                             &unmanaged_spec,
+		                                             &unrecognized_spec)) {
+			nm_utils_error_set (error, NM_UTILS_ERROR_UNKNOWN,
+			                    "invalid unhandled spec \"%s\"",
+			                    unhandled_spec);
+			nm_assert_not_reached ();
+			return NULL;
+		}
+		return nms_ifcfg_rh_storage_new_unhandled (self,
+		                                           filename,
+		                                           unmanaged_spec,
+		                                           unrecognized_spec);
+	}
 
-	/* Emit changes _after_ removing the connection */
-	if (unmanaged)
-		_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
-	if (unrecognized)
-		_nm_settings_plugin_emit_signal_unrecognized_specs_changed (NM_SETTINGS_PLUGIN (self));
+	return nms_ifcfg_rh_storage_new_connection (self,
+	                                            filename,
+	                                            g_steal_pointer (&connection),
+	                                            &st.st_mtim);
 }
 
-static NMIfcfgConnection *
-find_by_path (SettingsPluginIfcfg *self, const char *path)
+static void
+_load_dir (NMSIfcfgRHPlugin *self,
+           NMSettUtilStorages *storages)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
-	GHashTableIter iter;
-	NMSettingsConnection *candidate = NULL;
+	gs_unref_hashtable GHashTable *dupl_filenames = NULL;
+	gs_free_error GError *local = NULL;
+	const char *f_filename;
+	GDir *dir;
+
+	dir = g_dir_open (IFCFG_DIR, 0, &local);
+	if (!dir) {
+		_LOGT ("Could not read directory '%s': %s", IFCFG_DIR, local->message);
+		return;
+	}
+
+	dupl_filenames = g_hash_table_new_full (nm_str_hash, g_str_equal, NULL, g_free);
+
+	while ((f_filename = g_dir_read_name (dir))) {
+		gs_free char *full_path = NULL;
+		NMSIfcfgRHStorage *storage;
+		char *full_filename;
+
+		full_path = g_build_filename (IFCFG_DIR, f_filename, NULL);
+		full_filename = utils_detect_ifcfg_path (full_path, TRUE);
+		if (!full_filename)
+			continue;
 
-	g_return_val_if_fail (path != NULL, NULL);
+		if (!g_hash_table_add (dupl_filenames, full_filename))
+			continue;
 
-	g_hash_table_iter_init (&iter, priv->connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer) &candidate)) {
-		if (g_strcmp0 (path, nm_settings_connection_get_filename (candidate)) == 0)
-			return NM_IFCFG_CONNECTION (candidate);
+		nm_assert (!nm_sett_util_storages_lookup_by_filename (storages, full_filename));
+
+		storage = _load_file (self,
+		                      full_filename,
+		                      NULL);
+		if (storage)
+			nm_sett_util_storages_add_take (storages, storage);
 	}
-	return NULL;
+	g_dir_close (dir);
 }
 
-static NMIfcfgConnection *
-update_connection (SettingsPluginIfcfg *self,
-                   NMConnection *source,
-                   const char *full_path,
-                   NMIfcfgConnection *connection,
-                   gboolean protect_existing_connection,
-                   GHashTable *protected_connections,
-                   GError **error)
+static void
+_storages_consolidate (NMSIfcfgRHPlugin *self,
+                       NMSettUtilStorages *storages_new,
+                       gboolean replace_all,
+                       GHashTable *storages_replaced,
+                       NMSettingsPluginConnectionLoadCallback callback,
+                       gpointer user_data)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
-	NMIfcfgConnection *connection_new;
-	NMIfcfgConnection *connection_by_uuid;
-	GError *local = NULL;
-	const char *new_unmanaged = NULL, *old_unmanaged = NULL;
-	const char *new_unrecognized = NULL, *old_unrecognized = NULL;
-	gboolean unmanaged_changed = FALSE, unrecognized_changed = FALSE;
-	const char *uuid;
-	gboolean ignore_error = FALSE;
-
-	g_return_val_if_fail (!source || NM_IS_CONNECTION (source), NULL);
-	g_return_val_if_fail (full_path || source, NULL);
-
-	if (full_path)
-		_LOGD ("loading from file \"%s\"...", full_path);
-
-	/* Create a NMIfcfgConnection instance, either by reading from @full_path or
-	 * based on @source. */
-	connection_new = nm_ifcfg_connection_new (source, full_path, &local, &ignore_error);
-	if (!connection_new) {
-		/* Unexpected failure. Probably the file is invalid? */
-		if (   connection
-		    && !protect_existing_connection
-		    && (!protected_connections || !g_hash_table_contains (protected_connections, connection)))
-			remove_connection (self, connection);
-		if (!source) {
-			_NMLOG (ignore_error ? LOGL_DEBUG : LOGL_WARN,
-			        "loading \"%s\" fails: %s", full_path, local ? local->message : "(unknown reason)");
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	CList lst_conn_info_deleted = C_LIST_INIT (lst_conn_info_deleted);
+	gs_unref_ptrarray GPtrArray *storages_modified = NULL;
+	CList storages_deleted;
+	NMSIfcfgRHStorage *storage_safe;
+	NMSIfcfgRHStorage *storage_new;
+	NMSIfcfgRHStorage *storage_old;
+	NMSIfcfgRHStorage *storage;
+	guint i;
+
+	/* when we reload all files, we must signal add/update/modify of profiles one-by-one.
+	 * NMSettings then goes ahead and emits further signals and a lot of things happen.
+	 *
+	 * So, first, emit an update of the unmanaged/unrecognized specs that contains *all*
+	 * the unmanaged/unrecognized devices from before and after. Since both unmanaged/unrecognized
+	 * specs have the meaning of "not doing something", it makes sense that we temporarily
+	 * disable that action for the sum of before and after. */
+	_unhandled_specs_merge_storages (self, storages_new);
+
+	storages_modified = g_ptr_array_new_with_free_func (g_object_unref);
+	c_list_init (&storages_deleted);
+
+	c_list_for_each_entry (storage_old, &priv->storages._storage_lst_head, parent._storage_lst)
+		storage_old->dirty = TRUE;
+
+	c_list_for_each_entry_safe (storage_new, storage_safe, &storages_new->_storage_lst_head, parent._storage_lst) {
+		storage_old = nm_sett_util_storages_lookup_by_filename (&priv->storages, nms_ifcfg_rh_storage_get_filename (storage_new));
+
+		nm_sett_util_storages_steal (storages_new, storage_new);
+
+		if (   !storage_old
+		    || !nms_ifcfg_rh_storage_equal_type (storage_new, storage_old)) {
+			if (storage_old) {
+				nm_sett_util_storages_steal (&priv->storages, storage_old);
+				if (nms_ifcfg_rh_storage_get_uuid_opt (storage_old))
+					c_list_link_tail (&storages_deleted, &storage_old->parent._storage_lst);
+				else
+					nms_ifcfg_rh_storage_destroy (storage_old);
+			}
+			storage_new->dirty = FALSE;
+			nm_sett_util_storages_add_take (&priv->storages, storage_new);
+			g_ptr_array_add (storages_modified, g_object_ref (storage_new));
+			continue;
+		}
+
+		storage_old->dirty = FALSE;
+		nms_ifcfg_rh_storage_copy_content (storage_old, storage_new);
+		nms_ifcfg_rh_storage_destroy (storage_new);
+		g_ptr_array_add (storages_modified, g_object_ref (storage_old));
+	}
+
+	c_list_for_each_entry_safe (storage_old, storage_safe, &priv->storages._storage_lst_head, parent._storage_lst) {
+		if (!storage_old->dirty)
+			continue;
+		if (   replace_all
+		    || (   storages_replaced
+		        && g_hash_table_contains (storages_replaced, storage_old))) {
+			nm_sett_util_storages_steal (&priv->storages, storage_old);
+			if (nms_ifcfg_rh_storage_get_uuid_opt (storage_old))
+				c_list_link_tail (&storages_deleted, &storage_old->parent._storage_lst);
+			else
+				nms_ifcfg_rh_storage_destroy (storage_old);
 		}
-		g_propagate_error (error, local);
-		return NULL;
 	}
 
-	uuid = nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (connection_new));
-	connection_by_uuid = g_hash_table_lookup (priv->connections, uuid);
+	/* raise events. */
 
-	if (   connection
-	    && connection != connection_by_uuid) {
+	for (i = 0; i < storages_modified->len; i++) {
+		storage = storages_modified->pdata[i];
+		storage->dirty = TRUE;
+	}
 
-		if (   (protect_existing_connection && connection_by_uuid != NULL)
-		    || (protected_connections && g_hash_table_contains (protected_connections, connection))) {
-			NMIfcfgConnection *conflicting = (protect_existing_connection && connection_by_uuid != NULL) ? connection_by_uuid : connection;
+	for (i = 0; i < storages_modified->len; i++) {
+		gs_unref_object NMConnection *connection = NULL;
+		storage = storages_modified->pdata[i];
 
-			if (source)
-				_LOGW ("cannot update protected connection "NM_IFCFG_CONNECTION_LOG_FMT" due to conflicting UUID %s", NM_IFCFG_CONNECTION_LOG_ARG (conflicting), uuid);
-			else
-				_LOGW ("cannot load %s due to conflicting UUID for "NM_IFCFG_CONNECTION_LOG_FMT, full_path, NM_IFCFG_CONNECTION_LOG_ARG (conflicting));
-			g_object_unref (connection_new);
-			g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-			                     "Cannot update protected connection due to conflicting UUID");
-			return NULL;
+		if (!storage->dirty) {
+			/* the entry is no longer dirty. In the meantime we already emited
+			 * another signal for it. */
+			continue;
+		}
+		storage->dirty = FALSE;
+		if (storage != nm_sett_util_storages_lookup_by_filename (&priv->storages, nms_ifcfg_rh_storage_get_filename (storage))) {
+			/* hm? The profile was deleted in the meantime? That is only possible
+			 * if the signal handler called again into the plugin. In any case, the event
+			 * was already emitted. Skip. */
+			continue;
 		}
 
-		/* The new connection has a different UUID then the original one that we
-		 * are about to update. Remove @connection. */
-		remove_connection (self, connection);
+		connection = nms_ifcfg_rh_storage_steal_connection (storage);
+		if (!connection) {
+			nm_assert (!nms_ifcfg_rh_storage_get_uuid_opt (storage));
+			continue;
+		}
+
+		nm_assert (NM_IS_CONNECTION (connection));
+		nm_assert (nms_ifcfg_rh_storage_get_uuid_opt (storage));
+		callback (NM_SETTINGS_PLUGIN (self),
+		          NM_SETTINGS_STORAGE (storage),
+		          connection,
+		          user_data);
 	}
 
-	/* Check if the found connection with the same UUID is not protected from updating. */
-	if (   connection_by_uuid
-	    && (   (!connection && protect_existing_connection)
-	        || (protected_connections && g_hash_table_contains (protected_connections, connection_by_uuid)))) {
-		if (source)
-			_LOGW ("cannot update connection due to conflicting UUID for "NM_IFCFG_CONNECTION_LOG_FMT, NM_IFCFG_CONNECTION_LOG_ARG (connection_by_uuid));
-		else
-			_LOGW ("cannot load %s due to conflicting UUID for "NM_IFCFG_CONNECTION_LOG_FMT, full_path, NM_IFCFG_CONNECTION_LOG_ARG (connection_by_uuid));
-		g_object_unref (connection_new);
-		g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-		                      "Skip updating protected connection during reload");
-		return NULL;
+	while ((storage = c_list_first_entry (&storages_deleted, NMSIfcfgRHStorage, parent._storage_lst))) {
+		c_list_unlink (&storage->parent._storage_lst);
+		callback (NM_SETTINGS_PLUGIN (self),
+		          NM_SETTINGS_STORAGE (storage),
+		          NULL,
+		          user_data);
+		nms_ifcfg_rh_storage_destroy (storage);
 	}
+}
 
-	/* Evaluate unmanaged/unrecognized flags. */
-	if (connection_by_uuid)
-		old_unmanaged = nm_ifcfg_connection_get_unmanaged_spec (connection_by_uuid);
-	new_unmanaged = nm_ifcfg_connection_get_unmanaged_spec (connection_new);
-	unmanaged_changed = g_strcmp0 (old_unmanaged, new_unmanaged);
-
-	if (connection_by_uuid)
-		old_unrecognized = nm_ifcfg_connection_get_unrecognized_spec (connection_by_uuid);
-	new_unrecognized = nm_ifcfg_connection_get_unrecognized_spec (connection_new);
-	unrecognized_changed = g_strcmp0 (old_unrecognized, new_unrecognized);
-
-	if (connection_by_uuid) {
-		const char *old_path;
-
-		old_path = nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection_by_uuid));
-
-		if (   !unmanaged_changed
-		    && !unrecognized_changed
-		    && nm_connection_compare (nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection_by_uuid)),
-		                              nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection_new)),
-		                              NM_SETTING_COMPARE_FLAG_IGNORE_AGENT_OWNED_SECRETS |
-		                              NM_SETTING_COMPARE_FLAG_IGNORE_NOT_SAVED_SECRETS)) {
-			if (   old_path
-			    && !nm_streq0 (old_path, full_path)) {
-				_LOGI ("rename \"%s\" to "NM_IFCFG_CONNECTION_LOG_FMT" without other changes",
-				       nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection_by_uuid)),
-				       NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-			}
-		} else {
+/*****************************************************************************/
+
+static void
+load_connections (NMSettingsPlugin *plugin,
+                  NMSettingsPluginConnectionLoadEntry *entries,
+                  gsize n_entries,
+                  NMSettingsPluginConnectionLoadCallback callback,
+                  gpointer user_data)
+{
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (plugin);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	nm_auto_clear_sett_util_storages NMSettUtilStorages storages_new = NM_SETT_UTIL_STORAGES_INIT (storages_new, nms_ifcfg_rh_storage_destroy);
+	gs_unref_hashtable GHashTable *dupl_filenames = NULL;
+	gs_unref_hashtable GHashTable *storages_replaced = NULL;
+	gs_unref_hashtable GHashTable *loaded_uuids = NULL;
+	const char *loaded_uuid;
+	GHashTableIter h_iter;
+	gsize i;
+
+	if (n_entries == 0)
+		return;
 
-			/*******************************************************
-			 * UPDATE
-			 *******************************************************/
+	dupl_filenames = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
 
-			if (source)
-				_LOGI ("update "NM_IFCFG_CONNECTION_LOG_FMT" from %s", NM_IFCFG_CONNECTION_LOG_ARG (connection_new), NM_IFCFG_CONNECTION_LOG_PATH (old_path));
-			else if (nm_streq0 (old_path, nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection_new))))
-				_LOGI ("update "NM_IFCFG_CONNECTION_LOG_FMT, NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-			else if (old_path)
-				_LOGI ("rename \"%s\" to "NM_IFCFG_CONNECTION_LOG_FMT, old_path, NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-			else
-				_LOGI ("update and persist "NM_IFCFG_CONNECTION_LOG_FMT, NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-
-			g_object_set (connection_by_uuid,
-			              NM_IFCFG_CONNECTION_UNMANAGED_SPEC, new_unmanaged,
-			              NM_IFCFG_CONNECTION_UNRECOGNIZED_SPEC, new_unrecognized,
-			              NULL);
-
-			if (!nm_settings_connection_update (NM_SETTINGS_CONNECTION (connection_by_uuid),
-			                                    nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection_new)),
-			                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP_SAVED,
-			                                    NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-			                                    "ifcfg-update",
-			                                    &local)) {
-				/* Shouldn't ever get here as 'connection_new' was verified by the reader already
-				 * and the UUID did not change. */
-				g_assert_not_reached ();
-			}
-			g_assert_no_error (local);
-
-			if (new_unmanaged || new_unrecognized) {
-				if (!old_unmanaged && !old_unrecognized) {
-					/* ref connection first, because we put it into priv->connections below.
-					 * Emitting signal-removed might otherwise delete it. */
-					g_object_ref (connection_by_uuid);
-
-					/* Unexport the connection by telling the settings service it's
-					 * been removed.
-					 */
-					nm_settings_connection_signal_remove (NM_SETTINGS_CONNECTION (connection_by_uuid));
-
-					/* signal_remove() will end up removing the connection from our hash,
-					 * so add it back now.
-					 */
-					g_hash_table_insert (priv->connections,
-					                     g_strdup (nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (connection_by_uuid))),
-					                     connection_by_uuid /* we took reference above and pass it on */);
-				}
-			} else {
-				if (old_unmanaged /* && !new_unmanaged */) {
-					_LOGI ("Managing connection "NM_IFCFG_CONNECTION_LOG_FMT" and its device because NM_CONTROLLED was true.",
-					       NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-					_nm_settings_plugin_emit_signal_connection_added (NM_SETTINGS_PLUGIN (self),
-					                                                  NM_SETTINGS_CONNECTION (connection_by_uuid));
-				} else if (old_unrecognized /* && !new_unrecognized */) {
-					_LOGI ("Managing connection "NM_IFCFG_CONNECTION_LOG_FMT" because it is now a recognized type.",
-					       NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-					_nm_settings_plugin_emit_signal_connection_added (NM_SETTINGS_PLUGIN (self),
-					                                                  NM_SETTINGS_CONNECTION (connection_by_uuid));
-				}
-			}
+	loaded_uuids = g_hash_table_new (nm_str_hash, g_str_equal);
+
+	storages_replaced = g_hash_table_new_full (nm_direct_hash, NULL, g_object_unref, NULL);
+
+	for (i = 0; i < n_entries; i++) {
+		NMSettingsPluginConnectionLoadEntry *const entry = &entries[i];
+		gs_free_error GError *local = NULL;
+		const char *full_filename;
+		const char *uuid;
+		gs_free char *full_filename_keep = NULL;
+		NMSettingsPluginConnectionLoadEntry *dupl_content_entry;
+		gs_unref_object NMSIfcfgRHStorage *storage = NULL;
 
-			if (unmanaged_changed)
-				_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
-			if (unrecognized_changed)
-				_nm_settings_plugin_emit_signal_unrecognized_specs_changed (NM_SETTINGS_PLUGIN (self));
+		if (entry->handled)
+			continue;
+
+		if (entry->filename[0] != '/')
+			continue;
+
+		full_filename_keep = utils_detect_ifcfg_path (entry->filename, FALSE);
+
+		if (!full_filename_keep) {
+			if (nm_utils_file_is_in_path (entry->filename, IFCFG_DIR)) {
+				nm_utils_error_set (&entry->error,
+				                    NM_UTILS_ERROR_UNKNOWN,
+				                    ("path is not a valid name for an ifcfg-rh file"));
+				entry->handled = TRUE;
+			}
+			continue;
 		}
-		nm_settings_connection_set_filename (NM_SETTINGS_CONNECTION (connection_by_uuid), full_path);
-		g_object_unref (connection_new);
-		return connection_by_uuid;
-	} else {
 
-		/*******************************************************
-		 * ADD
-		 *******************************************************/
+		if ((dupl_content_entry = g_hash_table_lookup (dupl_filenames, full_filename_keep))) {
+			/* we already visited this file. */
+			entry->handled = dupl_content_entry->handled;
+			if (dupl_content_entry->error) {
+				g_set_error_literal (&entry->error,
+				                     dupl_content_entry->error->domain,
+				                     dupl_content_entry->error->code,
+				                     dupl_content_entry->error->message);
+			}
+			continue;
+		}
 
-		if (source)
-			_LOGI ("add connection "NM_IFCFG_CONNECTION_LOG_FMT, NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-		else
-			_LOGI ("new connection "NM_IFCFG_CONNECTION_LOG_FMT, NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-		g_hash_table_insert (priv->connections,
-		                     g_strdup (uuid),
-		                     connection_new /* take reference */);
-
-		g_signal_connect (connection_new, NM_SETTINGS_CONNECTION_REMOVED,
-		                  G_CALLBACK (connection_removed_cb),
-		                  self);
-
-		if (nm_ifcfg_connection_get_unmanaged_spec (connection_new)) {
-			_LOGI ("Ignoring connection "NM_IFCFG_CONNECTION_LOG_FMT" due to NM_CONTROLLED=no. Unmanaged: %s.",
-			       NM_IFCFG_CONNECTION_LOG_ARG (connection_new),
-			       nm_ifcfg_connection_get_unmanaged_spec (connection_new));
-		} else if (nm_ifcfg_connection_get_unrecognized_spec (connection_new))
-			_LOGW ("Ignoring connection "NM_IFCFG_CONNECTION_LOG_FMT" of unrecognized type.", NM_IFCFG_CONNECTION_LOG_ARG (connection_new));
-
-		if (!source) {
-			/* Only raise the signal if we were called without source, i.e. if we read the connection from file.
-			 * Otherwise, we were called by add_connection() which does not expect the signal. */
-			if (nm_ifcfg_connection_get_unmanaged_spec (connection_new))
-				_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
-			else if (nm_ifcfg_connection_get_unrecognized_spec (connection_new))
-				_nm_settings_plugin_emit_signal_unrecognized_specs_changed (NM_SETTINGS_PLUGIN (self));
-			else {
-				_nm_settings_plugin_emit_signal_connection_added (NM_SETTINGS_PLUGIN (self),
-				                                                  NM_SETTINGS_CONNECTION (connection_new));
+		entry->handled = TRUE;
+
+		full_filename = full_filename_keep;
+		if (!g_hash_table_insert (dupl_filenames, g_steal_pointer (&full_filename_keep), entry))
+			nm_assert_not_reached ();
+
+		storage = _load_file (self,
+		                      full_filename,
+		                      &local);
+		if (!storage) {
+			if (nm_utils_file_stat (full_filename, NULL) == -ENOENT) {
+				NMSIfcfgRHStorage *storage2;
+
+				/* the file does not exist. We take that as indication to unload the file
+				 * that was previously loaded... */
+				storage2 = nm_sett_util_storages_lookup_by_filename (&priv->storages, full_filename);
+				if (storage2)
+					g_hash_table_add (storages_replaced, g_object_ref (storage2));
+				continue;
 			}
+			g_propagate_error (&entry->error, g_steal_pointer (&local));
+			continue;
 		}
-		return connection_new;
+
+		uuid = nms_ifcfg_rh_storage_get_uuid_opt (storage);
+		if (uuid)
+			g_hash_table_add (loaded_uuids, (char *) uuid);
+
+		nm_sett_util_storages_add_take (&storages_new, g_steal_pointer (&storage));
 	}
-}
 
-static GHashTable *
-_paths_from_connections (GHashTable *connections)
-{
-	GHashTableIter iter;
-	NMIfcfgConnection *connection;
-	GHashTable *paths = g_hash_table_new (nm_str_hash, g_str_equal);
+	/* now we visit all UUIDs that are about to change... */
+	g_hash_table_iter_init (&h_iter, loaded_uuids);
+	while (g_hash_table_iter_next (&h_iter, (gpointer *) &loaded_uuid, NULL)) {
+		NMSIfcfgRHStorage *storage;
+		NMSettUtilStorageByUuidHead *sbuh;
+
+		sbuh = nm_sett_util_storages_lookup_by_uuid (&priv->storages, loaded_uuid);
+		if (!sbuh)
+			continue;
+
+		c_list_for_each_entry (storage, &sbuh->_storage_by_uuid_lst_head, parent._storage_by_uuid_lst) {
+			const char *full_filename = nms_ifcfg_rh_storage_get_filename (storage);
+			gs_unref_object NMSIfcfgRHStorage *storage_new = NULL;
+			gs_free_error GError *local = NULL;
+
+			if (g_hash_table_contains (dupl_filenames, full_filename)) {
+				/* already re-loaded. */
+				continue;
+			}
 
-	g_hash_table_iter_init (&iter, connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer *) &connection)) {
-		const char *path = nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection));
+			/* @storage has a UUID that was just loaded from disk, but we have an entry in cache.
+			 * Reload that file too despite not being told to do so. The reason is to get
+			 * the latest file timestamp so that we get the priorities right. */
+
+			storage_new = _load_file (self,
+			                          full_filename,
+			                          &local);
+			if (   storage_new
+			    && !nm_streq0 (loaded_uuid, nms_ifcfg_rh_storage_get_uuid_opt (storage_new))) {
+				/* the file now references a different UUID. We are not told to reload
+				 * that file, so this means the existing storage (with the previous
+				 * filename and UUID tuple) is no longer valid. */
+				g_clear_object (&storage_new);
+			}
 
-		if (path)
-			g_hash_table_add (paths, (void *) path);
+			g_hash_table_add (storages_replaced, g_object_ref (storage));
+			if (storage_new)
+				nm_sett_util_storages_add_take (&storages_new, g_steal_pointer (&storage_new));
+		}
 	}
-	return paths;
+
+	nm_clear_pointer (&loaded_uuids, g_hash_table_destroy);
+	nm_clear_pointer (&dupl_filenames, g_hash_table_destroy);
+
+	_storages_consolidate (self,
+	                       &storages_new,
+	                       FALSE,
+	                       storages_replaced,
+	                       callback,
+	                       user_data);
 }
 
-static int
-_sort_paths (const char **f1, const char **f2, GHashTable *paths)
+static void
+reload_connections (NMSettingsPlugin *plugin,
+                    NMSettingsPluginConnectionLoadCallback callback,
+                    gpointer user_data)
 {
-	struct stat st;
-	gboolean c1, c2;
-	gint64 m1, m2;
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (plugin);
+	nm_auto_clear_sett_util_storages NMSettUtilStorages storages_new = NM_SETT_UTIL_STORAGES_INIT (storages_new, nms_ifcfg_rh_storage_destroy);
 
-	c1 = !!g_hash_table_contains (paths, *f1);
-	c2 = !!g_hash_table_contains (paths, *f2);
-	if (c1 != c2)
-		return c1 ? -1 : 1;
+	nm_assert_self (self, TRUE);
 
-	m1 = stat (*f1, &st) == 0 ? (gint64) st.st_mtime : G_MININT64;
-	m2 = stat (*f2, &st) == 0 ? (gint64) st.st_mtime : G_MININT64;
-	if (m1 != m2)
-		return m1 > m2 ? -1 : 1;
+	_load_dir (self, &storages_new);
 
-	return strcmp (*f1, *f2);
+	_storages_consolidate (self,
+	                       &storages_new,
+	                       TRUE,
+	                       NULL,
+	                       callback,
+	                       user_data);
+
+	nm_assert_self (self, FALSE);
 }
 
 static void
-read_connections (SettingsPluginIfcfg *plugin)
+load_connections_done (NMSettingsPlugin *plugin)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (plugin);
-	GDir *dir;
-	GError *err = NULL;
-	const char *item;
-	GHashTable *alive_connections;
-	GHashTableIter iter;
-	NMIfcfgConnection *connection;
-	GPtrArray *dead_connections = NULL;
-	guint i;
-	GPtrArray *filenames;
-	GHashTable *paths;
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (plugin);
 
-	dir = g_dir_open (IFCFG_DIR, 0, &err);
-	if (!dir) {
-		_LOGW ("Could not read directory '%s': %s", IFCFG_DIR, err->message);
-		g_error_free (err);
-		return;
+	/* at the beginning of a load, we emit a change signal for unmanaged/unrecognized
+	 * specs that contain the sum of before and after (_unhandled_specs_merge_storages()).
+	 *
+	 * The idea is that while we emit signals about changes to connection, we have
+	 * the sum of all unmanaged/unrecognized devices from before and after.
+	 *
+	 * This if triggered at the end, to reset the specs. */
+	_unhandled_specs_reset (self);
+
+	nm_assert_self (self, TRUE);
+}
+
+/*****************************************************************************/
+
+static gboolean
+add_connection (NMSettingsPlugin *plugin,
+                NMConnection *connection,
+                NMSettingsStorage **out_storage,
+                NMConnection **out_connection,
+                GError **error)
+{
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (plugin);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	gs_unref_object NMSIfcfgRHStorage *storage = NULL;
+	gs_unref_object NMConnection *reread = NULL;
+	gs_free char *full_filename = NULL;
+	GError *local = NULL;
+	gboolean reread_same;
+	struct timespec mtime;
+
+	nm_assert_self (self, TRUE);
+	nm_assert (NM_IS_CONNECTION (connection));
+	nm_assert (out_storage && !*out_storage);
+	nm_assert (out_connection && !*out_connection);
+
+	if (!nms_ifcfg_rh_writer_write_connection (connection,
+	                                           IFCFG_DIR,
+	                                           NULL,
+	                                           nm_sett_util_allow_filename_cb,
+	                                           NM_SETT_UTIL_ALLOW_FILENAME_DATA (&priv->storages, NULL),
+	                                           &full_filename,
+	                                           &reread,
+	                                           &reread_same,
+	                                           &local)) {
+		_LOGT ("commit: %s (%s): failed to add: %s",
+		       nm_connection_get_uuid (connection),
+		       nm_connection_get_id (connection),
+		       local->message);
+		g_propagate_error (error, local);
+		return FALSE;
 	}
 
-	alive_connections = g_hash_table_new (nm_direct_hash, NULL);
+	if (   !reread
+	    || reread_same)
+		nm_g_object_ref_set (&reread, connection);
 
-	filenames = g_ptr_array_new_with_free_func (g_free);
-	while ((item = g_dir_read_name (dir))) {
-		char *full_path, *real_path;
+	nm_assert (full_filename && full_filename[0] == '/');
 
-		full_path = g_build_filename (IFCFG_DIR, item, NULL);
-		real_path = utils_detect_ifcfg_path (full_path, TRUE);
+	_LOGT ("commit: %s (%s) added as \"%s\"",
+	       nm_connection_get_uuid (reread),
+	       nm_connection_get_id (reread),
+	       full_filename);
 
-		if (real_path)
-			g_ptr_array_add (filenames, real_path);
-		g_free (full_path);
-	}
-	g_dir_close (dir);
+	storage = nms_ifcfg_rh_storage_new_connection (self,
+	                                               full_filename,
+	                                               g_steal_pointer (&reread),
+	                                               nm_sett_util_stat_mtime (full_filename, FALSE, &mtime));
 
-	/* While reloading, we don't replace connections that we already loaded while
-	 * iterating over the files.
-	 *
-	 * To have sensible, reproducible behavior, sort the paths by last modification
-	 * time preferring older files.
-	 */
-	paths = _paths_from_connections (priv->connections);
-	g_ptr_array_sort_with_data (filenames, (GCompareDataFunc) _sort_paths, paths);
-	g_hash_table_destroy (paths);
-
-	for (i = 0; i < filenames->len; i++) {
-		connection = update_connection (plugin, NULL, filenames->pdata[i], NULL, FALSE, alive_connections, NULL);
-		if (connection)
-			g_hash_table_add (alive_connections, connection);
-	}
-	g_ptr_array_free (filenames, TRUE);
-
-	g_hash_table_iter_init (&iter, priv->connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer *) &connection)) {
-		if (   !g_hash_table_contains (alive_connections, connection)
-		    && nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection))) {
-			if (!dead_connections)
-				dead_connections = g_ptr_array_new ();
-			g_ptr_array_add (dead_connections, connection);
-		}
-	}
-	g_hash_table_destroy (alive_connections);
+	nm_sett_util_storages_add_take (&priv->storages, g_object_ref (storage));
 
-	if (dead_connections) {
-		for (i = 0; i < dead_connections->len; i++)
-			remove_connection (plugin, dead_connections->pdata[i]);
-		g_ptr_array_free (dead_connections, TRUE);
-	}
+	*out_connection = nms_ifcfg_rh_storage_steal_connection (storage);
+	*out_storage = NM_SETTINGS_STORAGE (g_steal_pointer (&storage));
+
+	nm_assert_self (self, TRUE);
+
+	return TRUE;
 }
 
-static GSList *
-get_connections (NMSettingsPlugin *config)
+static gboolean
+update_connection (NMSettingsPlugin *plugin,
+                   NMSettingsStorage *storage_x,
+                   NMConnection *connection,
+                   NMSettingsStorage **out_storage,
+                   NMConnection **out_connection,
+                   GError **error)
 {
-	SettingsPluginIfcfg *plugin = SETTINGS_PLUGIN_IFCFG (config);
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (plugin);
-	GSList *list = NULL;
-	GHashTableIter iter;
-	NMIfcfgConnection *connection;
-
-	if (!priv->initialized) {
-		read_connections (plugin);
-		priv->initialized = TRUE;
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (plugin);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	NMSIfcfgRHStorage *storage = NMS_IFCFG_RH_STORAGE (storage_x);
+	const char *full_filename;
+	const char *uuid;
+	GError *local = NULL;
+	gs_unref_object NMConnection *reread = NULL;
+	gboolean reread_same;
+	struct timespec mtime;
+
+	nm_assert_self (self, TRUE);
+	nm_assert (NM_IS_CONNECTION (connection));
+	nm_assert (NMS_IS_IFCFG_RH_STORAGE (storage));
+	nm_assert (_nm_connection_verify (connection, NULL) == NM_SETTING_VERIFY_SUCCESS);
+	nm_assert (!error || !*error);
+
+	uuid = nms_ifcfg_rh_storage_get_uuid_opt (storage);
+
+	nm_assert (uuid && nm_streq0 (uuid, nm_connection_get_uuid (connection)));
+
+	full_filename = nms_ifcfg_rh_storage_get_filename (storage);
+
+	nm_assert (full_filename);
+	nm_assert (storage == nm_sett_util_storages_lookup_by_filename (&priv->storages, full_filename));
+
+	if (!nms_ifcfg_rh_writer_write_connection (connection,
+	                                           IFCFG_DIR,
+	                                           full_filename,
+	                                           nm_sett_util_allow_filename_cb,
+	                                           NM_SETT_UTIL_ALLOW_FILENAME_DATA (&priv->storages, full_filename),
+	                                           NULL,
+	                                           &reread,
+	                                           &reread_same,
+	                                           &local)) {
+		_LOGT ("commit: failure to write %s (%s) to \"%s\": %s",
+		       nm_connection_get_uuid (connection),
+		       nm_connection_get_id (connection),
+		       full_filename,
+		       local->message);
+		g_propagate_error (error, local);
+		return FALSE;
 	}
 
-	g_hash_table_iter_init (&iter, priv->connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer) &connection)) {
-		if (   !nm_ifcfg_connection_get_unmanaged_spec (connection)
-		    && !nm_ifcfg_connection_get_unrecognized_spec (connection))
-			list = g_slist_prepend (list, connection);
-	}
+	if (   !reread
+	    || reread_same)
+		nm_g_object_ref_set (&reread, connection);
 
-	return list;
+	_LOGT ("commit: \"%s\": profile %s (%s) written",
+	       full_filename,
+	       uuid,
+	       nm_connection_get_id (connection));
+
+	storage->stat_mtime = *nm_sett_util_stat_mtime (full_filename, FALSE, &mtime);
+
+	*out_storage = NM_SETTINGS_STORAGE (g_object_ref (storage));
+	*out_connection = g_steal_pointer (&reread);
+
+	nm_assert_self (self, TRUE);
+
+	return TRUE;
 }
 
 static gboolean
-load_connection (NMSettingsPlugin *config,
-                 const char *filename)
+delete_connection (NMSettingsPlugin *plugin,
+                   NMSettingsStorage *storage_x,
+                   GError **error)
 {
-	SettingsPluginIfcfg *plugin = SETTINGS_PLUGIN_IFCFG (config);
-	NMIfcfgConnection *connection;
-	char *ifcfg_path;
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (plugin);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	NMSIfcfgRHStorage *storage = NMS_IFCFG_RH_STORAGE (storage_x);
+	const char *operation_message;
+	const char *full_filename;
 
-	if (!nm_utils_file_is_in_path (filename, IFCFG_DIR))
-		return FALSE;
+	nm_assert_self (self, TRUE);
+	nm_assert (!error || !*error);
+	nm_assert (NMS_IS_IFCFG_RH_STORAGE (storage));
 
-	/* get the real ifcfg-path. This allows us to properly
-	 * handle load command using a route-* file etc. */
-	ifcfg_path = utils_detect_ifcfg_path (filename, FALSE);
-	if (!ifcfg_path)
-		return FALSE;
+	full_filename = nms_ifcfg_rh_storage_get_filename (storage);
+	nm_assert (full_filename);
+
+	nm_assert (nms_ifcfg_rh_storage_get_uuid_opt (storage));
+
+	nm_assert (storage == nm_sett_util_storages_lookup_by_filename (&priv->storages, full_filename));
+
+	{
+		gs_free char *keyfile = utils_get_keys_path (full_filename);
+		gs_free char *routefile = utils_get_route_path (full_filename);
+		gs_free char *route6file = utils_get_route6_path (full_filename);
+		const char *const files[] = { full_filename, keyfile, routefile, route6file };
+		gboolean any_deleted = FALSE;
+		gboolean any_failure = FALSE;
+		int i;
+
+		for (i = 0; i < G_N_ELEMENTS (files); i++) {
+			int errsv;
+
+			if (unlink (files[i]) == 0) {
+				any_deleted = TRUE;
+				continue;
+			}
+			errsv = errno;
+			if (errsv == ENOENT)
+				continue;
+
+			_LOGW ("commit: failure to delete file \"%s\": %s",
+			       files[i],
+			       nm_strerror_native (errsv));
+			any_failure = TRUE;
+		}
+		if (any_failure)
+			operation_message = "failed to delete files from disk";
+		else if (any_deleted)
+			operation_message = "deleted from disk";
+		else
+			operation_message = "does not exist on disk";
+	}
 
-	connection = find_by_path (plugin, ifcfg_path);
-	update_connection (plugin, NULL, ifcfg_path, connection, TRUE, NULL, NULL);
-	if (!connection)
-		connection = find_by_path (plugin, ifcfg_path);
+	_LOGT ("commit: deleted \"%s\", profile %s (%s)",
+	       full_filename,
+	       nms_ifcfg_rh_storage_get_uuid_opt (storage),
+	       operation_message);
 
-	g_free (ifcfg_path);
-	return (connection != NULL);
+	nm_sett_util_storages_steal (&priv->storages, storage);
+	nms_ifcfg_rh_storage_destroy (storage);
+
+	nm_assert_self (self, TRUE);
+
+	return TRUE;
 }
 
+/*****************************************************************************/
+
 static void
-reload_connections (NMSettingsPlugin *config)
+_unhandled_specs_reset (NMSIfcfgRHPlugin *self)
 {
-	SettingsPluginIfcfg *plugin = SETTINGS_PLUGIN_IFCFG (config);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	gs_unref_hashtable GHashTable *unmanaged_specs = NULL;
+	gs_unref_hashtable GHashTable *unrecognized_specs = NULL;
+	NMSIfcfgRHStorage *storage;
+
+	unmanaged_specs = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
+	unrecognized_specs = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
+
+	c_list_for_each_entry (storage, &priv->storages._storage_lst_head, parent._storage_lst) {
+		if (storage->unmanaged_spec)
+			g_hash_table_add (unmanaged_specs, g_strdup (storage->unmanaged_spec));
+		if (storage->unrecognized_spec)
+			g_hash_table_add (unrecognized_specs, g_strdup (storage->unrecognized_spec));
+	}
+
+	if (!nm_utils_hashtable_same_keys (unmanaged_specs, priv->unmanaged_specs)) {
+		g_hash_table_unref (priv->unmanaged_specs);
+		priv->unmanaged_specs = g_steal_pointer (&unmanaged_specs);
+	}
+	if (!nm_utils_hashtable_same_keys (unrecognized_specs, priv->unrecognized_specs)) {
+		g_hash_table_unref (priv->unrecognized_specs);
+		priv->unrecognized_specs = g_steal_pointer (&unrecognized_specs);
+	}
 
-	read_connections (plugin);
+	if (!unmanaged_specs)
+		_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
+	if (!unrecognized_specs)
+		_nm_settings_plugin_emit_signal_unrecognized_specs_changed (NM_SETTINGS_PLUGIN (self));
 }
 
-static GSList *
-get_unhandled_specs (NMSettingsPlugin *config,
-                     const char *property)
+static void
+_unhandled_specs_merge_storages (NMSIfcfgRHPlugin *self,
+                                 NMSettUtilStorages *storages)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE ((SettingsPluginIfcfg *) config);
-	GSList *list = NULL, *list_iter;
-	GHashTableIter iter;
-	gpointer connection;
-	char *spec;
-	gboolean found;
-
-	g_hash_table_iter_init (&iter, priv->connections);
-	while (g_hash_table_iter_next (&iter, NULL, &connection)) {
-		g_object_get (connection, property, &spec, NULL);
-		if (spec) {
-			/* Ignore duplicates */
-			for (list_iter = list, found = FALSE; list_iter; list_iter = g_slist_next (list_iter)) {
-				if (g_str_equal (list_iter->data, spec)) {
-					found = TRUE;
-					break;
-				}
-			}
-			if (found)
-				g_free (spec);
-			else
-				list = g_slist_prepend (list, spec);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	gboolean unmanaged_changed = FALSE;
+	gboolean unrecognized_changed = FALSE;
+	NMSIfcfgRHStorage *storage;
+
+	c_list_for_each_entry (storage, &storages->_storage_lst_head, parent._storage_lst) {
+		if (   storage->unmanaged_spec
+		    && !g_hash_table_contains (priv->unmanaged_specs, storage->unmanaged_spec)) {
+			unmanaged_changed = TRUE;
+			g_hash_table_add (priv->unmanaged_specs, g_strdup (storage->unmanaged_spec));
+		}
+		if (   storage->unrecognized_spec
+		    && !g_hash_table_contains (priv->unrecognized_specs, storage->unrecognized_spec)) {
+			unrecognized_changed = TRUE;
+			g_hash_table_add (priv->unrecognized_specs, g_strdup (storage->unrecognized_spec));
 		}
 	}
-	return list;
+
+	if (unmanaged_changed)
+		_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
+	if (unrecognized_changed)
+		_nm_settings_plugin_emit_signal_unrecognized_specs_changed (NM_SETTINGS_PLUGIN (self));
 }
 
 static GSList *
-get_unmanaged_specs (NMSettingsPlugin *config)
+_unhandled_specs_from_hashtable (GHashTable *hash)
 {
-	return get_unhandled_specs (config, NM_IFCFG_CONNECTION_UNMANAGED_SPEC);
+	gs_free const char **keys = NULL;
+	GSList *list = NULL;
+	guint i, l;
+
+	keys = nm_utils_strdict_get_keys (hash, TRUE, &l);
+	for (i = l; i > 0; ) {
+		i--;
+		list = g_slist_prepend (list, g_strdup (keys[i]));
+	}
+	return list;
 }
 
 static GSList *
-get_unrecognized_specs (NMSettingsPlugin *config)
+get_unmanaged_specs (NMSettingsPlugin *plugin)
 {
-	return get_unhandled_specs (config, NM_IFCFG_CONNECTION_UNRECOGNIZED_SPEC);
+	return _unhandled_specs_from_hashtable (NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (plugin)->unmanaged_specs);
 }
 
-static NMSettingsConnection *
-add_connection (NMSettingsPlugin *config,
-                NMConnection *connection,
-                gboolean save_to_disk,
-                GError **error)
+static GSList *
+get_unrecognized_specs (NMSettingsPlugin *plugin)
 {
-	SettingsPluginIfcfg *self = SETTINGS_PLUGIN_IFCFG (config);
-	gs_free char *path = NULL;
-	gs_unref_object NMConnection *reread = NULL;
-
-	if (save_to_disk) {
-		if (!nms_ifcfg_rh_writer_write_connection (connection, IFCFG_DIR, NULL, NULL, NULL, &path, &reread, NULL, error))
-			return NULL;
-	} else {
-		if (!nms_ifcfg_rh_writer_can_write_connection (connection, error))
-			return NULL;
-	}
-	return NM_SETTINGS_CONNECTION (update_connection (self, reread ?: connection, path, NULL, FALSE, NULL, error));
+	return _unhandled_specs_from_hashtable (NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (plugin)->unrecognized_specs);
 }
 
+/*****************************************************************************/
+
 static void
-impl_ifcfgrh_get_ifcfg_details (SettingsPluginIfcfg *plugin,
+impl_ifcfgrh_get_ifcfg_details (NMSIfcfgRHPlugin *self,
                                 GDBusMethodInvocation *context,
                                 const char *in_ifcfg)
 {
-	NMIfcfgConnection *connection;
-	NMSettingConnection *s_con;
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+	gs_free char *ifcfg_path = NULL;
+	NMSIfcfgRHStorage *storage;
 	const char *uuid;
 	const char *path;
-	gs_free char *ifcfg_path = NULL;
 
-	if (!g_path_is_absolute (in_ifcfg)) {
+	if (in_ifcfg[0] != '/') {
 		g_dbus_method_invocation_return_error (context,
 		                                       NM_SETTINGS_ERROR,
 		                                       NM_SETTINGS_ERROR_INVALID_CONNECTION,
@@ -631,10 +916,8 @@ impl_ifcfgrh_get_ifcfg_details (SettingsPluginIfcfg *plugin,
 		return;
 	}
 
-	connection = find_by_path (plugin, ifcfg_path);
-	if (   !connection
-	    || nm_ifcfg_connection_get_unmanaged_spec (connection)
-	    || nm_ifcfg_connection_get_unrecognized_spec (connection)) {
+	storage = nm_sett_util_storages_lookup_by_filename (&priv->storages, ifcfg_path);
+	if (!storage) {
 		g_dbus_method_invocation_return_error (context,
 		                                       NM_SETTINGS_ERROR,
 		                                       NM_SETTINGS_ERROR_INVALID_CONNECTION,
@@ -642,25 +925,23 @@ impl_ifcfgrh_get_ifcfg_details (SettingsPluginIfcfg *plugin,
 		return;
 	}
 
-	s_con = nm_connection_get_setting_connection (nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection)));
-	if (!s_con) {
-		g_dbus_method_invocation_return_error (context,
-		                                       NM_SETTINGS_ERROR,
-		                                       NM_SETTINGS_ERROR_FAILED,
-		                                       "unable to retrieve the connection setting");
-		return;
-	}
-
-	uuid = nm_setting_connection_get_uuid (s_con);
+	uuid = nms_ifcfg_rh_storage_get_uuid_opt (storage);
 	if (!uuid) {
 		g_dbus_method_invocation_return_error (context,
 		                                       NM_SETTINGS_ERROR,
-		                                       NM_SETTINGS_ERROR_FAILED,
-		                                       "unable to get the UUID");
+		                                       NM_SETTINGS_ERROR_INVALID_CONNECTION,
+		                                       "ifcfg file '%s' not managed by NetworkManager", in_ifcfg);
 		return;
 	}
 
-	path = nm_dbus_object_get_path (NM_DBUS_OBJECT (connection));
+	/* It is ugly that the ifcfg-rh plugin needs to call back into NMSettings this
+	 * way.
+	 * There are alternatives (like invoking a signal), but they are all significant
+	 * extra code (and performance overhead). So the quick and dirty solution here
+	 * is likely to be simpler than getting this right (also from point of readability!).
+	 */
+	path = nm_settings_get_dbus_path_for_uuid (nm_settings_get (), uuid);
+
 	if (!path) {
 		g_dbus_method_invocation_return_error (context,
 		                                       NM_SETTINGS_ERROR,
@@ -676,9 +957,9 @@ impl_ifcfgrh_get_ifcfg_details (SettingsPluginIfcfg *plugin,
 /*****************************************************************************/
 
 static void
-_dbus_clear (SettingsPluginIfcfg *self)
+_dbus_clear (NMSIfcfgRHPlugin *self)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 	guint id;
 
 	nm_clear_g_signal_handler (priv->dbus.connection, &priv->dbus.signal_id);
@@ -700,7 +981,7 @@ _dbus_connection_closed (GDBusConnection *connection,
                          gpointer         user_data)
 {
 	_LOGW ("dbus: %s bus closed", IFCFGRH1_BUS_NAME);
-	_dbus_clear (SETTINGS_PLUGIN_IFCFG (user_data));
+	_dbus_clear (NMS_IFCFG_RH_PLUGIN (user_data));
 
 	/* Retry or recover? */
 }
@@ -715,21 +996,23 @@ _method_call (GDBusConnection *connection,
               GDBusMethodInvocation *invocation,
               gpointer user_data)
 {
-	SettingsPluginIfcfg *self = SETTINGS_PLUGIN_IFCFG (user_data);
-	const char *ifcfg;
-
-	if (   !nm_streq (interface_name, IFCFGRH1_IFACE1_NAME)
-	    || !nm_streq (method_name, IFCFGRH1_IFACE1_METHOD_GET_IFCFG_DETAILS)) {
-		g_dbus_method_invocation_return_error (invocation,
-		                                       G_DBUS_ERROR,
-		                                       G_DBUS_ERROR_UNKNOWN_METHOD,
-		                                       "Unknown method %s",
-		                                       method_name);
-		return;
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (user_data);
+
+	if (nm_streq (interface_name, IFCFGRH1_IFACE1_NAME)) {
+		if (nm_streq (method_name, IFCFGRH1_IFACE1_METHOD_GET_IFCFG_DETAILS)) {
+			const char *ifcfg;
+
+			g_variant_get (parameters, "(&s)", &ifcfg);
+			impl_ifcfgrh_get_ifcfg_details (self, invocation, ifcfg);
+			return;
+		}
 	}
 
-	g_variant_get (parameters, "(&s)", &ifcfg);
-	impl_ifcfgrh_get_ifcfg_details (self, invocation, ifcfg);
+	g_dbus_method_invocation_return_error (invocation,
+	                                       G_DBUS_ERROR,
+	                                       G_DBUS_ERROR_UNKNOWN_METHOD,
+	                                       "Unknown method %s",
+	                                       method_name);
 }
 
 static GDBusInterfaceInfo *const interface_info = NM_DEFINE_GDBUS_INTERFACE_INFO (
@@ -754,8 +1037,8 @@ _dbus_request_name_done (GObject *source_object,
                          gpointer user_data)
 {
 	GDBusConnection *connection = G_DBUS_CONNECTION (source_object);
-	SettingsPluginIfcfg *self;
-	SettingsPluginIfcfgPrivate *priv;
+	NMSIfcfgRHPlugin *self;
+	NMSIfcfgRHPluginPrivate *priv;
 	gs_free_error GError *error = NULL;
 	gs_unref_variant GVariant *ret = NULL;
 	guint32 result;
@@ -764,8 +1047,8 @@ _dbus_request_name_done (GObject *source_object,
 	if (g_error_matches (error, G_IO_ERROR, G_IO_ERROR_CANCELLED))
 		return;
 
-	self = SETTINGS_PLUGIN_IFCFG (user_data);
-	priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	self = NMS_IFCFG_RH_PLUGIN (user_data);
+	priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 
 	g_clear_object (&priv->dbus.cancellable);
 
@@ -812,8 +1095,8 @@ _dbus_create_done (GObject *source_object,
                    GAsyncResult *res,
                    gpointer user_data)
 {
-	SettingsPluginIfcfg *self;
-	SettingsPluginIfcfgPrivate *priv;
+	NMSIfcfgRHPlugin *self;
+	NMSIfcfgRHPluginPrivate *priv;
 	gs_free_error GError *error = NULL;
 	GDBusConnection *connection;
 
@@ -821,8 +1104,8 @@ _dbus_create_done (GObject *source_object,
 	if (g_error_matches (error, G_IO_ERROR, G_IO_ERROR_CANCELLED))
 		return;
 
-	self = SETTINGS_PLUGIN_IFCFG (user_data);
-	priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	self = NMS_IFCFG_RH_PLUGIN (user_data);
+	priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 
 	g_clear_object (&priv->dbus.cancellable);
 
@@ -856,9 +1139,9 @@ _dbus_create_done (GObject *source_object,
 }
 
 static void
-_dbus_setup (SettingsPluginIfcfg *self)
+_dbus_setup (NMSIfcfgRHPlugin *self)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 	gs_free char *address = NULL;
 	gs_free_error GError *error = NULL;
 
@@ -886,9 +1169,9 @@ config_changed_cb (NMConfig *config,
                    NMConfigData *config_data,
                    NMConfigChangeFlags changes,
                    NMConfigData *old_data,
-                   SettingsPluginIfcfg *self)
+                   NMSIfcfgRHPlugin *self)
 {
-	SettingsPluginIfcfgPrivate *priv;
+	NMSIfcfgRHPluginPrivate *priv;
 
 	/* If the dbus connection for some reason is borked the D-Bus service
 	 * won't be offered.
@@ -900,7 +1183,7 @@ config_changed_cb (NMConfig *config,
 	                            | NM_CONFIG_CHANGE_CAUSE_SIGUSR1))
 		return;
 
-	priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 	if (   !priv->dbus.connection
 	    && !priv->dbus.cancellable)
 		_dbus_setup (self);
@@ -909,23 +1192,26 @@ config_changed_cb (NMConfig *config,
 /*****************************************************************************/
 
 static void
-settings_plugin_ifcfg_init (SettingsPluginIfcfg *plugin)
+nms_ifcfg_rh_plugin_init (NMSIfcfgRHPlugin *self)
 {
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE ((SettingsPluginIfcfg *) plugin);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
+
+	priv->config = g_object_ref (nm_config_get ());
 
-	priv->connections = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, g_object_unref);
+	priv->unmanaged_specs = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
+	priv->unrecognized_specs = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
+
+	priv->storages = (NMSettUtilStorages) NM_SETT_UTIL_STORAGES_INIT (priv->storages, nms_ifcfg_rh_storage_destroy);
 }
 
 static void
 constructed (GObject *object)
 {
-	SettingsPluginIfcfg *self = SETTINGS_PLUGIN_IFCFG (object);
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (object);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 
-	G_OBJECT_CLASS (settings_plugin_ifcfg_parent_class)->constructed (object);
+	G_OBJECT_CLASS (nms_ifcfg_rh_plugin_parent_class)->constructed (object);
 
-	priv->config = nm_config_get ();
-	g_object_add_weak_pointer ((GObject *) priv->config, (gpointer *) &priv->config);
 	g_signal_connect (priv->config,
 	                  NM_CONFIG_SIGNAL_CONFIG_CHANGED,
 	                  G_CALLBACK (config_changed_cb),
@@ -937,40 +1223,44 @@ constructed (GObject *object)
 static void
 dispose (GObject *object)
 {
-	SettingsPluginIfcfg *self = SETTINGS_PLUGIN_IFCFG (object);
-	SettingsPluginIfcfgPrivate *priv = SETTINGS_PLUGIN_IFCFG_GET_PRIVATE (self);
+	NMSIfcfgRHPlugin *self = NMS_IFCFG_RH_PLUGIN (object);
+	NMSIfcfgRHPluginPrivate *priv = NMS_IFCFG_RH_PLUGIN_GET_PRIVATE (self);
 
-	if (priv->config) {
-		g_object_remove_weak_pointer ((GObject *) priv->config, (gpointer *) &priv->config);
+	if (priv->config)
 		g_signal_handlers_disconnect_by_func (priv->config, config_changed_cb, self);
-		priv->config = NULL;
-	}
 
+	/* FIXME(shutdown) we need a stop method so that we can unregistering the D-Bus service
+	 * when NMSettings is shutting down, and not when the instance gets destroyed. */
 	_dbus_clear (self);
 
-	if (priv->connections) {
-		g_hash_table_destroy (priv->connections);
-		priv->connections = NULL;
-	}
+	nm_sett_util_storages_clear (&priv->storages);
+
+	g_clear_object (&priv->config);
+
+	G_OBJECT_CLASS (nms_ifcfg_rh_plugin_parent_class)->dispose (object);
 
-	G_OBJECT_CLASS (settings_plugin_ifcfg_parent_class)->dispose (object);
+	nm_clear_pointer (&priv->unmanaged_specs, g_hash_table_destroy);
+	nm_clear_pointer (&priv->unrecognized_specs, g_hash_table_destroy);
 }
 
 static void
-settings_plugin_ifcfg_class_init (SettingsPluginIfcfgClass *klass)
+nms_ifcfg_rh_plugin_class_init (NMSIfcfgRHPluginClass *klass)
 {
 	GObjectClass *object_class = G_OBJECT_CLASS (klass);
 	NMSettingsPluginClass *plugin_class = NM_SETTINGS_PLUGIN_CLASS (klass);
 
 	object_class->constructed = constructed;
-	object_class->dispose = dispose;
+	object_class->dispose     = dispose;
 
-	plugin_class->get_connections = get_connections;
-	plugin_class->add_connection = add_connection;
-	plugin_class->load_connection = load_connection;
-	plugin_class->reload_connections = reload_connections;
-	plugin_class->get_unmanaged_specs = get_unmanaged_specs;
+	plugin_class->plugin_name            = "ifcfg-rh";
+	plugin_class->get_unmanaged_specs    = get_unmanaged_specs;
 	plugin_class->get_unrecognized_specs = get_unrecognized_specs;
+	plugin_class->reload_connections     = reload_connections;
+	plugin_class->load_connections       = load_connections;
+	plugin_class->load_connections_done  = load_connections_done;
+	plugin_class->add_connection         = add_connection;
+	plugin_class->update_connection      = update_connection;
+	plugin_class->delete_connection      = delete_connection;
 }
 
 /*****************************************************************************/
@@ -978,5 +1268,5 @@ settings_plugin_ifcfg_class_init (SettingsPluginIfcfgClass *klass)
 G_MODULE_EXPORT NMSettingsPlugin *
 nm_settings_plugin_factory (void)
 {
-	return g_object_new (SETTINGS_TYPE_PLUGIN_IFCFG, NULL);
+	return g_object_new (NMS_TYPE_IFCFG_RH_PLUGIN, NULL);
 }
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.h b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.h
index 88c3dc7939..1db36083ff 100644
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.h
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-plugin.h
@@ -20,19 +20,19 @@
  * Copyright (C) 2007 - 2008 Red Hat, Inc.
  */
 
-#ifndef _PLUGIN_H_
-#define _PLUGIN_H_
+#ifndef __NMS_IFCFG_RH_PLUGIN_H__
+#define __NMS_IFCFG_RH_PLUGIN_H__
 
-#define SETTINGS_TYPE_PLUGIN_IFCFG            (settings_plugin_ifcfg_get_type ())
-#define SETTINGS_PLUGIN_IFCFG(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), SETTINGS_TYPE_PLUGIN_IFCFG, SettingsPluginIfcfg))
-#define SETTINGS_PLUGIN_IFCFG_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), SETTINGS_TYPE_PLUGIN_IFCFG, SettingsPluginIfcfgClass))
-#define SETTINGS_IS_PLUGIN_IFCFG(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), SETTINGS_TYPE_PLUGIN_IFCFG))
-#define SETTINGS_IS_PLUGIN_IFCFG_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), SETTINGS_TYPE_PLUGIN_IFCFG))
-#define SETTINGS_PLUGIN_IFCFG_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), SETTINGS_TYPE_PLUGIN_IFCFG, SettingsPluginIfcfgClass))
+#define NMS_TYPE_IFCFG_RH_PLUGIN            (nms_ifcfg_rh_plugin_get_type ())
+#define NMS_IFCFG_RH_PLUGIN(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NMS_TYPE_IFCFG_RH_PLUGIN, NMSIfcfgRHPlugin))
+#define NMS_IFCFG_RH_PLUGIN_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NMS_TYPE_IFCFG_RH_PLUGIN, NMSIfcfgRHPluginClass))
+#define NMS_IS_IFCFG_RH_PLUGIN(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NMS_TYPE_IFCFG_RH_PLUGIN))
+#define NMS_IS_IFCFG_RH_PLUGIN_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NMS_TYPE_IFCFG_RH_PLUGIN))
+#define NMS_IFCFG_RH_PLUGIN_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NMS_TYPE_IFCFG_RH_PLUGIN, NMSIfcfgRHPluginClass))
 
-typedef struct _SettingsPluginIfcfg SettingsPluginIfcfg;
-typedef struct _SettingsPluginIfcfgClass SettingsPluginIfcfgClass;
+typedef struct _NMSIfcfgRHPlugin NMSIfcfgRHPlugin;
+typedef struct _NMSIfcfgRHPluginClass NMSIfcfgRHPluginClass;
 
-GType settings_plugin_ifcfg_get_type (void);
+GType nms_ifcfg_rh_plugin_get_type (void);
 
-#endif /* _PLUGIN_H_ */
+#endif /* __NMS_IFCFG_RH_PLUGIN_H__ */
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.c b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.c
index 99e48d6b2a..177c60ff4a 100644
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.c
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.c
@@ -6028,20 +6028,3 @@ nmtst_connection_from_file (const char *filename,
 	                                  error,
 	                                  NULL);
 }
-
-guint
-devtimeout_from_file (const char *filename)
-{
-	shvarFile *ifcfg;
-	guint devtimeout;
-
-	g_return_val_if_fail (filename != NULL, 0);
-
-	ifcfg = svOpenFile (filename, NULL);
-	if (!ifcfg)
-		return 0;
-
-	devtimeout = svGetValueInt64 (ifcfg, "DEVTIMEOUT", 10, 0, G_MAXUINT, 0);
-	svCloseFile (ifcfg);
-	return devtimeout;
-}
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.h b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.h
index be402565a8..8008e052bc 100644
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.h
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.h
@@ -17,8 +17,8 @@
  * Copyright (C) 2008 Red Hat, Inc.
  */
 
-#ifndef __READER_H__
-#define __READER_H__
+#ifndef __NMS_IFCFG_RH_READER_H__
+#define __NMS_IFCFG_RH_READER_H__
 
 #include "nm-connection.h"
 
@@ -27,12 +27,10 @@ NMConnection *connection_from_file (const char *filename,
                                     GError **error,
                                     gboolean *out_ignore_error);
 
-guint devtimeout_from_file (const char *filename);
-
 NMConnection *nmtst_connection_from_file (const char *filename,
                                           const char *network_file,
                                           const char *test_type,
                                           char **out_unhandled,
                                           GError **error);
 
-#endif  /* __READER_H__ */
+#endif  /* __NMS_IFCFG_RH_READER_H__ */
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.c b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.c
new file mode 100644
index 0000000000..760d924d4a
--- /dev/null
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.c
@@ -0,0 +1,198 @@
+/* NetworkManager
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Copyright (C) 2018 Red Hat, Inc.
+ */
+
+#include "nm-default.h"
+
+#include "nms-ifcfg-rh-storage.h"
+
+#include "nm-utils.h"
+#include "nm-core-internal.h"
+#include "nm-connection.h"
+#include "nms-ifcfg-rh-plugin.h"
+
+/*****************************************************************************/
+
+struct _NMSIfcfgRHStorageClass {
+	NMSettingsStorageClass parent;
+};
+
+G_DEFINE_TYPE (NMSIfcfgRHStorage, nms_ifcfg_rh_storage, NM_TYPE_SETTINGS_STORAGE)
+
+/*****************************************************************************/
+
+gboolean
+nms_ifcfg_rh_storage_equal_type (const NMSIfcfgRHStorage *self_a,
+                                 const NMSIfcfgRHStorage *self_b)
+{
+	return    (self_a == self_b)
+	       || (   self_a
+	           && self_b
+	           && nm_streq0 (nms_ifcfg_rh_storage_get_uuid_opt (self_a),
+	                         nms_ifcfg_rh_storage_get_uuid_opt (self_b))
+	           && nm_streq0 (self_a->unmanaged_spec,
+	                         self_b->unmanaged_spec)
+	           && nm_streq0 (self_a->unrecognized_spec,
+	                         self_b->unrecognized_spec));
+}
+
+void
+nms_ifcfg_rh_storage_copy_content (NMSIfcfgRHStorage *dst,
+                                   const NMSIfcfgRHStorage *src)
+{
+	nm_assert (src != dst);
+	nm_assert (src && dst);
+	nm_assert (nms_ifcfg_rh_storage_equal_type (dst, src));
+	nm_assert (   nms_ifcfg_rh_storage_get_filename (dst)
+	           && nm_streq (nms_ifcfg_rh_storage_get_filename (dst),
+	                        nms_ifcfg_rh_storage_get_filename (src)));
+
+	nm_g_object_ref_set (&dst->connection, src->connection);
+	g_free (dst->unmanaged_spec);
+	g_free (dst->unrecognized_spec);
+	dst->unmanaged_spec    = g_strdup (src->unmanaged_spec);
+	dst->unrecognized_spec = g_strdup (src->unrecognized_spec);
+	dst->stat_mtime        = src->stat_mtime;
+}
+
+NMConnection *
+nms_ifcfg_rh_storage_steal_connection (NMSIfcfgRHStorage *self)
+{
+	nm_assert (NMS_IS_IFCFG_RH_STORAGE (self));
+
+	return g_steal_pointer (&self->connection);
+}
+
+/*****************************************************************************/
+
+static int
+cmp_fcn (const NMSIfcfgRHStorage *a,
+         const NMSIfcfgRHStorage *b)
+{
+	nm_assert (NMS_IS_IFCFG_RH_STORAGE (a));
+	nm_assert (NMS_IS_IFCFG_RH_STORAGE (b));
+	nm_assert (a != b);
+
+	/* newer files are more important. */
+	NM_CMP_FIELD (b, a, stat_mtime.tv_sec);
+	NM_CMP_FIELD (b, a, stat_mtime.tv_nsec);
+
+	NM_CMP_DIRECT_STRCMP (nms_ifcfg_rh_storage_get_filename (a), nms_ifcfg_rh_storage_get_filename (b));
+
+	return 0;
+}
+
+/*****************************************************************************/
+
+static void
+nms_ifcfg_rh_storage_init (NMSIfcfgRHStorage *self)
+{
+}
+
+static NMSIfcfgRHStorage *
+_storage_new (NMSIfcfgRHPlugin *plugin,
+              const char *uuid,
+              const char *filename)
+{
+	nm_assert (NMS_IS_IFCFG_RH_PLUGIN (plugin));
+	nm_assert (!uuid || nm_utils_is_uuid (uuid));
+	nm_assert (filename && filename[0] == '/');
+
+	return g_object_new (NMS_TYPE_IFCFG_RH_STORAGE,
+	                     NM_SETTINGS_STORAGE_PLUGIN, plugin,
+	                     NM_SETTINGS_STORAGE_UUID, uuid,
+	                     NM_SETTINGS_STORAGE_FILENAME, filename,
+	                     NULL);
+}
+
+NMSIfcfgRHStorage *
+nms_ifcfg_rh_storage_new_connection (NMSIfcfgRHPlugin *plugin,
+                                     const char *filename,
+                                     NMConnection *connection_take,
+                                     const struct timespec *mtime)
+{
+	NMSIfcfgRHStorage *self;
+
+	nm_assert (NM_IS_CONNECTION (connection_take));
+	nm_assert (_nm_connection_verify (connection_take, NULL) == NM_SETTING_VERIFY_SUCCESS);
+	nmtst_connection_assert_unchanging (connection_take);
+
+	self = _storage_new (plugin,
+	                     nm_connection_get_uuid (connection_take),
+	                     filename);
+	self->connection = connection_take;
+	if (mtime)
+		self->stat_mtime = *mtime;
+	return self;
+}
+
+NMSIfcfgRHStorage *
+nms_ifcfg_rh_storage_new_unhandled (NMSIfcfgRHPlugin *plugin,
+                                    const char *filename,
+                                    const char *unmanaged_spec,
+                                    const char *unrecognized_spec)
+{
+	NMSIfcfgRHStorage *self;
+
+	nm_assert (unmanaged_spec || unrecognized_spec);
+
+	self = _storage_new (plugin,
+	                     NULL,
+	                     filename);
+	self->unmanaged_spec = g_strdup (unmanaged_spec);
+	self->unrecognized_spec = g_strdup (unrecognized_spec);
+	return self;
+}
+
+static void
+_storage_clear (NMSIfcfgRHStorage *self)
+{
+	c_list_unlink (&self->parent._storage_lst);
+	c_list_unlink (&self->parent._storage_by_uuid_lst);
+	nm_clear_g_free (&self->unmanaged_spec);
+	nm_clear_g_free (&self->unrecognized_spec);
+	g_clear_object (&self->connection);
+}
+
+static void
+dispose (GObject *object)
+{
+	NMSIfcfgRHStorage *self = NMS_IFCFG_RH_STORAGE (object);
+
+	_storage_clear (self);
+
+	G_OBJECT_CLASS (nms_ifcfg_rh_storage_parent_class)->dispose (object);
+}
+
+void
+nms_ifcfg_rh_storage_destroy (NMSIfcfgRHStorage *self)
+{
+	_storage_clear (self);
+	g_object_unref (self);
+}
+
+static void
+nms_ifcfg_rh_storage_class_init (NMSIfcfgRHStorageClass *klass)
+{
+	GObjectClass *object_class = G_OBJECT_CLASS (klass);
+	NMSettingsStorageClass *storage_class = NM_SETTINGS_STORAGE_CLASS (klass);
+
+	object_class->dispose = dispose;
+
+	storage_class->cmp_fcn = (int (*) (NMSettingsStorage *, NMSettingsStorage *)) cmp_fcn;
+}
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.h b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.h
new file mode 100644
index 0000000000..e1165f5093
--- /dev/null
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-storage.h
@@ -0,0 +1,93 @@
+/* NetworkManager
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Copyright (C) 2019 Red Hat, Inc.
+ */
+
+#ifndef __NMS_IFCFG_RH_STORAGE_H__
+#define __NMS_IFCFG_RH_STORAGE_H__
+
+#include "c-list/src/c-list.h"
+#include "settings/nm-settings-storage.h"
+
+/*****************************************************************************/
+
+#define NMS_TYPE_IFCFG_RH_STORAGE            (nms_ifcfg_rh_storage_get_type ())
+#define NMS_IFCFG_RH_STORAGE(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NMS_TYPE_IFCFG_RH_STORAGE, NMSIfcfgRHStorage))
+#define NMS_IFCFG_RH_STORAGE_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NMS_TYPE_IFCFG_RH_STORAGE, NMSIfcfgRHStorageClass))
+#define NMS_IS_IFCFG_RH_STORAGE(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NMS_TYPE_IFCFG_RH_STORAGE))
+#define NMS_IS_IFCFG_RH_STORAGE_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NMS_TYPE_IFCFG_RH_STORAGE))
+#define NMS_IFCFG_RH_STORAGE_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NMS_TYPE_IFCFG_RH_STORAGE, NMSIfcfgRHStorageClass))
+
+typedef struct {
+	NMSettingsStorage parent;
+
+	NMConnection *connection;
+
+	char *unmanaged_spec;
+	char *unrecognized_spec;
+
+	/* The timestamp (stat's mtime) of the file. Newer files have
+	 * higher priority. */
+	struct timespec stat_mtime;
+
+	bool dirty:1;
+
+} NMSIfcfgRHStorage;
+
+typedef struct _NMSIfcfgRHStorageClass NMSIfcfgRHStorageClass;
+
+GType nms_ifcfg_rh_storage_get_type (void);
+
+struct _NMSIfcfgRHPlugin;
+
+NMSIfcfgRHStorage *nms_ifcfg_rh_storage_new_connection (struct _NMSIfcfgRHPlugin *plugin,
+                                                        const char *filename,
+                                                        NMConnection *connection_take,
+                                                        const struct timespec *mtime);
+
+NMSIfcfgRHStorage *nms_ifcfg_rh_storage_new_unhandled (struct _NMSIfcfgRHPlugin *plugin,
+                                                       const char *filename,
+                                                       const char *unmanaged_spec,
+                                                       const char *unrecognized_spec);
+
+void nms_ifcfg_rh_storage_destroy (NMSIfcfgRHStorage *self);
+
+/*****************************************************************************/
+
+gboolean nms_ifcfg_rh_storage_equal_type (const NMSIfcfgRHStorage *self_a,
+                                          const NMSIfcfgRHStorage *self_b);
+
+void nms_ifcfg_rh_storage_copy_content (NMSIfcfgRHStorage *dst,
+                                        const NMSIfcfgRHStorage *src);
+
+NMConnection *nms_ifcfg_rh_storage_steal_connection (NMSIfcfgRHStorage *self);
+
+/*****************************************************************************/
+
+static inline const char *
+nms_ifcfg_rh_storage_get_uuid_opt (const NMSIfcfgRHStorage *self)
+{
+	return nm_settings_storage_get_uuid_opt ((const NMSettingsStorage *) self);
+}
+
+static inline const char *
+nms_ifcfg_rh_storage_get_filename (const NMSIfcfgRHStorage *self)
+{
+	return nm_settings_storage_get_filename ((const NMSettingsStorage *) self);
+}
+
+#endif /* __NMS_IFCFG_RH_STORAGE_H__ */
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.c b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.c
index 038f3dacc9..cb1fc23a3a 100644
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.c
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.c
@@ -28,6 +28,32 @@
 
 #include "nms-ifcfg-rh-common.h"
 
+/*****************************************************************************/
+
+gboolean
+nms_ifcfg_rh_util_parse_unhandled_spec (const char *unhandled_spec,
+                                        const char **out_unmanaged_spec,
+                                        const char **out_unrecognized_spec)
+{
+	if (unhandled_spec) {
+		if (NM_STR_HAS_PREFIX (unhandled_spec, "unmanaged:")) {
+			NM_SET_OUT (out_unmanaged_spec, &unhandled_spec[NM_STRLEN ("unmanaged:")]);
+			NM_SET_OUT (out_unrecognized_spec, NULL);
+			return TRUE;
+		}
+		if (NM_STR_HAS_PREFIX (unhandled_spec, "unrecognized:")) {
+			NM_SET_OUT (out_unmanaged_spec, NULL);
+			NM_SET_OUT (out_unrecognized_spec, &unhandled_spec[NM_STRLEN ("unrecognized:")]);
+			return TRUE;
+		}
+	}
+	NM_SET_OUT (out_unmanaged_spec, NULL);
+	NM_SET_OUT (out_unrecognized_spec, NULL);
+	return FALSE;
+}
+
+/*****************************************************************************/
+
 /*
  * Check ';[a-fA-F0-9]{8}' file suffix used for temporary files by rpm when
  * installing packages.
diff --git a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.h b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.h
index c9e3446809..20d6f72dd8 100644
--- a/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.h
+++ b/src/settings/plugins/ifcfg-rh/nms-ifcfg-rh-utils.h
@@ -25,6 +25,10 @@
 
 #include "shvar.h"
 
+gboolean nms_ifcfg_rh_util_parse_unhandled_spec (const char *unhandled_spec,
+                                                 const char **out_unmanaged_spec,
+                                                 const char **out_unrecognized_spec);
+
 #define NM_IFCFG_CONNECTION_LOG_PATH(path)  ((path) ?: "in-memory")
 #define NM_IFCFG_CONNECTION_LOG_FMT         "%s (%s,\"%s\")"
 #define NM_IFCFG_CONNECTION_LOG_ARG(con)    NM_IFCFG_CONNECTION_LOG_PATH (nm_settings_connection_get_filename ((NMSettingsConnection *) (con))), nm_settings_connection_get_uuid ((NMSettingsConnection *) (con)), nm_settings_connection_get_id ((NMSettingsConnection *) (con))
diff --git a/src/settings/plugins/ifupdown/meson.build b/src/settings/plugins/ifupdown/meson.build
index 42edd4388a..365ae1a9c5 100644
--- a/src/settings/plugins/ifupdown/meson.build
+++ b/src/settings/plugins/ifupdown/meson.build
@@ -15,7 +15,6 @@ libnms_ifupdown_core = static_library(
 )
 
 sources = files(
-  'nms-ifupdown-connection.c',
   'nms-ifupdown-plugin.c',
 )
 
diff --git a/src/settings/plugins/ifupdown/nms-ifupdown-connection.c b/src/settings/plugins/ifupdown/nms-ifupdown-connection.c
deleted file mode 100644
index 13187c4454..0000000000
--- a/src/settings/plugins/ifupdown/nms-ifupdown-connection.c
+++ /dev/null
@@ -1,91 +0,0 @@
-/* NetworkManager system settings service (ifupdown)
- *
- * Alexander Sack <asac@ubuntu.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * (C) Copyright 2007,2008 Canonical Ltd.
- */
-
-#include "nm-default.h"
-
-#include "nms-ifupdown-connection.h"
-
-#include <glib/gstdio.h>
-
-#include "nm-dbus-interface.h"
-#include "nm-utils.h"
-#include "nm-setting-wireless-security.h"
-#include "settings/nm-settings-connection.h"
-#include "settings/nm-settings-plugin.h"
-
-#include "nms-ifupdown-parser.h"
-
-/*****************************************************************************/
-
-struct _NMIfupdownConnection {
-	NMSettingsConnection parent;
-};
-
-struct _NMIfupdownConnectionClass {
-	NMSettingsConnectionClass parent;
-};
-
-G_DEFINE_TYPE (NMIfupdownConnection, nm_ifupdown_connection, NM_TYPE_SETTINGS_CONNECTION)
-
-/*****************************************************************************/
-
-#define _NMLOG_PREFIX_NAME      "ifupdown"
-#define _NMLOG_DOMAIN           LOGD_SETTINGS
-#define _NMLOG(level, ...) \
-    nm_log ((level), _NMLOG_DOMAIN, NULL, NULL, \
-            "%s" _NM_UTILS_MACRO_FIRST (__VA_ARGS__), \
-            _NMLOG_PREFIX_NAME": " \
-            _NM_UTILS_MACRO_REST (__VA_ARGS__))
-
-/*****************************************************************************/
-
-static void
-nm_ifupdown_connection_init (NMIfupdownConnection *connection)
-{
-}
-
-NMIfupdownConnection *
-nm_ifupdown_connection_new (if_block *block)
-{
-	NMIfupdownConnection *connection;
-	GError *error = NULL;
-
-	g_return_val_if_fail (block != NULL, NULL);
-
-	connection = g_object_new (NM_TYPE_IFUPDOWN_CONNECTION, NULL);
-
-	/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-	if (!ifupdown_update_connection_from_if_block (nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection)),
-	                                               block,
-	                                               &error)) {
-		_LOGW ("invalid connection read from /etc/network/interfaces: %s",
-		       error->message);
-		g_object_unref (connection);
-		return NULL;
-	}
-
-	return connection;
-}
-
-static void
-nm_ifupdown_connection_class_init (NMIfupdownConnectionClass *ifupdown_connection_class)
-{
-}
diff --git a/src/settings/plugins/ifupdown/nms-ifupdown-connection.h b/src/settings/plugins/ifupdown/nms-ifupdown-connection.h
deleted file mode 100644
index eee3ae0ff1..0000000000
--- a/src/settings/plugins/ifupdown/nms-ifupdown-connection.h
+++ /dev/null
@@ -1,43 +0,0 @@
-/* NetworkManager system settings service (ifupdown)
- *
- * Alexander Sack <asac@ubuntu.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * (C) Copyright 2008 Canonical Ltd.
- */
-
-#ifndef __NETWORKMANAGER_IFUPDOWN_CONNECTION_H__
-#define __NETWORKMANAGER_IFUPDOWN_CONNECTION_H__
-
-#include "settings/nm-settings-connection.h"
-
-#include "nms-ifupdown-interface-parser.h"
-
-#define NM_TYPE_IFUPDOWN_CONNECTION            (nm_ifupdown_connection_get_type ())
-#define NM_IFUPDOWN_CONNECTION(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NM_TYPE_IFUPDOWN_CONNECTION, NMIfupdownConnection))
-#define NM_IFUPDOWN_CONNECTION_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NM_TYPE_IFUPDOWN_CONNECTION, NMIfupdownConnectionClass))
-#define NM_IS_IFUPDOWN_CONNECTION(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NM_TYPE_IFUPDOWN_CONNECTION))
-#define NM_IS_IFUPDOWN_CONNECTION_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NM_TYPE_IFUPDOWN_CONNECTION))
-#define NM_IFUPDOWN_CONNECTION_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NM_TYPE_IFUPDOWN_CONNECTION, NMIfupdownConnectionClass))
-
-typedef struct _NMIfupdownConnection NMIfupdownConnection;
-typedef struct _NMIfupdownConnectionClass NMIfupdownConnectionClass;
-
-GType nm_ifupdown_connection_get_type (void);
-
-NMIfupdownConnection *nm_ifupdown_connection_new (if_block *block);
-
-#endif /* __NETWORKMANAGER_IFUPDOWN_CONNECTION_H__ */
diff --git a/src/settings/plugins/ifupdown/nms-ifupdown-parser.c b/src/settings/plugins/ifupdown/nms-ifupdown-parser.c
index f83c5ea0ef..b65013a431 100644
--- a/src/settings/plugins/ifupdown/nms-ifupdown-parser.c
+++ b/src/settings/plugins/ifupdown/nms-ifupdown-parser.c
@@ -651,22 +651,21 @@ update_ip6_setting_from_if_block (NMConnection *connection,
 	return TRUE;
 }
 
-gboolean
-ifupdown_update_connection_from_if_block (NMConnection *connection,
-                                          if_block *block,
-                                          GError **error)
+NMConnection *
+ifupdown_new_connection_from_if_block (if_block *block,
+                                       gboolean autoconnect,
+                                       GError **error)
 {
+	gs_unref_object NMConnection *connection = NULL;
 	const char *type;
 	gs_free char *idstr = NULL;
 	gs_free char *uuid = NULL;
 	NMSettingConnection *s_con;
-	gboolean success = FALSE;
 
-	s_con = nm_connection_get_setting_connection (connection);
-	if (!s_con) {
-		s_con = NM_SETTING_CONNECTION (nm_setting_connection_new ());
-		nm_connection_add_setting (connection, NM_SETTING (s_con));
-	}
+	connection = nm_simple_connection_new ();
+
+	s_con = NM_SETTING_CONNECTION (nm_setting_connection_new ());
+	nm_connection_add_setting (connection, NM_SETTING (s_con));
 
 	type = _ifupdownplugin_guess_connection_type (block);
 	idstr = g_strconcat ("Ifupdown (", block->name, ")", NULL);
@@ -678,10 +677,10 @@ ifupdown_update_connection_from_if_block (NMConnection *connection,
 	              NM_SETTING_CONNECTION_ID, idstr,
 	              NM_SETTING_CONNECTION_UUID, uuid,
 	              NM_SETTING_CONNECTION_READ_ONLY, TRUE,
-	              NM_SETTING_CONNECTION_AUTOCONNECT, FALSE,
+	              NM_SETTING_CONNECTION_AUTOCONNECT, (gboolean) (!!autoconnect),
 	              NULL);
 
-	_LOGI ("update_connection_setting_from_if_block: name:%s, type:%s, id:%s, uuid: %s",
+	_LOGD ("update_connection_setting_from_if_block: name:%s, type:%s, id:%s, uuid: %s",
 	       block->name, type, idstr, nm_setting_connection_get_uuid (s_con));
 
 	if (nm_streq (type, NM_SETTING_WIRED_SETTING_NAME))
@@ -691,13 +690,16 @@ ifupdown_update_connection_from_if_block (NMConnection *connection,
 		update_wireless_security_setting_from_if_block (connection, block);
 	}
 
-	if (ifparser_haskey (block, "inet6"))
-		success = update_ip6_setting_from_if_block (connection, block, error);
-	else
-		success = update_ip4_setting_from_if_block (connection, block, error);
+	if (ifparser_haskey (block, "inet6")) {
+		if (!update_ip6_setting_from_if_block (connection, block, error))
+			return FALSE;
+	} else {
+		if (!update_ip4_setting_from_if_block (connection, block, error))
+			return FALSE;
+	}
 
-	if (success == TRUE)
-		success = nm_connection_verify (connection, error);
+	if (!nm_connection_normalize (connection, NULL, NULL, error))
+		return NULL;
 
-	return success;
+	return g_steal_pointer (&connection);
 }
diff --git a/src/settings/plugins/ifupdown/nms-ifupdown-parser.h b/src/settings/plugins/ifupdown/nms-ifupdown-parser.h
index c17e92392f..7569648fab 100644
--- a/src/settings/plugins/ifupdown/nms-ifupdown-parser.h
+++ b/src/settings/plugins/ifupdown/nms-ifupdown-parser.h
@@ -19,16 +19,14 @@
  * (C) Copyright 2008 Canonical Ltd.
  */
 
-#ifndef __PARSER_H__
-#define __PARSER_H__
+#ifndef __NMS_IFUPDOWN_PARSER_H__
+#define __NMS_IFUPDOWN_PARSER_H__
 
 #include "nm-connection.h"
-
 #include "nms-ifupdown-interface-parser.h"
 
-gboolean
-ifupdown_update_connection_from_if_block (NMConnection *connection,
-                                          if_block *block,
-                                          GError **error);
+NMConnection *ifupdown_new_connection_from_if_block (if_block *block,
+                                                     gboolean autoconnect,
+                                                     GError **error);
 
-#endif /* __PARSER_H__ */
+#endif /* __NMS_IFUPDOWN_PARSER_H__ */
diff --git a/src/settings/plugins/ifupdown/nms-ifupdown-plugin.c b/src/settings/plugins/ifupdown/nms-ifupdown-plugin.c
index 23f4015609..94426d8b60 100644
--- a/src/settings/plugins/ifupdown/nms-ifupdown-plugin.c
+++ b/src/settings/plugins/ifupdown/nms-ifupdown-plugin.c
@@ -24,23 +24,12 @@
 
 #include "nms-ifupdown-plugin.h"
 
-#include <arpa/inet.h>
-#include <gmodule.h>
-
-#include "nm-setting-connection.h"
-#include "nm-dbus-interface.h"
-#include "settings/nm-settings-plugin.h"
-#include "nm-setting-ip4-config.h"
-#include "nm-setting-wireless.h"
-#include "nm-setting-wired.h"
-#include "nm-setting-ppp.h"
-#include "nm-utils.h"
 #include "nm-core-internal.h"
-#include "NetworkManagerUtils.h"
 #include "nm-config.h"
+#include "settings/nm-settings-plugin.h"
+#include "settings/nm-settings-storage.h"
 
 #include "nms-ifupdown-interface-parser.h"
-#include "nms-ifupdown-connection.h"
 #include "nms-ifupdown-parser.h"
 
 #define ENI_INTERFACES_FILE "/etc/network/interfaces"
@@ -50,28 +39,35 @@
 /*****************************************************************************/
 
 typedef struct {
+	NMConnection *connection;
+	NMSettingsStorage *storage;
+} StorageData;
+
+typedef struct {
 	/* Stores an entry for blocks/interfaces read from /e/n/i and (if exists)
-	 * the NMIfupdownConnection associated with the block.
+	 * the StorageData associated with the block.
 	 */
 	GHashTable *eni_ifaces;
 
 	bool ifupdown_managed:1;
 
 	bool initialized:1;
-} SettingsPluginIfupdownPrivate;
 
-struct _SettingsPluginIfupdown {
+	bool already_reloaded:1;
+} NMSIfupdownPluginPrivate;
+
+struct _NMSIfupdownPlugin {
 	NMSettingsPlugin parent;
-	SettingsPluginIfupdownPrivate _priv;
+	NMSIfupdownPluginPrivate _priv;
 };
 
-struct _SettingsPluginIfupdownClass {
+struct _NMSIfupdownPluginClass {
 	NMSettingsPluginClass parent;
 };
 
-G_DEFINE_TYPE (SettingsPluginIfupdown, settings_plugin_ifupdown, NM_TYPE_SETTINGS_PLUGIN)
+G_DEFINE_TYPE (NMSIfupdownPlugin, nms_ifupdown_plugin, NM_TYPE_SETTINGS_PLUGIN)
 
-#define SETTINGS_PLUGIN_IFUPDOWN_GET_PRIVATE(self) _NM_GET_PRIVATE (self, SettingsPluginIfupdown, SETTINGS_IS_PLUGIN_IFUPDOWN)
+#define NMS_IFUPDOWN_PLUGIN_GET_PRIVATE(self) _NM_GET_PRIVATE (self, NMSIfupdownPlugin, NMS_IS_IFUPDOWN_PLUGIN)
 
 /*****************************************************************************/
 
@@ -85,53 +81,143 @@ G_DEFINE_TYPE (SettingsPluginIfupdown, settings_plugin_ifupdown, NM_TYPE_SETTING
 
 /*****************************************************************************/
 
-static void initialize (SettingsPluginIfupdown *self);
+static GHashTable *load_eni_ifaces (NMSIfupdownPlugin *self);
 
 /*****************************************************************************/
 
-/* Returns the plugins currently known list of connections.  The returned
- * list is freed by the system settings service.
- */
-static GSList*
-get_connections (NMSettingsPlugin *plugin)
+static void
+_storage_data_destroy (StorageData *sd)
+{
+	nm_g_object_unref (sd->connection);
+	nm_g_object_unref (sd->storage);
+	g_slice_free (StorageData, sd);
+}
+
+/*****************************************************************************/
+
+static void
+initialize (NMSIfupdownPlugin *self)
+{
+	NMSIfupdownPluginPrivate *priv = NMS_IFUPDOWN_PLUGIN_GET_PRIVATE (self);
+	gboolean ifupdown_managed;
+
+	nm_assert (!priv->initialized);
+
+	priv->initialized = TRUE;
+
+	ifupdown_managed = nm_config_data_get_value_boolean (NM_CONFIG_GET_DATA_ORIG,
+	                                                     NM_CONFIG_KEYFILE_GROUP_IFUPDOWN,
+	                                                     NM_CONFIG_KEYFILE_KEY_IFUPDOWN_MANAGED,
+	                                                     !IFUPDOWN_UNMANAGE_WELL_KNOWN_DEFAULT);
+	_LOGI ("management mode: %s", ifupdown_managed ? "managed" : "unmanaged");
+	priv->ifupdown_managed = ifupdown_managed;
+
+	priv->eni_ifaces = load_eni_ifaces (self);
+}
+
+static void
+reload_connections (NMSettingsPlugin *plugin,
+                    NMSettingsPluginConnectionLoadCallback callback,
+                    gpointer user_data)
 {
-	SettingsPluginIfupdown *self = SETTINGS_PLUGIN_IFUPDOWN (plugin);
-	SettingsPluginIfupdownPrivate *priv = SETTINGS_PLUGIN_IFUPDOWN_GET_PRIVATE (self);
-	GSList *list = NULL;
+	NMSIfupdownPlugin *self = NMS_IFUPDOWN_PLUGIN (plugin);
+	NMSIfupdownPluginPrivate *priv = NMS_IFUPDOWN_PLUGIN_GET_PRIVATE (self);
+	gs_unref_hashtable GHashTable *eni_ifaces_old = NULL;
 	GHashTableIter iter;
-	void *value;
+	StorageData *sd;
+	StorageData *sd2;
+	const char *block_name;
 
-	if (G_UNLIKELY (!priv->initialized))
+	if (!priv->initialized)
 		initialize (self);
+	else if (!priv->already_reloaded) {
+		/* This is the first call to reload, but we are already initialized.
+		 *
+		 * This happens because during start NMSettings first queries unmanaged-specs,
+		 * and then issues a reload call right away.
+		 *
+		 * On future reloads, we really want to load /e/n/i again. */
+		priv->already_reloaded = TRUE;
+	} else {
+		eni_ifaces_old = priv->eni_ifaces;
+		priv->eni_ifaces = load_eni_ifaces (self);
+
+		g_hash_table_iter_init (&iter, eni_ifaces_old);
+		while (g_hash_table_iter_next (&iter, (gpointer *) &block_name, (gpointer *) &sd)) {
+			if (!sd)
+				continue;
 
-	if (!priv->ifupdown_managed) {
-		_LOGD ("get_connections: not connections due to managed=false");
-		return NULL;
+			sd2 = g_hash_table_lookup (priv->eni_ifaces, block_name);
+			if (!sd2)
+				continue;
+
+			nm_assert (nm_streq (nm_settings_storage_get_uuid (sd->storage), nm_settings_storage_get_uuid (sd2->storage)));
+			nm_g_object_ref_set (&sd2->storage, sd->storage);
+			g_hash_table_iter_remove (&iter);
+		}
 	}
 
+	if (!priv->ifupdown_managed)
+		_LOGD ("load: no connections due to managed=false");
+
 	g_hash_table_iter_init (&iter, priv->eni_ifaces);
-	while (g_hash_table_iter_next (&iter, NULL, &value)) {
-		if (value)
-			list = g_slist_prepend (list, value);
+	while (g_hash_table_iter_next (&iter, NULL, (gpointer *) &sd)) {
+		gs_unref_object NMConnection *connection = NULL;
+
+		if (!sd)
+			continue;
+
+		connection = g_steal_pointer (&sd->connection);
+
+		if (!priv->ifupdown_managed)
+			continue;
+
+		_LOGD ("load: %s (%s)",
+		        nm_settings_storage_get_uuid (sd->storage),
+		        nm_connection_get_id (connection));
+		callback (plugin,
+		          sd->storage,
+		          connection,
+		          user_data);
+	}
+	if (   eni_ifaces_old
+	    && priv->ifupdown_managed) {
+		g_hash_table_iter_init (&iter, eni_ifaces_old);
+		while (g_hash_table_iter_next (&iter, NULL, (gpointer *) &sd)) {
+			if (!sd)
+				continue;
+			_LOGD ("unload: %s",
+			        nm_settings_storage_get_uuid (sd->storage));
+			callback (plugin,
+			          sd->storage,
+			          NULL,
+			          user_data);
+		}
 	}
+}
+
+/*****************************************************************************/
+
+static GSList *
+_unmanaged_specs (GHashTable *eni_ifaces)
+{
+	gs_free const char **keys = NULL;
+	GSList *specs = NULL;
+	guint i, len;
 
-	_LOGD ("get_connections: %u connections", g_slist_length (list));
-	return list;
+	keys = nm_utils_strdict_get_keys (eni_ifaces, TRUE, &len);
+	for (i = len; i > 0; ) {
+		i--;
+		specs = g_slist_prepend (specs, g_strdup_printf ("interface-name:=%s", keys[i]));
+	}
+	return specs;
 }
 
-/*
- * Return a list of device specifications which NetworkManager should not
- * manage.  Returned list will be freed by the system settings service, and
- * each element must be allocated using g_malloc() or its variants.
- */
 static GSList*
 get_unmanaged_specs (NMSettingsPlugin *plugin)
 {
-	SettingsPluginIfupdown *self = SETTINGS_PLUGIN_IFUPDOWN (plugin);
-	SettingsPluginIfupdownPrivate *priv = SETTINGS_PLUGIN_IFUPDOWN_GET_PRIVATE (self);
-	GSList *specs = NULL;
-	GHashTableIter iter;
-	const char *iface;
+	NMSIfupdownPlugin *self = NMS_IFUPDOWN_PLUGIN (plugin);
+	NMSIfupdownPluginPrivate *priv = NMS_IFUPDOWN_PLUGIN_GET_PRIVATE (self);
 
 	if (G_UNLIKELY (!priv->initialized))
 		initialize (self);
@@ -142,40 +228,46 @@ get_unmanaged_specs (NMSettingsPlugin *plugin)
 	_LOGD ("unmanaged-specs: unmanaged devices count %u",
 	       g_hash_table_size (priv->eni_ifaces));
 
-	g_hash_table_iter_init (&iter, priv->eni_ifaces);
-	while (g_hash_table_iter_next (&iter, (gpointer) &iface, NULL))
-		specs = g_slist_append (specs, g_strdup_printf ("interface-name:=%s", iface));
-	return specs;
+	return _unmanaged_specs (priv->eni_ifaces);
 }
 
 /*****************************************************************************/
 
-static void
-initialize (SettingsPluginIfupdown *self)
+static GHashTable *
+load_eni_ifaces (NMSIfupdownPlugin *self)
 {
-	SettingsPluginIfupdownPrivate *priv = SETTINGS_PLUGIN_IFUPDOWN_GET_PRIVATE (self);
+	gs_unref_hashtable GHashTable *eni_ifaces = NULL;
 	gs_unref_hashtable GHashTable *auto_ifaces = NULL;
 	nm_auto_ifparser if_parser *parser = NULL;
 	if_block *block;
-	GHashTableIter con_iter;
-	const char *block_name;
-	NMIfupdownConnection *conn;
+	StorageData *sd;
 
-	nm_assert (!priv->initialized);
-	priv->initialized = TRUE;
+	eni_ifaces = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, (GDestroyNotify) _storage_data_destroy);
 
 	parser = ifparser_parse (ENI_INTERFACES_FILE, 0);
 
 	c_list_for_each_entry (block, &parser->block_lst_head, block_lst) {
-
-		if (NM_IN_STRSET (block->type, "auto", "allow-hotplug")) {
+		if (NM_IN_STRSET (block->type, "auto",
+		                               "allow-hotplug")) {
 			if (!auto_ifaces)
-				auto_ifaces = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
-			g_hash_table_add (auto_ifaces, g_strdup (block->name));
-			continue;
+				auto_ifaces = g_hash_table_new (nm_str_hash, g_str_equal);
+			g_hash_table_add (auto_ifaces, (char *) block->name);
 		}
+	}
+
+	c_list_for_each_entry (block, &parser->block_lst_head, block_lst) {
+
+		if (NM_IN_STRSET (block->type, "auto",
+		                               "allow-hotplug"))
+			continue;
 
 		if (nm_streq (block->type, "iface")) {
+			gs_free_error GError *local = NULL;
+			gs_unref_object NMConnection *connection = NULL;
+			gs_unref_object NMSettingsStorage *storage = NULL;
+			const char *uuid = NULL;
+			StorageData *sd_repl;
+
 			/* Bridge configuration */
 			if (g_str_has_prefix (block->name, "br")) {
 				/* Try to find bridge ports */
@@ -208,13 +300,13 @@ initialize (SettingsPluginIfupdown *self)
 						if (nm_streq (token, "none"))
 							continue;
 						if (state == 0) {
-							conn = g_hash_table_lookup (priv->eni_ifaces, block->name);
-							if (!conn) {
+							sd = g_hash_table_lookup (eni_ifaces, block->name);
+							if (!sd) {
 								_LOGD ("parse: adding bridge port \"%s\"", token);
-								g_hash_table_insert (priv->eni_ifaces, g_strdup (token), NULL);
+								g_hash_table_insert (eni_ifaces, g_strdup (token), NULL);
 							} else {
 								_LOGD ("parse: adding bridge port \"%s\" (have connection %s)", token,
-								       nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (conn)));
+								       nm_settings_storage_get_uuid (sd->storage));
 							}
 						}
 					}
@@ -226,106 +318,91 @@ initialize (SettingsPluginIfupdown *self)
 			if (nm_streq (block->name, "lo"))
 				continue;
 
-			/* Remove any connection for this block that was previously found */
-			conn = g_hash_table_lookup (priv->eni_ifaces, block->name);
-			if (conn) {
+			sd_repl = g_hash_table_lookup (eni_ifaces, block->name);
+			if (sd_repl) {
+				storage = g_steal_pointer (&sd_repl->storage);
 				_LOGD ("parse: replace connection \"%s\" (%s)",
 				       block->name,
-				       nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (conn)));
-				nm_settings_connection_delete (NM_SETTINGS_CONNECTION (conn), NULL);
-				g_hash_table_remove (priv->eni_ifaces, block->name);
+				       nm_settings_storage_get_uuid (sd_repl->storage));
+				g_hash_table_remove (eni_ifaces, block->name);
+			}
+
+			connection = ifupdown_new_connection_from_if_block (block,
+			                                                       auto_ifaces
+			                                                    && g_hash_table_contains (auto_ifaces, block->name),
+			                                                    &local);
+
+			if (!connection) {
+				_LOGD ("parse: adding place holder for \"%s\"%s%s%s",
+				       block->name,
+				       NM_PRINT_FMT_QUOTED (local, " (", local->message, ")", ""));
+				sd = NULL;
+			} else {
+
+				nmtst_connection_assert_unchanging (connection);
+				uuid = nm_connection_get_uuid (connection);
+
+				if (!storage)
+					storage = nm_settings_storage_new (NM_SETTINGS_PLUGIN (self), uuid, NULL);
+
+				sd = g_slice_new (StorageData);
+				*sd = (StorageData) {
+					.connection = g_steal_pointer (&connection),
+					.storage    = g_steal_pointer (&storage),
+				};
+				_LOGD ("parse: adding connection \"%s\" (%s)", block->name, uuid);
 			}
 
-			/* add the new connection */
-			conn = nm_ifupdown_connection_new (block);
-			if (conn) {
-				_LOGD ("parse: adding connection \"%s\" (%s)", block->name,
-				       nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (conn)));
-			} else
-				_LOGD ("parse: adding place holder for connection \"%s\"", block->name);
-			g_hash_table_insert (priv->eni_ifaces, g_strdup (block->name), conn);
+			g_hash_table_replace (eni_ifaces, g_strdup (block->name), sd);
 			continue;
 		}
 
 		if (nm_streq (block->type, "mapping")) {
-			conn = g_hash_table_lookup (priv->eni_ifaces, block->name);
-			if (!conn) {
+			sd = g_hash_table_lookup (eni_ifaces, block->name);
+			if (!sd) {
 				_LOGD ("parse: adding mapping \"%s\"", block->name);
-				g_hash_table_insert (priv->eni_ifaces, g_strdup (block->name), NULL);
+				g_hash_table_insert (eni_ifaces, g_strdup (block->name), NULL);
 			} else {
 				_LOGD ("parse: adding mapping \"%s\" (have connection %s)", block->name,
-				       nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (conn)));
+				       nm_settings_storage_get_uuid (sd->storage));
 			}
 			continue;
 		}
 	}
 
-	/* Make 'auto' interfaces autoconnect=TRUE */
-	g_hash_table_iter_init (&con_iter, priv->eni_ifaces);
-	while (g_hash_table_iter_next (&con_iter, (gpointer) &block_name, (gpointer) &conn)) {
-		NMSettingConnection *setting;
-
-		if (   !conn
-		    || !auto_ifaces
-		    || !g_hash_table_contains (auto_ifaces, block_name))
-			continue;
-
-		/* FIXME(copy-on-write-connection): avoid modifying NMConnection instances and share them via copy-on-write. */
-		setting = nm_connection_get_setting_connection (nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (conn)));
-		g_object_set (setting, NM_SETTING_CONNECTION_AUTOCONNECT, TRUE, NULL);
-	}
+	nm_clear_pointer (&auto_ifaces, g_hash_table_destroy);
 
-	/* Check the config file to find out whether to manage interfaces */
-	priv->ifupdown_managed = nm_config_data_get_value_boolean (NM_CONFIG_GET_DATA_ORIG,
-	                                                           NM_CONFIG_KEYFILE_GROUP_IFUPDOWN,
-	                                                           NM_CONFIG_KEYFILE_KEY_IFUPDOWN_MANAGED,
-	                                                           !IFUPDOWN_UNMANAGE_WELL_KNOWN_DEFAULT);
-	_LOGI ("management mode: %s", priv->ifupdown_managed ? "managed" : "unmanaged");
-
-	/* Now if we're running in managed mode, let NM know there are new connections */
-	if (priv->ifupdown_managed) {
-		GHashTableIter iter;
-
-		g_hash_table_iter_init (&iter, priv->eni_ifaces);
-		while (g_hash_table_iter_next (&iter, NULL, (gpointer) &conn)) {
-			if (conn) {
-				_nm_settings_plugin_emit_signal_connection_added (NM_SETTINGS_PLUGIN (self),
-				                                                  NM_SETTINGS_CONNECTION (conn));
-			}
-		}
-	}
+	return g_steal_pointer (&eni_ifaces);
 }
 
 /*****************************************************************************/
 
 static void
-settings_plugin_ifupdown_init (SettingsPluginIfupdown *self)
+nms_ifupdown_plugin_init (NMSIfupdownPlugin *self)
 {
-	SettingsPluginIfupdownPrivate *priv = SETTINGS_PLUGIN_IFUPDOWN_GET_PRIVATE (self);
-
-	priv->eni_ifaces = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, g_object_unref);
 }
 
 static void
 dispose (GObject *object)
 {
-	SettingsPluginIfupdown *plugin = SETTINGS_PLUGIN_IFUPDOWN (object);
-	SettingsPluginIfupdownPrivate *priv = SETTINGS_PLUGIN_IFUPDOWN_GET_PRIVATE (plugin);
+	NMSIfupdownPlugin *plugin = NMS_IFUPDOWN_PLUGIN (object);
+	NMSIfupdownPluginPrivate *priv = NMS_IFUPDOWN_PLUGIN_GET_PRIVATE (plugin);
 
 	g_clear_pointer (&priv->eni_ifaces, g_hash_table_destroy);
 
-	G_OBJECT_CLASS (settings_plugin_ifupdown_parent_class)->dispose (object);
+	G_OBJECT_CLASS (nms_ifupdown_plugin_parent_class)->dispose (object);
 }
 
 static void
-settings_plugin_ifupdown_class_init (SettingsPluginIfupdownClass *klass)
+nms_ifupdown_plugin_class_init (NMSIfupdownPluginClass *klass)
 {
 	GObjectClass *object_class = G_OBJECT_CLASS (klass);
 	NMSettingsPluginClass *plugin_class = NM_SETTINGS_PLUGIN_CLASS (klass);
 
 	object_class->dispose = dispose;
 
-	plugin_class->get_connections     = get_connections;
+	plugin_class->plugin_name         = "ifupdown";
+	plugin_class->reload_connections  = reload_connections;
 	plugin_class->get_unmanaged_specs = get_unmanaged_specs;
 }
 
@@ -334,5 +411,5 @@ settings_plugin_ifupdown_class_init (SettingsPluginIfupdownClass *klass)
 G_MODULE_EXPORT NMSettingsPlugin *
 nm_settings_plugin_factory (void)
 {
-	return g_object_new (SETTINGS_TYPE_PLUGIN_IFUPDOWN, NULL);
+	return g_object_new (NMS_TYPE_IFUPDOWN_PLUGIN, NULL);
 }
diff --git a/src/settings/plugins/ifupdown/nms-ifupdown-plugin.h b/src/settings/plugins/ifupdown/nms-ifupdown-plugin.h
index 6e0b74ee6b..10ea2be44e 100644
--- a/src/settings/plugins/ifupdown/nms-ifupdown-plugin.h
+++ b/src/settings/plugins/ifupdown/nms-ifupdown-plugin.h
@@ -19,21 +19,21 @@
  * (C) Copyright 2008 Canonical Ltd.
  */
 
-#ifndef _PLUGIN_H_
-#define _PLUGIN_H_
+#ifndef __NMS_IFUPDOWN_PLUGIN_H__
+#define __NMS_IFUPDOWN_PLUGIN_H__
 
 #define PLUGIN_NAME "ifupdown"
 
-#define SETTINGS_TYPE_PLUGIN_IFUPDOWN            (settings_plugin_ifupdown_get_type ())
-#define SETTINGS_PLUGIN_IFUPDOWN(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), SETTINGS_TYPE_PLUGIN_IFUPDOWN, SettingsPluginIfupdown))
-#define SETTINGS_PLUGIN_IFUPDOWN_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), SETTINGS_TYPE_PLUGIN_IFUPDOWN, SettingsPluginIfupdownClass))
-#define SETTINGS_IS_PLUGIN_IFUPDOWN(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), SETTINGS_TYPE_PLUGIN_IFUPDOWN))
-#define SETTINGS_IS_PLUGIN_IFUPDOWN_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), SETTINGS_TYPE_PLUGIN_IFUPDOWN))
-#define SETTINGS_PLUGIN_IFUPDOWN_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), SETTINGS_TYPE_PLUGIN_IFUPDOWN, SettingsPluginIfupdownClass))
+#define NMS_TYPE_IFUPDOWN_PLUGIN            (nms_ifupdown_plugin_get_type ())
+#define NMS_IFUPDOWN_PLUGIN(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NMS_TYPE_IFUPDOWN_PLUGIN, NMSIfupdownPlugin))
+#define NMS_IFUPDOWN_PLUGIN_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NMS_TYPE_IFUPDOWN_PLUGIN, NMSIfupdownPluginClass))
+#define NMS_IS_IFUPDOWN_PLUGIN(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NMS_TYPE_IFUPDOWN_PLUGIN))
+#define NMS_IS_IFUPDOWN_PLUGIN_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NMS_TYPE_IFUPDOWN_PLUGIN))
+#define NMS_IFUPDOWN_PLUGIN_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NMS_TYPE_IFUPDOWN_PLUGIN, NMSIfupdownPluginClass))
 
-typedef struct _SettingsPluginIfupdown SettingsPluginIfupdown;
-typedef struct _SettingsPluginIfupdownClass SettingsPluginIfupdownClass;
+typedef struct _NMSIfupdownPlugin NMSIfupdownPlugin;
+typedef struct _NMSIfupdownPluginClass NMSIfupdownPluginClass;
 
-GType settings_plugin_ifupdown_get_type (void);
+GType nms_ifupdown_plugin_get_type (void);
 
-#endif /* _PLUGIN_H_ */
+#endif /* __NMS_IFUPDOWN_PLUGIN_H__ */
diff --git a/src/settings/plugins/ifupdown/tests/test-ifupdown.c b/src/settings/plugins/ifupdown/tests/test-ifupdown.c
index 1191047dc2..4adcf085e5 100644
--- a/src/settings/plugins/ifupdown/tests/test-ifupdown.c
+++ b/src/settings/plugins/ifupdown/tests/test-ifupdown.c
@@ -30,6 +30,29 @@
 
 /*****************************************************************************/
 
+#define _connection_from_if_block(block) \
+	({ \
+		NMConnection *_con; \
+		if_block *_block = (block); \
+		GError *_local = NULL; \
+		\
+		g_assert (_block); \
+		_con = ifupdown_new_connection_from_if_block (_block, FALSE, &_local); \
+		nmtst_assert_success (NM_IS_CONNECTION (_con), _local); \
+		nmtst_assert_connection_verifies_without_normalization (_con); \
+		_con; \
+	})
+
+#define _connection_first_from_parser(parser) \
+	({ \
+		if_parser *_parser = (parser); \
+		\
+		g_assert (_parser); \
+		_connection_from_if_block (ifparser_getfirst (_parser)); \
+	})
+
+/*****************************************************************************/
+
 typedef struct {
 	char *key;
 	char *data;
@@ -452,26 +475,14 @@ test16_missing_newline (void)
 static void
 test17_read_static_ipv4 (void)
 {
-	NMConnection *connection;
+	gs_unref_object NMConnection *connection = NULL;
 	NMSettingConnection *s_con;
 	NMSettingIPConfig *s_ip4;
 	NMSettingWired *s_wired;
-	GError *error = NULL;
-	gboolean success;
 	NMIPAddress *ip4_addr;
-	if_block *block = NULL;
 	nm_auto_ifparser if_parser *parser = init_ifparser_with_file ("test17-wired-static-verify-ip4");
 
-	block = ifparser_getfirst (parser);
-	connection = nm_simple_connection_new();
-	g_assert (connection);
-
-	ifupdown_update_connection_from_if_block (connection, block, &error);
-	g_assert_no_error (error);
-
-	success = nm_connection_verify (connection, &error);
-	g_assert_no_error (error);
-	g_assert (success);
+	connection = _connection_first_from_parser (parser);
 
 	/* ===== CONNECTION SETTING ===== */
 	s_con = nm_connection_get_setting_connection (connection);
@@ -500,32 +511,19 @@ test17_read_static_ipv4 (void)
 	g_assert_cmpint (nm_setting_ip_config_get_num_dns_searches (s_ip4), ==, 2);
 	g_assert_cmpstr (nm_setting_ip_config_get_dns_search (s_ip4, 0), ==, "example.com");
 	g_assert_cmpstr (nm_setting_ip_config_get_dns_search (s_ip4, 1), ==, "foo.example.com");
-
-	g_object_unref (connection);
 }
 
 static void
 test18_read_static_ipv6 (void)
 {
-	NMConnection *connection;
+	gs_unref_object NMConnection *connection = NULL;
 	NMSettingConnection *s_con;
 	NMSettingIPConfig *s_ip6;
 	NMSettingWired *s_wired;
-	GError *error = NULL;
-	gboolean success;
 	NMIPAddress *ip6_addr;
-	if_block *block = NULL;
 	nm_auto_ifparser if_parser *parser = init_ifparser_with_file ("test18-wired-static-verify-ip6");
 
-	block = ifparser_getfirst (parser);
-	connection = nm_simple_connection_new();
-	g_assert (connection);
-	ifupdown_update_connection_from_if_block (connection, block, &error);
-	g_assert_no_error (error);
-
-	success = nm_connection_verify (connection, &error);
-	g_assert_no_error (error);
-	g_assert (success);
+	connection = _connection_first_from_parser (parser);
 
 	/* ===== CONNECTION SETTING ===== */
 	s_con = nm_connection_get_setting_connection (connection);
@@ -554,30 +552,17 @@ test18_read_static_ipv6 (void)
 	g_assert_cmpint (nm_setting_ip_config_get_num_dns_searches (s_ip6), ==, 2);
 	g_assert_cmpstr (nm_setting_ip_config_get_dns_search (s_ip6, 0), ==, "example.com");
 	g_assert_cmpstr (nm_setting_ip_config_get_dns_search (s_ip6, 1), ==, "foo.example.com");
-
-	g_object_unref (connection);
 }
 
 static void
 test19_read_static_ipv4_plen (void)
 {
-	NMConnection *connection;
+	gs_unref_object NMConnection *connection = NULL;
 	NMSettingIPConfig *s_ip4;
-	GError *error = NULL;
 	NMIPAddress *ip4_addr;
-	if_block *block = NULL;
-	gboolean success;
 	nm_auto_ifparser if_parser *parser = init_ifparser_with_file ("test19-wired-static-verify-ip4-plen");
 
-	block = ifparser_getfirst (parser);
-	connection = nm_simple_connection_new();
-	g_assert (connection);
-	ifupdown_update_connection_from_if_block (connection, block, &error);
-	g_assert_no_error (error);
-
-	success = nm_connection_verify (connection, &error);
-	g_assert_no_error (error);
-	g_assert (success);
+	connection = _connection_first_from_parser (parser);
 
 	/* ===== IPv4 SETTING ===== */
 	s_ip4 = nm_connection_get_setting_ip4_config (connection);
@@ -588,8 +573,6 @@ test19_read_static_ipv4_plen (void)
 	g_assert (ip4_addr != NULL);
 	g_assert_cmpstr (nm_ip_address_get_address (ip4_addr), ==, "10.0.0.3");
 	g_assert_cmpint (nm_ip_address_get_prefix (ip4_addr), ==, 8);
-
-	g_object_unref (connection);
 }
 
 static void
@@ -670,4 +653,3 @@ main (int argc, char **argv)
 
 	return g_test_run ();
 }
-
diff --git a/src/settings/plugins/keyfile/nms-keyfile-connection.c b/src/settings/plugins/keyfile/nms-keyfile-connection.c
deleted file mode 100644
index faf39abbfd..0000000000
--- a/src/settings/plugins/keyfile/nms-keyfile-connection.c
+++ /dev/null
@@ -1,187 +0,0 @@
-/* NetworkManager system settings service - keyfile plugin
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Copyright (C) 2008 Novell, Inc.
- * Copyright (C) 2008 - 2012 Red Hat, Inc.
- */
-
-#include "nm-default.h"
-
-#include "nms-keyfile-connection.h"
-
-#include <glib/gstdio.h>
-
-#include "nm-dbus-interface.h"
-#include "nm-setting-connection.h"
-#include "nm-utils.h"
-
-#include "settings/nm-settings-plugin.h"
-
-#include "nms-keyfile-reader.h"
-#include "nms-keyfile-writer.h"
-#include "nms-keyfile-utils.h"
-
-/*****************************************************************************/
-
-struct _NMSKeyfileConnection {
-	NMSettingsConnection parent;
-};
-
-struct _NMSKeyfileConnectionClass {
-	NMSettingsConnectionClass parent;
-};
-
-G_DEFINE_TYPE (NMSKeyfileConnection, nms_keyfile_connection, NM_TYPE_SETTINGS_CONNECTION)
-
-/*****************************************************************************/
-
-static gboolean
-commit_changes (NMSettingsConnection *connection,
-                NMConnection *new_connection,
-                NMSettingsConnectionCommitReason commit_reason,
-                NMConnection **out_reread_connection,
-                char **out_logmsg_change,
-                GError **error)
-{
-	gs_free char *path = NULL;
-	gs_unref_object NMConnection *reread = NULL;
-	gboolean reread_same = FALSE;
-
-	nm_assert (out_reread_connection && !*out_reread_connection);
-	nm_assert (!out_logmsg_change || !*out_logmsg_change);
-
-	if (!nms_keyfile_writer_connection (new_connection,
-	                                    TRUE,
-	                                    nm_settings_connection_get_filename (connection),
-	                                    NM_FLAGS_ALL (commit_reason,   NM_SETTINGS_CONNECTION_COMMIT_REASON_USER_ACTION
-	                                                                 | NM_SETTINGS_CONNECTION_COMMIT_REASON_ID_CHANGED),
-	                                    NULL,
-	                                    NULL,
-	                                    &path,
-	                                    &reread,
-	                                    &reread_same,
-	                                    error))
-		return FALSE;
-
-	if (!nm_streq0 (path, nm_settings_connection_get_filename (connection))) {
-		gs_free char *old_path = g_strdup (nm_settings_connection_get_filename (connection));
-
-		nm_settings_connection_set_filename (connection, path);
-		if (old_path) {
-			NM_SET_OUT (out_logmsg_change,
-			            g_strdup_printf ("keyfile: update "NMS_KEYFILE_CONNECTION_LOG_FMT" and rename from \"%s\"",
-			                             NMS_KEYFILE_CONNECTION_LOG_ARG (connection),
-			                             old_path));
-		} else {
-			NM_SET_OUT (out_logmsg_change,
-			            g_strdup_printf ("keyfile: update "NMS_KEYFILE_CONNECTION_LOG_FMT" and persist connection",
-			                             NMS_KEYFILE_CONNECTION_LOG_ARG (connection)));
-		}
-	} else {
-		NM_SET_OUT (out_logmsg_change,
-		            g_strdup_printf ("keyfile: update "NMS_KEYFILE_CONNECTION_LOG_FMT,
-		                             NMS_KEYFILE_CONNECTION_LOG_ARG (connection)));
-	}
-
-	if (reread && !reread_same)
-		*out_reread_connection = g_steal_pointer (&reread);
-
-	return TRUE;
-}
-
-static gboolean
-delete (NMSettingsConnection *connection,
-        GError **error)
-{
-	const char *path;
-
-	path = nm_settings_connection_get_filename (connection);
-	if (path)
-		g_unlink (path);
-	return TRUE;
-}
-
-/*****************************************************************************/
-
-static void
-nms_keyfile_connection_init (NMSKeyfileConnection *connection)
-{
-}
-
-NMSKeyfileConnection *
-nms_keyfile_connection_new (NMConnection *source,
-                            const char *full_path,
-                            const char *profile_dir,
-                            GError **error)
-{
-	GObject *object;
-	NMConnection *tmp;
-	const char *uuid;
-	gboolean update_unsaved = TRUE;
-
-	nm_assert (source || full_path);
-	nm_assert (!full_path || full_path[0] == '/');
-	nm_assert (!profile_dir || profile_dir[0] == '/');
-
-	/* If we're given a connection already, prefer that instead of re-reading */
-	if (source)
-		tmp = g_object_ref (source);
-	else {
-		tmp = nms_keyfile_reader_from_file (full_path, profile_dir, error);
-		if (!tmp)
-			return NULL;
-
-		uuid = nm_connection_get_uuid (tmp);
-		if (!uuid) {
-			g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_INVALID_CONNECTION,
-			             "Connection in file %s had no UUID", full_path);
-			g_object_unref (tmp);
-			return NULL;
-		}
-
-		/* If we just read the connection from disk, it's clearly not Unsaved */
-		update_unsaved = FALSE;
-	}
-
-	object = g_object_new (NMS_TYPE_KEYFILE_CONNECTION,
-	                       NM_SETTINGS_CONNECTION_FILENAME, full_path,
-	                       NULL);
-
-	/* Update our settings with what was read from the file */
-	if (!nm_settings_connection_update (NM_SETTINGS_CONNECTION (object),
-	                                    tmp,
-	                                    update_unsaved
-	                                      ? NM_SETTINGS_CONNECTION_PERSIST_MODE_UNSAVED
-	                                      : NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP_SAVED,
-	                                    NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-	                                    NULL,
-	                                    error)) {
-		g_object_unref (object);
-		object = NULL;
-	}
-
-	g_object_unref (tmp);
-	return (NMSKeyfileConnection *) object;
-}
-
-static void
-nms_keyfile_connection_class_init (NMSKeyfileConnectionClass *keyfile_connection_class)
-{
-	NMSettingsConnectionClass *settings_class = NM_SETTINGS_CONNECTION_CLASS (keyfile_connection_class);
-
-	settings_class->commit_changes = commit_changes;
-	settings_class->delete = delete;
-}
diff --git a/src/settings/plugins/keyfile/nms-keyfile-connection.h b/src/settings/plugins/keyfile/nms-keyfile-connection.h
deleted file mode 100644
index 2e7a59d910..0000000000
--- a/src/settings/plugins/keyfile/nms-keyfile-connection.h
+++ /dev/null
@@ -1,43 +0,0 @@
-/* NetworkManager system settings service - keyfile plugin
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Copyright (C) 2008 Novell, Inc.
- * Copyright (C) 2008 - 2012 Red Hat, Inc.
- */
-
-#ifndef __NMS_KEYFILE_CONNECTION_H__
-#define __NMS_KEYFILE_CONNECTION_H__
-
-#include "settings/nm-settings-connection.h"
-
-#define NMS_TYPE_KEYFILE_CONNECTION            (nms_keyfile_connection_get_type ())
-#define NMS_KEYFILE_CONNECTION(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NMS_TYPE_KEYFILE_CONNECTION, NMSKeyfileConnection))
-#define NMS_KEYFILE_CONNECTION_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NMS_TYPE_KEYFILE_CONNECTION, NMSKeyfileConnectionClass))
-#define NMS_IS_KEYFILE_CONNECTION(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NMS_TYPE_KEYFILE_CONNECTION))
-#define NMS_IS_KEYFILE_CONNECTION_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NMS_TYPE_KEYFILE_CONNECTION))
-#define NMS_KEYFILE_CONNECTION_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NMS_TYPE_KEYFILE_CONNECTION, NMSKeyfileConnectionClass))
-
-typedef struct _NMSKeyfileConnection NMSKeyfileConnection;
-typedef struct _NMSKeyfileConnectionClass NMSKeyfileConnectionClass;
-
-GType nms_keyfile_connection_get_type (void);
-
-NMSKeyfileConnection *nms_keyfile_connection_new (NMConnection *source,
-                                                  const char *full_path,
-                                                  const char *profile_dir,
-                                                  GError **error);
-
-#endif /* __NMS_KEYFILE_CONNECTION_H__ */
diff --git a/src/settings/plugins/keyfile/nms-keyfile-plugin.c b/src/settings/plugins/keyfile/nms-keyfile-plugin.c
index 46432fd257..d2e99ad012 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-plugin.c
+++ b/src/settings/plugins/keyfile/nms-keyfile-plugin.c
@@ -15,7 +15,7 @@
  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  *
  * Copyright (C) 2008 Novell, Inc.
- * Copyright (C) 2008 - 2013 Red Hat, Inc.
+ * Copyright (C) 2008 - 2018 Red Hat, Inc.
  */
 
 #include "nm-default.h"
@@ -25,7 +25,11 @@
 #include <sys/stat.h>
 #include <unistd.h>
 #include <sys/types.h>
-#include <glib/gstdio.h>
+#include <sys/time.h>
+
+#include "nm-std-aux/c-list-util.h"
+#include "nm-glib-aux/nm-c-list.h"
+#include "nm-glib-aux/nm-io-utils.h"
 
 #include "nm-connection.h"
 #include "nm-setting.h"
@@ -35,20 +39,42 @@
 #include "nm-core-internal.h"
 #include "nm-keyfile-internal.h"
 
+#include "systemd/nm-sd-utils-shared.h"
+
 #include "settings/nm-settings-plugin.h"
+#include "settings/nm-settings-storage.h"
+#include "settings/nm-settings-utils.h"
 
-#include "nms-keyfile-connection.h"
+#include "nms-keyfile-storage.h"
 #include "nms-keyfile-writer.h"
+#include "nms-keyfile-reader.h"
 #include "nms-keyfile-utils.h"
 
 /*****************************************************************************/
 
 typedef struct {
-	GHashTable *connections;  /* uuid::connection */
-
-	gboolean initialized;
 
 	NMConfig *config;
+
+	/* there can/could be multiple read-only directories. For example, one
+	 * could set dirname_libs to
+	 *   - /usr/lib/NetworkManager/profiles/
+	 *   - /etc/NetworkManager/system-connections
+	 * and leave dirname_etc unset. In this case, there would be multiple
+	 * read-only directories.
+	 *
+	 * Directories that come later have higher priority and shadow profiles
+	 * from earlier directories.
+	 *
+	 * Currently, this is only an array with zero or one elements. It could be
+	 * easily extended to support multiple read-only directories.
+	 */
+	char *dirname_libs[2];
+	char *dirname_etc;
+	char *dirname_run;
+
+	NMSettUtilStorages storages;
+
 } NMSKeyfilePluginPrivate;
 
 struct _NMSKeyfilePlugin {
@@ -62,7 +88,7 @@ struct _NMSKeyfilePluginClass {
 
 G_DEFINE_TYPE (NMSKeyfilePlugin, nms_keyfile_plugin, NM_TYPE_SETTINGS_PLUGIN)
 
-#define NMS_KEYFILE_PLUGIN_GET_PRIVATE(self) _NM_GET_PRIVATE (self, NMSKeyfilePlugin, NMS_IS_KEYFILE_PLUGIN)
+#define NMS_KEYFILE_PLUGIN_GET_PRIVATE(self) _NM_GET_PRIVATE (self, NMSKeyfilePlugin, NMS_IS_KEYFILE_PLUGIN, NMSettingsPlugin)
 
 /*****************************************************************************/
 
@@ -76,430 +102,1047 @@ G_DEFINE_TYPE (NMSKeyfilePlugin, nms_keyfile_plugin, NM_TYPE_SETTINGS_PLUGIN)
 
 /*****************************************************************************/
 
-static void
-connection_removed_cb (NMSettingsConnection *sett_conn, NMSKeyfilePlugin *self)
+static const char *
+_extra_flags_to_string (char *str, gsize str_len, gboolean is_nm_generated, gboolean is_volatile)
 {
-	g_hash_table_remove (NMS_KEYFILE_PLUGIN_GET_PRIVATE (self)->connections,
-	                     nm_settings_connection_get_uuid (sett_conn));
+	const char *str0 = str;
+
+	if (   !is_nm_generated
+	    && !is_volatile)
+		nm_utils_strbuf_append_str (&str, &str_len, "");
+	else {
+		nm_utils_strbuf_append_str (&str, &str_len, " (");
+		if (is_nm_generated) {
+			nm_utils_strbuf_append_str (&str, &str_len, "nm-generated");
+			if (is_volatile)
+				nm_utils_strbuf_append_c (&str, &str_len, ',');
+		}
+		if (is_volatile)
+			nm_utils_strbuf_append_str (&str, &str_len, "volatile");
+		nm_utils_strbuf_append_c (&str, &str_len, ')');
+	}
+
+	return str0;
 }
 
-/* Monitoring */
+static gboolean
+_ignore_filename (NMSKeyfileStorageType storage_type,
+                  const char *filename)
+{
+	/* for backward-compatibility, we don't require an extension for
+	 * files under "/etc/...". */
+	return nm_keyfile_utils_ignore_filename (filename,
+	                                         (storage_type != NMS_KEYFILE_STORAGE_TYPE_ETC));
+}
 
-static void
-remove_connection (NMSKeyfilePlugin *self, NMSKeyfileConnection *connection)
+static const char *
+_get_plugin_dir (NMSKeyfilePluginPrivate *priv)
 {
-	gboolean removed;
+	/* the plugin dir is only needed to generate connection.uuid value via
+	 * nm_keyfile_read_ensure_uuid(). This is either the configured /etc
+	 * directory, of the compile-time default (in case the /etc directory
+	 * is disabled). */
+	return priv->dirname_etc ?: NM_KEYFILE_PATH_NAME_ETC_DEFAULT;
+}
 
-	g_return_if_fail (connection != NULL);
+static gboolean
+_path_detect_storage_type (const char *full_filename,
+                           const char *const*dirname_libs,
+                           const char *dirname_etc,
+                           const char *dirname_run,
+                           NMSKeyfileStorageType *out_storage_type,
+                           const char **out_dirname,
+                           const char **out_filename,
+                           gboolean *out_is_nmmeta_file,
+                           gboolean *out_failed_due_to_invalid_filename)
+{
+	NMSKeyfileStorageType storage_type;
+	const char *filename = NULL;
+	const char *dirname = NULL;
+	guint i;
+	gboolean is_nmmeta_file = FALSE;
 
-	_LOGI ("removed " NMS_KEYFILE_CONNECTION_LOG_FMT, NMS_KEYFILE_CONNECTION_LOG_ARG (connection));
+	NM_SET_OUT (out_failed_due_to_invalid_filename, FALSE);
 
-	/* Removing from the hash table should drop the last reference */
-	g_object_ref (connection);
-	g_signal_handlers_disconnect_by_func (connection, connection_removed_cb, self);
-	removed = g_hash_table_remove (NMS_KEYFILE_PLUGIN_GET_PRIVATE (self)->connections,
-	                               nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (connection)));
-	nm_settings_connection_signal_remove (NM_SETTINGS_CONNECTION (connection));
-	g_object_unref (connection);
+	if (full_filename[0] != '/')
+		return FALSE;
 
-	g_return_if_fail (removed);
-}
+	if (   dirname_run
+	    && (filename = nm_utils_file_is_in_path (full_filename, dirname_run))) {
+		storage_type = NMS_KEYFILE_STORAGE_TYPE_RUN;
+		dirname = dirname_run;
+	} else if (   dirname_etc
+	           && (filename = nm_utils_file_is_in_path (full_filename, dirname_etc))) {
+		storage_type = NMS_KEYFILE_STORAGE_TYPE_ETC;
+		dirname = dirname_etc;
+	} else {
+		for (i = 0; dirname_libs && dirname_libs[i]; i++) {
+			if ((filename = nm_utils_file_is_in_path (full_filename, dirname_libs[i]))) {
+				storage_type = NMS_KEYFILE_STORAGE_TYPE_LIB (i);
+				dirname = dirname_libs[i];
+				break;
+			}
+		}
+		if (!dirname)
+			return FALSE;
+	}
 
-static NMSKeyfileConnection *
-find_by_path (NMSKeyfilePlugin *self, const char *path)
-{
-	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
-	GHashTableIter iter;
-	NMSettingsConnection *candidate = NULL;
+	if (_ignore_filename (storage_type, filename)) {
 
-	g_return_val_if_fail (path != NULL, NULL);
+		/* we accept nmmeta files, but only in /etc and /run directories. */
+
+		if (   !NM_IN_SET (storage_type, NMS_KEYFILE_STORAGE_TYPE_RUN,
+		                                 NMS_KEYFILE_STORAGE_TYPE_ETC)
+		    || !nms_keyfile_nmmeta_check_filename (filename, NULL)) {
+			NM_SET_OUT (out_failed_due_to_invalid_filename, TRUE);
+			return FALSE;
+		}
 
-	g_hash_table_iter_init (&iter, priv->connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer) &candidate)) {
-		if (g_strcmp0 (path, nm_settings_connection_get_filename (candidate)) == 0)
-			return NMS_KEYFILE_CONNECTION (candidate);
+		is_nmmeta_file = TRUE;
 	}
-	return NULL;
+
+	NM_SET_OUT (out_storage_type, storage_type);
+	NM_SET_OUT (out_dirname, dirname);
+	NM_SET_OUT (out_filename, filename);
+	NM_SET_OUT (out_is_nmmeta_file, is_nmmeta_file);
+	return TRUE;
 }
 
-/* update_connection:
- * @self: the plugin instance
- * @source: if %NULL, this re-reads the connection from @full_path
- *   and updates it. When passing @source, this adds a connection from
- *   memory.
- * @full_path: the filename of the keyfile to be loaded
- * @connection: an existing connection that might be updated.
- *   If given, @connection must be an existing connection that is currently
- *   owned by the plugin.
- * @protect_existing_connection: if %TRUE, and !@connection, we don't allow updating
- *   an existing connection with the same UUID.
- *   If %TRUE and @connection, allow updating only if the reload would modify
- *   @connection (without changing its UUID) or if we would create a new connection.
- *   In other words, if this parameter is %TRUE, we only allow creating a
- *   new connection (with an unseen UUID) or updating the passed in @connection
- *   (whereas the UUID cannot change).
- *   Note, that this allows for @connection to be replaced by a new connection.
- * @protected_connections: (allow-none): if given, we only update an
- *   existing connection if it is not contained in this hash.
- * @error: error in case of failure
- *
- * Loads a connection from file @full_path. This can both be used to
- * load a connection initially or to update an existing connection.
- *
- * If you pass in an existing connection and the reloaded file happens
- * to have a different UUID, the connection is deleted.
- * Beware, that means that after the function, you have a dangling pointer
- * if the returned connection is different from @connection.
- *
- * Returns: the updated connection.
- * */
-static NMSKeyfileConnection *
-update_connection (NMSKeyfilePlugin *self,
-                   NMConnection *source,
-                   const char *full_path,
-                   NMSKeyfileConnection *connection,
-                   gboolean protect_existing_connection,
-                   GHashTable *protected_connections,
-                   GError **error)
+/*****************************************************************************/
+
+static NMConnection *
+_read_from_file (const char *full_filename,
+                 const char *plugin_dir,
+                 struct stat *out_stat,
+                 NMTernary *out_is_nm_generated,
+                 NMTernary *out_is_volatile,
+                 GError **error)
 {
-	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
-	NMSKeyfileConnection *connection_new;
-	NMSKeyfileConnection *connection_by_uuid;
-	GError *local = NULL;
-	const char *uuid;
+	NMConnection *connection;
 
-	g_return_val_if_fail (!source || NM_IS_CONNECTION (source), NULL);
-	g_return_val_if_fail (full_path || source, NULL);
+	nm_assert (full_filename && full_filename[0] == '/');
 
-	if (full_path)
-		_LOGD ("loading from file \"%s\"...", full_path);
+	connection = nms_keyfile_reader_from_file (full_filename, plugin_dir, out_stat, out_is_nm_generated, out_is_volatile, error);
 
-	if (   !nm_utils_file_is_in_path (full_path, nms_keyfile_utils_get_path ())
-	    && !nm_utils_file_is_in_path (full_path, NM_KEYFILE_PATH_NAME_RUN)) {
-		g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-		                     "File not in recognized system-connections directory");
-		return FALSE;
-	}
+	nm_assert (!connection || (_nm_connection_verify (connection, NULL) == NM_SETTING_VERIFY_SUCCESS));
+	nm_assert (!connection || nm_utils_is_uuid (nm_connection_get_uuid (connection)));
 
-	connection_new = nms_keyfile_connection_new (source, full_path, nms_keyfile_utils_get_path (), &local);
-	if (!connection_new) {
-		/* Error; remove the connection */
-		if (source)
-			_LOGW ("error creating connection %s: %s", nm_connection_get_uuid (source), local->message);
-		else
-			_LOGW ("error loading connection from file %s: %s", full_path, local->message);
-		if (   connection
-		    && !protect_existing_connection
-		    && (!protected_connections || !g_hash_table_contains (protected_connections, connection)))
-			remove_connection (self, connection);
-		g_propagate_error (error, local);
-		return NULL;
+	return connection;
+}
+
+/*****************************************************************************/
+
+static void
+_nm_assert_storage (gpointer plugin  /* NMSKeyfilePlugin  */,
+                    gpointer storage /* NMSKeyfileStorage */,
+                    gboolean tracked)
+{
+#if NM_MORE_ASSERTS
+	NMSettUtilStorageByUuidHead *sbuh;
+	const char *uuid;
+
+	nm_assert (!plugin || NMS_IS_KEYFILE_PLUGIN (plugin));
+	nm_assert (NMS_IS_KEYFILE_STORAGE (storage));
+	nm_assert (!plugin || plugin == nm_settings_storage_get_plugin (storage));
+	nm_assert (!((NMSKeyfileStorage *) storage)->is_tombstone || !(((NMSKeyfileStorage *) storage)->connection));
+	nm_assert (({
+	                const char *f = nms_keyfile_storage_get_filename (storage);
+	                f && f[0] == '/';
+	            }));
+
+	uuid = nms_keyfile_storage_get_uuid (storage);
+
+	nm_assert (nm_utils_is_uuid (uuid));
+
+	nm_assert (   !tracked
+	           || !plugin
+	           || c_list_contains (&NMS_KEYFILE_PLUGIN_GET_PRIVATE (plugin)->storages._storage_lst_head,
+	                               &NMS_KEYFILE_STORAGE (storage)->parent._storage_lst));
+
+	nm_assert (   !tracked
+	           || !plugin
+	           || storage == g_hash_table_lookup (NMS_KEYFILE_PLUGIN_GET_PRIVATE (plugin)->storages.idx_by_filename,
+	                                              nms_keyfile_storage_get_filename (storage)));
+
+	if (tracked && plugin) {
+		sbuh = g_hash_table_lookup (NMS_KEYFILE_PLUGIN_GET_PRIVATE (plugin)->storages.idx_by_uuid, &uuid);
+		nm_assert (sbuh);
+		nm_assert (c_list_contains (&sbuh->_storage_by_uuid_lst_head, &((NMSKeyfileStorage *) storage)->parent._storage_by_uuid_lst));
 	}
+#endif
+}
 
-	uuid = nm_settings_connection_get_uuid (NM_SETTINGS_CONNECTION (connection_new));
-	connection_by_uuid = g_hash_table_lookup (priv->connections, uuid);
+/*****************************************************************************/
 
-	if (   connection
-	    && connection != connection_by_uuid) {
+static NMSKeyfileStorage *
+_load_file (NMSKeyfilePlugin *self,
+            const char *dirname,
+            const char *filename,
+            NMSKeyfileStorageType storage_type,
+            GError **error)
+{
+	NMSKeyfilePluginPrivate *priv;
+	gs_unref_object NMConnection *connection = NULL;
+	NMTernary is_volatile_opt;
+	NMTernary is_nm_generated_opt;
+	gs_free_error GError *local = NULL;
+	gs_free char *full_filename = NULL;
+	struct stat st;
 
-		if (   (protect_existing_connection && connection_by_uuid != NULL)
-		    || (protected_connections && g_hash_table_contains (protected_connections, connection))) {
-			NMSKeyfileConnection *conflicting = (protect_existing_connection && connection_by_uuid != NULL) ? connection_by_uuid : connection;
+	if (_ignore_filename (storage_type, filename)) {
+		gs_free char *nmmeta = NULL;
+		gs_free char *loaded_path = NULL;
 
-			if (source)
-				_LOGW ("cannot update protected "NMS_KEYFILE_CONNECTION_LOG_FMT" connection due to conflicting UUID %s", NMS_KEYFILE_CONNECTION_LOG_ARG (conflicting), uuid);
+		if (!nms_keyfile_nmmeta_check_filename (filename, NULL)) {
+			if (error)
+				nm_utils_error_set (error, NM_UTILS_ERROR_UNKNOWN, "skip due to invalid filename");
 			else
-				_LOGW ("cannot load %s due to conflicting UUID for "NMS_KEYFILE_CONNECTION_LOG_FMT, full_path, NMS_KEYFILE_CONNECTION_LOG_ARG (conflicting));
-			g_object_unref (connection_new);
-			g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-			                      "Cannot update protected connection due to conflicting UUID");
+				_LOGT ("load: \"%s/%s\": skip file due to invalid filename", dirname, filename);
+			return NULL;
+		}
+		if (!nms_keyfile_nmmeta_read (dirname,
+		                              filename,
+		                              &full_filename,
+		                              &nmmeta,
+		                              &loaded_path,
+		                              NULL)) {
+			if (error)
+				nm_utils_error_set (error, NM_UTILS_ERROR_UNKNOWN, "skip unreadable nmmeta file");
+			else
+				_LOGT ("load: \"%s/%s\": skip unreadable nmmeta file", dirname, filename);
+			return NULL;
+		}
+		nm_assert (loaded_path);
+		if (!NM_IN_SET (storage_type, NMS_KEYFILE_STORAGE_TYPE_RUN,
+		                              NMS_KEYFILE_STORAGE_TYPE_ETC)) {
+			if (error)
+				nm_utils_error_set (error, NM_UTILS_ERROR_UNKNOWN, "skip nmmeta file from read-only directory");
+			else
+				_LOGT ("load: \"%s/%s\": skip nmmeta file from read-only directory", dirname, filename);
+			return NULL;
+		}
+		if (!nm_streq (loaded_path, NM_KEYFILE_PATH_NMMETA_SYMLINK_NULL)) {
+			if (error)
+				nm_utils_error_set (error, NM_UTILS_ERROR_UNKNOWN, "skip nmmeta file not symlinking %s", NM_KEYFILE_PATH_NMMETA_SYMLINK_NULL);
+			else
+				_LOGT ("load: \"%s/%s\": skip nmmeta file not symlinking to %s", dirname, filename, NM_KEYFILE_PATH_NMMETA_SYMLINK_NULL);
 			return NULL;
 		}
 
-		/* The new connection has a different UUID then the original one.
-		 * Remove @connection. */
-		remove_connection (self, connection);
+		return nms_keyfile_storage_new_tombstone (self,
+		                                          nmmeta,
+		                                          full_filename,
+		                                          storage_type);
 	}
 
-	if (   connection_by_uuid
-	    && (   (!connection && protect_existing_connection)
-	        || (protected_connections && g_hash_table_contains (protected_connections, connection_by_uuid)))) {
-		if (source)
-			_LOGW ("cannot update connection due to conflicting UUID for "NMS_KEYFILE_CONNECTION_LOG_FMT, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_by_uuid));
+	full_filename = g_build_filename (dirname, filename, NULL);
+
+	priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+
+	connection = _read_from_file (full_filename,
+	                              _get_plugin_dir (priv),
+	                              &st,
+	                              &is_nm_generated_opt,
+	                              &is_volatile_opt,
+	                              &local);
+	if (!connection) {
+		if (error)
+			g_propagate_error (error, local);
 		else
-			_LOGW ("cannot load %s due to conflicting UUID for "NMS_KEYFILE_CONNECTION_LOG_FMT, full_path, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_by_uuid));
-		g_object_unref (connection_new);
-		g_set_error_literal (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
-		                      "Skip updating protected connection during reload");
+			_LOGW ("load: \"%s\": failed to load connection: %s", full_filename, local->message);
 		return NULL;
 	}
 
-	if (connection_by_uuid) {
-		const char *old_path;
-
-		old_path = nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection_by_uuid));
-
-		if (nm_connection_compare (nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection_by_uuid)),
-		                           nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection_new)),
-		                           NM_SETTING_COMPARE_FLAG_IGNORE_AGENT_OWNED_SECRETS |
-		                           NM_SETTING_COMPARE_FLAG_IGNORE_NOT_SAVED_SECRETS)) {
-			/* Nothing to do... except updating the path. */
-			if (old_path && g_strcmp0 (old_path, full_path) != 0)
-				_LOGI ("rename \"%s\" to "NMS_KEYFILE_CONNECTION_LOG_FMT" without other changes", old_path, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new));
-		} else {
-			/* An existing connection changed. */
-			if (source)
-				_LOGI ("update "NMS_KEYFILE_CONNECTION_LOG_FMT" from %s", NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new), NMS_KEYFILE_CONNECTION_LOG_PATH (old_path));
-			else if (!g_strcmp0 (old_path, nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection_new))))
-				_LOGI ("update "NMS_KEYFILE_CONNECTION_LOG_FMT, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new));
-			else if (old_path)
-				_LOGI ("rename \"%s\" to "NMS_KEYFILE_CONNECTION_LOG_FMT, old_path, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new));
-			else
-				_LOGI ("update and persist "NMS_KEYFILE_CONNECTION_LOG_FMT, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new));
-
-			if (!nm_settings_connection_update (NM_SETTINGS_CONNECTION (connection_by_uuid),
-			                                    nm_settings_connection_get_connection (NM_SETTINGS_CONNECTION (connection_new)),
-			                                    NM_SETTINGS_CONNECTION_PERSIST_MODE_KEEP_SAVED,
-			                                    NM_SETTINGS_CONNECTION_COMMIT_REASON_NONE,
-			                                    "keyfile-update",
-			                                    &local)) {
-				/* Shouldn't ever get here as 'connection_new' was verified by the reader already
-				 * and the UUID did not change. */
-				g_assert_not_reached ();
-			}
-			g_assert_no_error (local);
-		}
-		nm_settings_connection_set_filename (NM_SETTINGS_CONNECTION (connection_by_uuid), full_path);
-		g_object_unref (connection_new);
-		return connection_by_uuid;
-	} else {
-		if (source)
-			_LOGI ("add connection "NMS_KEYFILE_CONNECTION_LOG_FMT, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new));
-		else
-			_LOGI ("new connection "NMS_KEYFILE_CONNECTION_LOG_FMT, NMS_KEYFILE_CONNECTION_LOG_ARG (connection_new));
-		g_hash_table_insert (priv->connections, g_strdup (uuid), connection_new);
-
-		g_signal_connect (connection_new, NM_SETTINGS_CONNECTION_REMOVED,
-		                  G_CALLBACK (connection_removed_cb),
-		                  self);
-
-		if (!source) {
-			/* Only raise the signal if we were called without source, i.e. if we read the connection from file.
-			 * Otherwise, we were called by add_connection() which does not expect the signal. */
-			_nm_settings_plugin_emit_signal_connection_added (NM_SETTINGS_PLUGIN (self),
-			                                                  NM_SETTINGS_CONNECTION (connection_new));
-		}
+	return nms_keyfile_storage_new_connection (self,
+	                                           g_steal_pointer (&connection),
+	                                           full_filename,
+	                                           storage_type,
+	                                           is_nm_generated_opt,
+	                                           is_volatile_opt,
+	                                           &st.st_mtim);
+}
 
-		return connection_new;
-	}
+static NMSKeyfileStorage *
+_load_file_from_path (NMSKeyfilePlugin *self,
+                      const char *full_filename,
+                      NMSKeyfileStorageType storage_type,
+                      GError **error)
+{
+	gs_free char *f_dirname_free = NULL;
+	const char *f_filename;
+	const char *f_dirname;
+
+	nm_assert (full_filename && full_filename[0] == '/');
+
+	f_filename = strrchr (full_filename, '/');
+	f_dirname = nm_strndup_a (300, full_filename, f_filename - full_filename, &f_dirname_free);
+	f_filename++;
+	return _load_file (self,
+	                   f_dirname,
+	                   f_filename,
+	                   storage_type,
+	                   error);
 }
 
 static void
-config_changed_cb (NMConfig *config,
-                   NMConfigData *config_data,
-                   NMConfigChangeFlags changes,
-                   NMConfigData *old_data,
-                   NMSKeyfilePlugin *self)
+_load_dir (NMSKeyfilePlugin *self,
+           NMSKeyfileStorageType storage_type,
+           const char *dirname,
+           NMSettUtilStorages *storages)
 {
-	gs_free char *old_value = NULL;
-	gs_free char *new_value = NULL;
+	const char *filename;
+	GDir *dir;
+	gs_unref_hashtable GHashTable *dupl_filenames = NULL;
 
-	old_value = nm_config_data_get_value (old_data, NM_CONFIG_KEYFILE_GROUP_KEYFILE, NM_CONFIG_KEYFILE_KEY_KEYFILE_UNMANAGED_DEVICES, NM_CONFIG_GET_VALUE_TYPE_SPEC);
-	new_value = nm_config_data_get_value (config_data, NM_CONFIG_KEYFILE_GROUP_KEYFILE, NM_CONFIG_KEYFILE_KEY_KEYFILE_UNMANAGED_DEVICES, NM_CONFIG_GET_VALUE_TYPE_SPEC);
+	dir = g_dir_open (dirname, 0, NULL);
+	if (!dir)
+		return;
 
-	if (!nm_streq0 (old_value, new_value))
-		_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
-}
+	dupl_filenames = g_hash_table_new_full (nm_str_hash, g_str_equal, NULL, g_free);
 
-static GHashTable *
-_paths_from_connections (GHashTable *connections)
-{
-	GHashTableIter iter;
-	NMSKeyfileConnection *connection;
-	GHashTable *paths = g_hash_table_new (nm_str_hash, g_str_equal);
+	while ((filename = g_dir_read_name (dir))) {
+		gs_unref_object NMSKeyfileStorage *storage = NULL;
 
-	g_hash_table_iter_init (&iter, connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer *) &connection)) {
-		const char *path = nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection));
+		filename = g_strdup (filename);
+		if (!g_hash_table_add (dupl_filenames, (char *) filename))
+			continue;
 
-		if (path)
-			g_hash_table_add (paths, (void *) path);
-	}
-	return paths;
-}
+		storage = _load_file (self,
+		                      dirname,
+		                      filename,
+		                      storage_type,
+		                      NULL);
+		if (!storage)
+			continue;
 
-static int
-_sort_paths (const char **f1, const char **f2, GHashTable *paths)
-{
-	struct stat st;
-	gboolean c1, c2;
-	gint64 m1, m2;
+		nm_sett_util_storages_add_take (storages, g_steal_pointer (&storage));
+	}
 
-	c1 = !!g_hash_table_contains (paths, *f1);
-	c2 = !!g_hash_table_contains (paths, *f2);
-	if (c1 != c2)
-		return c1 ? -1 : 1;
+	g_dir_close (dir);
 
-	m1 = stat (*f1, &st) == 0 ? (gint64) st.st_mtime : G_MININT64;
-	m2 = stat (*f2, &st) == 0 ? (gint64) st.st_mtime : G_MININT64;
-	if (m1 != m2)
-		return m1 > m2 ? -1 : 1;
+#if NM_MORE_ASSERTS
+	{
+		NMSKeyfileStorage *storage;
 
-	return strcmp (*f1, *f2);
+		c_list_for_each_entry (storage, &storages->_storage_lst_head, parent._storage_lst)
+			nm_assert (NMS_IS_KEYFILE_STORAGE (storage));
+	}
+#endif
 }
 
+/*****************************************************************************/
+
 static void
-_read_dir (GPtrArray *filenames,
-           const char *path,
-           gboolean require_extension)
+_storages_consolidate (NMSKeyfilePlugin *self,
+                       NMSettUtilStorages *storages_new,
+                       gboolean replace_all,
+                       GHashTable *storages_replaced,
+                       NMSettingsPluginConnectionLoadCallback callback,
+                       gpointer user_data)
 {
-	GDir *dir;
-	const char *item;
-	GError *error = NULL;
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+	CList lst_conn_info_deleted = C_LIST_INIT (lst_conn_info_deleted);
+	gs_unref_ptrarray GPtrArray *storages_modified = NULL;
+	CList storages_deleted;
+	NMSKeyfileStorage *storage_safe;
+	NMSKeyfileStorage *storage_new;
+	NMSKeyfileStorage *storage_old;
+	NMSKeyfileStorage *storage;
+	guint i;
 
-	dir = g_dir_open (path, 0, &error);
-	if (!dir) {
-		_LOGD ("cannot read directory '%s': %s", path, error->message);
-		g_clear_error (&error);
-		return;
+	storages_modified = g_ptr_array_new_with_free_func (g_object_unref);
+	c_list_init (&storages_deleted);
+
+	c_list_for_each_entry (storage_old, &priv->storages._storage_lst_head, parent._storage_lst)
+		storage_old->dirty = TRUE;
+
+	c_list_for_each_entry_safe (storage_new, storage_safe, &storages_new->_storage_lst_head, parent._storage_lst) {
+		storage_old = nm_sett_util_storages_lookup_by_filename (&priv->storages, nms_keyfile_storage_get_filename (storage_new));
+
+		nm_sett_util_storages_steal (storages_new, storage_new);
+
+		if (   !storage_old
+		    || !nm_streq (nms_keyfile_storage_get_uuid (storage_new), nms_keyfile_storage_get_uuid (storage_old))) {
+			if (storage_old) {
+				nm_sett_util_storages_steal (&priv->storages, storage_old);
+				c_list_link_tail (&storages_deleted, &storage_old->parent._storage_lst);
+			}
+			storage_new->dirty = FALSE;
+			nm_sett_util_storages_add_take (&priv->storages, storage_new);
+			g_ptr_array_add (storages_modified, g_object_ref (storage_new));
+			continue;
+		}
+
+		storage_old->dirty = FALSE;
+		nms_keyfile_storage_copy_content (storage_old, storage_new);
+		nms_keyfile_storage_destroy (storage_new);
+		g_ptr_array_add (storages_modified, g_object_ref (storage_old));
 	}
 
-	while ((item = g_dir_read_name (dir))) {
-		if (nm_keyfile_utils_ignore_filename (item, require_extension))
+	c_list_for_each_entry_safe (storage_old, storage_safe, &priv->storages._storage_lst_head, parent._storage_lst) {
+		if (!storage_old->dirty)
 			continue;
-		g_ptr_array_add (filenames, g_build_filename (path, item, NULL));
+		if (   replace_all
+		    || (   storages_replaced
+		        && g_hash_table_contains (storages_replaced, storage_old))) {
+			nm_sett_util_storages_steal (&priv->storages, storage_old);
+			c_list_link_tail (&storages_deleted, &storage_old->parent._storage_lst);
+		}
+	}
+
+	/* raise events. */
+
+	for (i = 0; i < storages_modified->len; i++) {
+		storage = storages_modified->pdata[i];
+		storage->dirty = TRUE;
+	}
+
+	for (i = 0; i < storages_modified->len; i++) {
+		gs_unref_object NMConnection *connection = NULL;
+
+		storage = storages_modified->pdata[i];
+
+		if (!storage->dirty) {
+			/* the entry is no longer dirty. In the meantime we already emited
+			 * another signal for it. */
+			continue;
+		}
+		storage->dirty = FALSE;
+		if (storage != nm_sett_util_storages_lookup_by_filename (&priv->storages, nms_keyfile_storage_get_filename (storage))) {
+			/* hm? The profile was deleted in the meantime? That is only possible
+			 * if the signal handler called again into the plugin. In any case, the event
+			 * was already emitted. Skip. */
+			continue;
+		}
+
+		connection = nms_keyfile_storage_steal_connection (storage);
+		nm_assert (NM_IS_CONNECTION (connection));
+
+		callback (NM_SETTINGS_PLUGIN (self),
+		          NM_SETTINGS_STORAGE (storage),
+		          connection,
+		          user_data);
+	}
+
+	while ((storage = c_list_first_entry (&storages_deleted, NMSKeyfileStorage, parent._storage_lst))) {
+		c_list_unlink (&storage->parent._storage_lst);
+		storage->is_tombstone = FALSE;
+		callback (NM_SETTINGS_PLUGIN (self),
+		          NM_SETTINGS_STORAGE (storage),
+		          NULL,
+		          user_data);
+		nms_keyfile_storage_destroy (storage);
 	}
-	g_dir_close (dir);
 }
 
+static void
+reload_connections (NMSettingsPlugin *plugin,
+                    NMSettingsPluginConnectionLoadCallback callback,
+                    gpointer user_data)
+{
+	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN (plugin);
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+	nm_auto_clear_sett_util_storages NMSettUtilStorages storages_new = NM_SETT_UTIL_STORAGES_INIT (storages_new, nms_keyfile_storage_destroy);
+	int i;
+
+	_load_dir (self, NMS_KEYFILE_STORAGE_TYPE_RUN, priv->dirname_run, &storages_new);
+	if (priv->dirname_etc)
+		_load_dir (self, NMS_KEYFILE_STORAGE_TYPE_ETC, priv->dirname_etc, &storages_new);
+	for (i = 0; priv->dirname_libs[i]; i++)
+		_load_dir (self, NMS_KEYFILE_STORAGE_TYPE_LIB (i), priv->dirname_libs[i], &storages_new);
+
+	_storages_consolidate (self,
+	                       &storages_new,
+	                       TRUE,
+	                       NULL,
+	                       callback,
+	                       user_data);
+}
 
 static void
-read_connections (NMSettingsPlugin *config)
+load_connections (NMSettingsPlugin *plugin,
+                  NMSettingsPluginConnectionLoadEntry *entries,
+                  gsize n_entries,
+                  NMSettingsPluginConnectionLoadCallback callback,
+                  gpointer user_data)
 {
-	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN (config);
+	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN (plugin);
 	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
-	GHashTable *alive_connections;
-	GHashTableIter iter;
-	NMSKeyfileConnection *connection;
-	GPtrArray *dead_connections = NULL;
-	guint i;
-	GPtrArray *filenames;
-	GHashTable *paths;
+	nm_auto_clear_sett_util_storages NMSettUtilStorages storages_new = NM_SETT_UTIL_STORAGES_INIT (storages_new, nms_keyfile_storage_destroy);
+	gs_unref_hashtable GHashTable *dupl_filenames = NULL;
+	gs_unref_hashtable GHashTable *storages_replaced = NULL;
+	gs_unref_hashtable GHashTable *loaded_uuids = NULL;
+	const char *loaded_uuid;
+	GHashTableIter h_iter;
+	gsize i;
+
+	if (n_entries == 0)
+		return;
 
-	filenames = g_ptr_array_new_with_free_func (g_free);
+	dupl_filenames = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, NULL);
 
-	_read_dir (filenames, NM_KEYFILE_PATH_NAME_RUN, TRUE);
-	_read_dir (filenames, nms_keyfile_utils_get_path (), FALSE);
+	loaded_uuids = g_hash_table_new (nm_str_hash, g_str_equal);
 
-	alive_connections = g_hash_table_new (nm_direct_hash, NULL);
+	storages_replaced = g_hash_table_new_full (nm_direct_hash, NULL, g_object_unref, NULL);
 
-	/* While reloading, we don't replace connections that we already loaded while
-	 * iterating over the files.
-	 *
-	 * To have sensible, reproducible behavior, sort the paths by last modification
-	 * time preferring older files.
-	 */
-	paths = _paths_from_connections (priv->connections);
-	g_ptr_array_sort_with_data (filenames, (GCompareDataFunc) _sort_paths, paths);
-	g_hash_table_destroy (paths);
-
-	for (i = 0; i < filenames->len; i++) {
-		connection = update_connection (self, NULL, filenames->pdata[i], NULL, FALSE, alive_connections, NULL);
-		if (connection)
-			g_hash_table_add (alive_connections, connection);
-	}
-	g_ptr_array_free (filenames, TRUE);
-
-	g_hash_table_iter_init (&iter, priv->connections);
-	while (g_hash_table_iter_next (&iter, NULL, (gpointer *) &connection)) {
-		if (   !g_hash_table_contains (alive_connections, connection)
-		    && nm_settings_connection_get_filename (NM_SETTINGS_CONNECTION (connection))) {
-			if (!dead_connections)
-				dead_connections = g_ptr_array_new ();
-			g_ptr_array_add (dead_connections, connection);
+	for (i = 0; i < n_entries; i++) {
+		NMSettingsPluginConnectionLoadEntry *const entry = &entries[i];
+		NMSKeyfileStorageType storage_type;
+		gs_free_error GError *local = NULL;
+		const char *f_filename;
+		const char *f_dirname;
+		const char *full_filename;
+		gs_free char *full_filename_keep = NULL;
+		gboolean is_nmmeta_file;
+		NMSettingsPluginConnectionLoadEntry *dupl_content_entry;
+		gboolean failed_due_to_invalid_filename;
+		gs_unref_object NMSKeyfileStorage *storage = NULL;
+
+		if (entry->handled)
+			continue;
+
+		if (!_path_detect_storage_type (entry->filename,
+		                                (const char *const*) priv->dirname_libs,
+		                                priv->dirname_etc,
+		                                priv->dirname_run,
+		                                &storage_type,
+		                                &f_dirname,
+		                                &f_filename,
+		                                &is_nmmeta_file,
+		                                &failed_due_to_invalid_filename)) {
+			if (failed_due_to_invalid_filename) {
+				entry->handled = TRUE;
+				nm_utils_error_set (&entry->error, NM_UTILS_ERROR_UNKNOWN, "filename is not valid for a keyfile");
+			}
+			continue;
 		}
-	}
-	g_hash_table_destroy (alive_connections);
 
-	if (dead_connections) {
-		for (i = 0; i < dead_connections->len; i++)
-			remove_connection (self, dead_connections->pdata[i]);
-		g_ptr_array_free (dead_connections, TRUE);
+		full_filename_keep = g_build_filename (f_dirname, f_filename, NULL);
+
+		if ((dupl_content_entry = g_hash_table_lookup (dupl_filenames, full_filename_keep))) {
+			/* we already visited this file. */
+			entry->handled = dupl_content_entry->handled;
+			if (dupl_content_entry->error) {
+				g_set_error_literal (&entry->error,
+				                     dupl_content_entry->error->domain,
+				                     dupl_content_entry->error->code,
+				                     dupl_content_entry->error->message);
+			}
+			continue;
+		}
+
+		entry->handled = TRUE;
+
+		full_filename = full_filename_keep;
+		if (!g_hash_table_insert (dupl_filenames, g_steal_pointer (&full_filename_keep), entry))
+			nm_assert_not_reached ();
+
+		storage = _load_file (self,
+		                      f_dirname,
+		                      f_filename,
+		                      storage_type,
+		                      &local);
+		if (!storage) {
+			if (nm_utils_file_stat (full_filename, NULL) == -ENOENT) {
+				NMSKeyfileStorage *storage2;
+
+				/* the file does not exist. We take that as indication to unload the file
+				 * that was previously loaded... */
+				storage2 = nm_sett_util_storages_lookup_by_filename (&priv->storages, full_filename);
+				if (storage2)
+					g_hash_table_add (storages_replaced, g_object_ref (storage2));
+				continue;
+			}
+			g_propagate_error (&entry->error, g_steal_pointer (&local));
+			continue;
+		}
+
+		g_hash_table_add (loaded_uuids, (char *) nms_keyfile_storage_get_uuid (storage));
+
+		nm_sett_util_storages_add_take (&storages_new, g_steal_pointer (&storage));
 	}
-}
 
-/*****************************************************************************/
+	/* now we visit all UUIDs that are about to change... */
+	g_hash_table_iter_init (&h_iter, loaded_uuids);
+	while (g_hash_table_iter_next (&h_iter, (gpointer *) &loaded_uuid, NULL)) {
+		NMSKeyfileStorage *storage;
+		NMSettUtilStorageByUuidHead *sbuh;
 
-static GSList *
-get_connections (NMSettingsPlugin *config)
-{
-	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE ((NMSKeyfilePlugin *) config);
+		sbuh = nm_sett_util_storages_lookup_by_uuid (&priv->storages, loaded_uuid);
+		if (!sbuh)
+			continue;
 
-	if (!priv->initialized) {
-		read_connections (config);
-		priv->initialized = TRUE;
+		c_list_for_each_entry (storage, &sbuh->_storage_by_uuid_lst_head, parent._storage_by_uuid_lst) {
+			const char *full_filename = nms_keyfile_storage_get_filename (storage);
+			gs_unref_object NMSKeyfileStorage *storage_new = NULL;
+			gs_free_error GError *local = NULL;
+
+			if (g_hash_table_contains (dupl_filenames, full_filename)) {
+				/* already re-loaded. */
+				continue;
+			}
+
+			/* @storage has a UUID that was just loaded from disk, but we have an entry in cache.
+			 * Reload that file too despite not being told to do so. The reason is to get
+			 * the latest file timestamp so that we get the priorities right. */
+
+			storage_new = _load_file_from_path (self,
+			                                    full_filename,
+			                                    storage->storage_type,
+			                                    &local);
+			if (   storage_new
+			    && !nm_streq (loaded_uuid, nms_keyfile_storage_get_uuid (storage_new))) {
+				/* the file now references a different UUID. We are not told to reload
+				 * that file, so this means the existing storage (with the previous
+				 * filename and UUID tuple) is no longer valid. */
+				g_clear_object (&storage_new);
+			}
+
+			g_hash_table_add (storages_replaced, g_object_ref (storage));
+			if (storage_new)
+				nm_sett_util_storages_add_take (&storages_new, g_steal_pointer (&storage_new));
+		}
 	}
-	return _nm_utils_hash_values_to_slist (priv->connections);
+
+	nm_clear_pointer (&loaded_uuids, g_hash_table_destroy);
+	nm_clear_pointer (&dupl_filenames, g_hash_table_destroy);
+
+	_storages_consolidate (self,
+	                       &storages_new,
+	                       FALSE,
+	                       storages_replaced,
+	                       callback,
+	                       user_data);
 }
 
-static gboolean
-load_connection (NMSettingsPlugin *config,
-                 const char *filename)
+gboolean
+nms_keyfile_plugin_add_connection (NMSKeyfilePlugin *self,
+                                   NMConnection *connection,
+                                   gboolean is_nm_generated,
+                                   gboolean is_volatile,
+                                   gboolean in_memory,
+                                   NMSettingsStorage **out_storage,
+                                   NMConnection **out_connection,
+                                   GError **error)
 {
-	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN ((NMSKeyfilePlugin *) config);
-	NMSKeyfileConnection *connection;
-	gboolean require_extension;
-
-	if (nm_utils_file_is_in_path (filename, nms_keyfile_utils_get_path ()))
-		require_extension = FALSE;
-	else if (nm_utils_file_is_in_path (filename, NM_KEYFILE_PATH_NAME_RUN))
-		require_extension = TRUE;
-	else
-		return FALSE;
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+	gs_unref_object NMConnection *reread = NULL;
+	gs_free char *full_filename = NULL;
+	NMSKeyfileStorageType storage_type;
+	gs_unref_object NMSKeyfileStorage *storage = NULL;
+	GError *local = NULL;
+	const char *uuid;
+	gboolean reread_same;
+	struct timespec mtime;
+	char strbuf[100];
+
+	nm_assert (NM_IS_CONNECTION (connection));
+	nm_assert (out_storage && !*out_storage);
+	nm_assert (out_connection && !*out_connection);
+
+	uuid = nm_connection_get_uuid (connection);
+
+	storage_type =   !in_memory && priv->dirname_etc
+	               ? NMS_KEYFILE_STORAGE_TYPE_ETC
+	               : NMS_KEYFILE_STORAGE_TYPE_RUN;
 
-	if (nm_keyfile_utils_ignore_filename (filename, require_extension))
+	if (!nms_keyfile_writer_connection (connection,
+	                                    is_nm_generated,
+	                                    is_volatile,
+	                                      storage_type == NMS_KEYFILE_STORAGE_TYPE_ETC
+	                                    ? priv->dirname_etc
+	                                    : priv->dirname_run,
+	                                    _get_plugin_dir (priv),
+	                                    NULL,
+	                                    FALSE,
+	                                    FALSE,
+	                                    nm_sett_util_allow_filename_cb,
+	                                    NM_SETT_UTIL_ALLOW_FILENAME_DATA (&priv->storages, NULL),
+	                                    &full_filename,
+	                                    &reread,
+	                                    &reread_same,
+	                                    &local)) {
+		_LOGT ("commit: %s (%s) failed to add: %s",
+		       nm_connection_get_uuid (connection),
+		       nm_connection_get_id (connection),
+		       local->message);
+		g_propagate_error (error, local);
 		return FALSE;
+	}
 
-	connection = update_connection (self, NULL, filename, find_by_path (self, filename), TRUE, NULL, NULL);
+	if (   !reread
+	    || reread_same)
+		nm_g_object_ref_set (&reread, connection);
 
-	return (connection != NULL);
-}
+	nm_assert (_nm_connection_verify (reread, NULL) == NM_SETTING_VERIFY_SUCCESS);
+	nm_assert (nm_streq0 (nm_connection_get_uuid (connection), nm_connection_get_uuid (reread)));
 
-static void
-reload_connections (NMSettingsPlugin *config)
-{
-	read_connections (config);
+	nm_assert (full_filename && full_filename[0] == '/');
+	nm_assert (!nm_sett_util_storages_lookup_by_filename (&priv->storages, full_filename));
+
+	_LOGT ("commit: %s (%s) added as \"%s\"%s",
+	       uuid,
+	       nm_connection_get_id (connection),
+	       full_filename,
+	       _extra_flags_to_string (strbuf, sizeof (strbuf), is_nm_generated, is_volatile));
+
+	storage = nms_keyfile_storage_new_connection (self,
+	                                              g_steal_pointer (&reread),
+	                                              full_filename,
+	                                              storage_type,
+	                                              is_nm_generated ? NM_TERNARY_TRUE : NM_TERNARY_FALSE,
+	                                              is_volatile ? NM_TERNARY_TRUE : NM_TERNARY_FALSE,
+	                                              nm_sett_util_stat_mtime (full_filename, FALSE, &mtime));
+
+	nm_sett_util_storages_add_take (&priv->storages, g_object_ref (storage));
+
+	*out_connection = nms_keyfile_storage_steal_connection (storage);
+	*out_storage = NM_SETTINGS_STORAGE (g_steal_pointer (&storage));
+
+	return TRUE;
 }
 
-static NMSettingsConnection *
-add_connection (NMSettingsPlugin *config,
+static gboolean
+add_connection (NMSettingsPlugin *plugin,
                 NMConnection *connection,
-                gboolean save_to_disk,
+                NMSettingsStorage **out_storage,
+                NMConnection **out_connection,
                 GError **error)
 {
-	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN (config);
-	gs_free char *path = NULL;
+	return nms_keyfile_plugin_add_connection (NMS_KEYFILE_PLUGIN (plugin),
+	                                          connection,
+	                                          FALSE,
+	                                          FALSE,
+	                                          FALSE,
+	                                          out_storage,
+	                                          out_connection,
+	                                          error);
+}
+
+gboolean
+nms_keyfile_plugin_update_connection (NMSKeyfilePlugin *self,
+                                      NMSettingsStorage *storage_x,
+                                      NMConnection *connection,
+                                      gboolean is_nm_generated,
+                                      gboolean is_volatile,
+                                      gboolean force_rename,
+                                      NMSettingsStorage **out_storage,
+                                      NMConnection **out_connection,
+                                      GError **error)
+{
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+	NMSKeyfileStorage *storage = NMS_KEYFILE_STORAGE (storage_x);
+	gs_unref_object NMConnection *connection_clone = NULL;
 	gs_unref_object NMConnection *reread = NULL;
+	gs_free char *full_filename = NULL;
+	gs_free_error GError *local = NULL;
+	struct timespec mtime;
+	const char *previous_filename;
+	gboolean reread_same;
+	const char *uuid;
+
+	_nm_assert_storage (self, storage, TRUE);
+	nm_assert (NM_IS_CONNECTION (connection));
+	nm_assert (_nm_connection_verify (connection, NULL) == NM_SETTING_VERIFY_SUCCESS);
+	nm_assert (nm_streq (nms_keyfile_storage_get_uuid (storage), nm_connection_get_uuid (connection)));
+	nm_assert (!error || !*error);
+	nm_assert (NM_IN_SET (storage->storage_type, NMS_KEYFILE_STORAGE_TYPE_ETC,
+	                                             NMS_KEYFILE_STORAGE_TYPE_RUN));
+	nm_assert (   (!is_nm_generated && !is_volatile)
+	           || storage->storage_type == NMS_KEYFILE_STORAGE_TYPE_RUN);
+	nm_assert  (   priv->dirname_etc
+	            || storage->storage_type != NMS_KEYFILE_STORAGE_TYPE_ETC);
+
+	previous_filename = nms_keyfile_storage_get_filename (storage);
+	uuid = nms_keyfile_storage_get_uuid (storage);
 
 	if (!nms_keyfile_writer_connection (connection,
-	                                    save_to_disk,
-	                                    NULL,
+	                                    is_nm_generated,
+	                                    is_volatile,
+	                                      storage->storage_type == NMS_KEYFILE_STORAGE_TYPE_ETC
+	                                    ? priv->dirname_etc
+	                                    : priv->dirname_run,
+	                                    _get_plugin_dir (priv),
+	                                    previous_filename,
 	                                    FALSE,
-	                                    NULL,
-	                                    NULL,
-	                                    &path,
+	                                    FALSE,
+	                                    nm_sett_util_allow_filename_cb,
+	                                    NM_SETT_UTIL_ALLOW_FILENAME_DATA (&priv->storages, previous_filename),
+	                                    &full_filename,
 	                                    &reread,
-	                                    NULL,
-	                                    error))
-		return NULL;
+	                                    &reread_same,
+	                                    &local)) {
+		_LOGW ("commit: failure to write %s (%s) to \"%s\": %s",
+		       uuid,
+		       nm_connection_get_id (connection_clone),
+		       previous_filename,
+		       local->message);
+		g_propagate_error (error, g_steal_pointer (&local));
+		return FALSE;
+	}
 
-	return NM_SETTINGS_CONNECTION (update_connection (self, reread ?: connection, path, NULL, FALSE, NULL, error));
+	nm_assert (   full_filename
+	           && nm_streq (full_filename, previous_filename));
+
+	if (   !reread
+	    || reread_same)
+		nm_g_object_ref_set (&reread, connection);
+
+	nm_assert (_nm_connection_verify (reread, NULL) == NM_SETTING_VERIFY_SUCCESS);
+	nm_assert (nm_streq (nm_connection_get_uuid (reread), uuid));
+
+	_LOGT ("commit: \"%s\": profile %s (%s) written",
+	       full_filename,
+	       uuid,
+	       nm_connection_get_id (connection));
+
+	storage->is_nm_generated = is_nm_generated;
+	storage->is_volatile     = is_volatile;
+	storage->stat_mtime      = *nm_sett_util_stat_mtime (full_filename, FALSE, &mtime);
+
+	*out_storage = g_object_ref (NM_SETTINGS_STORAGE (storage));
+	*out_connection = g_steal_pointer (&reread);
+	return TRUE;
+}
+
+static gboolean
+update_connection (NMSettingsPlugin *plugin,
+                   NMSettingsStorage *storage,
+                   NMConnection *connection,
+                   NMSettingsStorage **out_storage,
+                   NMConnection **out_connection,
+                   GError **error)
+{
+	return nms_keyfile_plugin_update_connection (NMS_KEYFILE_PLUGIN (plugin),
+	                                             storage,
+	                                             connection,
+	                                             FALSE,
+	                                             FALSE,
+	                                             FALSE,
+	                                             out_storage,
+	                                             out_connection,
+	                                             error);
+}
+
+static gboolean
+delete_connection (NMSettingsPlugin *plugin,
+                   NMSettingsStorage *storage_x,
+                   GError **error)
+{
+	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN (plugin);
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+	gs_unref_object NMSKeyfileStorage *storage = g_object_ref (NMS_KEYFILE_STORAGE (storage_x));
+	const char *remove_from_disk_errmsg = NULL;
+	const char *operation_message;
+	const char *previous_filename;
+	const char *uuid;
+	gboolean success = TRUE;
+
+	_nm_assert_storage (self, storage, TRUE);
+	nm_assert (!error || !*error);
+
+	previous_filename = nms_keyfile_storage_get_filename (storage);
+	uuid = nms_keyfile_storage_get_uuid (storage);
+
+	if (!NM_IN_SET (storage->storage_type, NMS_KEYFILE_STORAGE_TYPE_ETC,
+	                                       NMS_KEYFILE_STORAGE_TYPE_RUN)) {
+		nm_utils_error_set (error,
+		                    NM_UTILS_ERROR_UNKNOWN,
+		                    "profile in read-only storage cannot be deleted");
+		success = FALSE;
+		operation_message = "dropped readonly file from memory";
+	} else if (unlink (previous_filename) != 0) {
+		int errsv;
+
+		errsv = errno;
+		if (errsv != ENOENT) {
+			remove_from_disk_errmsg = nm_strerror_native (errsv);
+			operation_message = "failed to delete from disk";
+			success = FALSE;
+			nm_utils_error_set_errno (error,
+			                          errsv,
+			                          "failure to delete \"%s\": %s",
+			                          previous_filename);
+		} else
+			operation_message = "does not exist on disk";
+	} else
+		operation_message = "deleted from disk";
+
+	_LOGT ("commit: deleted \"%s\", %s %s (%s%s%s%s)",
+	       previous_filename,
+	       storage->is_tombstone ? "tombstone" : "profile",
+	       uuid,
+	       operation_message,
+	       NM_PRINT_FMT_QUOTED (remove_from_disk_errmsg, ": ", remove_from_disk_errmsg, "", ""));
+
+	if (success) {
+		nm_sett_util_storages_steal (&priv->storages, storage);
+		storage->is_tombstone = FALSE;
+		nms_keyfile_storage_destroy (storage);
+	}
+
+	return success;
+}
+
+/**
+ * nms_keyfile_plugin_set_nmmeta_tombstone:
+ * @self: the #NMSKeyfilePlugin instance
+ * @simulate: if %TRUE, don't do anything on the filename but just pretend
+ *   that the loaded UUID file gets tracked/untracked. In this mode, the function
+ *   cannot fail (except on hard-failure, see below).
+ *   The idea is that you first try without simulate to write to disk.
+ *   If that fails, you might still want to forcefully pretend (in-memory
+ *   only) that this uuid is marked as tombstone (or not), as desired.
+ *   So you repeate the call with @simulate %TRUE.
+ * @uuid: the UUID for which to write/delete the nmmeta file
+ * @in_memory: the storage type, either /etc or /run. Note that if @self
+ *   has no /etc directory configured, this results in a hard failure.
+ * @set: if %TRUE, write the symlink to point to /dev/null. If %FALSE,
+ *   delete the nmmeta file (if it exists).
+ * @out_storage: (transfer full) (allow-none): the storage element that changes, or
+ *   NULL if nothing changed. Note that the file on disk is already as
+ *   we want to write it, then this still counts as a change. No change only
+ *   means if we try to delete a storage (@set %FALSE) that did not
+ *   exist previously.
+ * @out_hard_failure: (allow-none): on failure, indicate that this is a hard failure.
+ *
+ * The function writes or deletes nmmeta files to/from filesystem. In this case,
+ * the nmmeta files can only be symlinks to /dev/null (to indicate tombstones).
+ *
+ * A hard failure can only happen if @self has no /etc directory configured
+ * and @in_memory is FALSE. In such case even @simulate call fails (which
+ * otherwise would always succeed).
+ * Also, if you get a hard-failure (with @simulate %FALSE) there is no point
+ * in retrying with @simulate %TRUE (contrary to all other cases!).
+ *
+ * Returns: %TRUE on success.
+ */
+gboolean
+nms_keyfile_plugin_set_nmmeta_tombstone (NMSKeyfilePlugin *self,
+                                         gboolean simulate,
+                                         const char *uuid,
+                                         gboolean in_memory,
+                                         gboolean set,
+                                         NMSettingsStorage **out_storage,
+                                         gboolean *out_hard_failure)
+{
+	NMSKeyfilePluginPrivate *priv;
+	gboolean hard_failure = FALSE;
+	NMSKeyfileStorage *storage;
+	gs_unref_object NMSKeyfileStorage *storage_result = NULL;
+	gboolean nmmeta_success = FALSE;
+	gs_free char *nmmeta_filename = NULL;
+	NMSKeyfileStorageType storage_type;
+	const char *loaded_path;
+	const char *dirname;
+
+	nm_assert (NMS_IS_KEYFILE_PLUGIN (self));
+	nm_assert (nm_utils_is_uuid (uuid));
+	nm_assert (!out_storage || !*out_storage);
+
+	priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
+
+	loaded_path =   set
+	              ? NM_KEYFILE_PATH_NMMETA_SYMLINK_NULL
+	              : NULL;
+
+	if (in_memory) {
+		storage_type = NMS_KEYFILE_STORAGE_TYPE_RUN;
+		dirname = priv->dirname_run;
+	} else {
+		if (!priv->dirname_etc) {
+			_LOGT ("commit: cannot %s%s nmmeta symlink for %s as there is no /etc directory",
+			       simulate ? "simulate " : "",
+			       loaded_path ? "write" : "delete",
+			       uuid);
+			hard_failure = TRUE;
+			goto out;
+		}
+		storage_type = NMS_KEYFILE_STORAGE_TYPE_ETC;
+		dirname = priv->dirname_etc;
+	}
+
+	if (simulate) {
+		nmmeta_success = TRUE;
+		nmmeta_filename = nms_keyfile_nmmeta_filename (dirname, uuid, FALSE);
+	} else {
+		nmmeta_success = nms_keyfile_nmmeta_write (dirname,
+		                                           uuid,
+		                                           loaded_path,
+		                                           FALSE,
+		                                           &nmmeta_filename);
+	}
+
+	_LOGT ("commit: %s nmmeta symlink \"%s\"%s%s%s %s",
+	       loaded_path ? "writing" : "deleting",
+	       nmmeta_filename,
+	       NM_PRINT_FMT_QUOTED (loaded_path, " (pointing to \"", loaded_path, "\")", ""),
+	       simulate
+	       ? "simulated"
+	       : (  nmmeta_success
+	          ? "succeeded"
+	          : "failed"));
+
+	if (!nmmeta_success)
+		goto out;
+
+	storage = nm_sett_util_storages_lookup_by_filename (&priv->storages, nmmeta_filename);
+
+	nm_assert (   !storage
+	           || (   storage->is_tombstone
+	               && storage->storage_type == storage_type
+	               && nm_streq (nms_keyfile_storage_get_uuid (storage), uuid)));
+
+	if (loaded_path) {
+
+		if (!storage) {
+			storage = nms_keyfile_storage_new_tombstone (self,
+			                                             uuid,
+			                                             nmmeta_filename,
+			                                             storage_type);
+			nm_sett_util_storages_add_take (&priv->storages, storage);
+		}
+
+		storage_result = g_object_ref (storage);
+	} else {
+		if (storage) {
+			storage_result = nm_sett_util_storages_steal (&priv->storages, storage);
+			storage_result->is_tombstone = FALSE;
+		}
+	}
+
+out:
+	nm_assert (!nmmeta_success || !hard_failure);
+	nm_assert (nmmeta_success  || !storage_result);
+
+	NM_SET_OUT (out_hard_failure, hard_failure);
+	NM_SET_OUT (out_storage, (NMSettingsStorage *) g_steal_pointer (&storage_result));
+	return nmmeta_success;
+}
+
+/*****************************************************************************/
+
+static void
+config_changed_cb (NMConfig *config,
+                   NMConfigData *config_data,
+                   NMConfigChangeFlags changes,
+                   NMConfigData *old_data,
+                   NMSKeyfilePlugin *self)
+{
+	gs_free char *old_value = NULL;
+	gs_free char *new_value = NULL;
+
+	old_value = nm_config_data_get_value (old_data,    NM_CONFIG_KEYFILE_GROUP_KEYFILE, NM_CONFIG_KEYFILE_KEY_KEYFILE_UNMANAGED_DEVICES, NM_CONFIG_GET_VALUE_TYPE_SPEC);
+	new_value = nm_config_data_get_value (config_data, NM_CONFIG_KEYFILE_GROUP_KEYFILE, NM_CONFIG_KEYFILE_KEY_KEYFILE_UNMANAGED_DEVICES, NM_CONFIG_GET_VALUE_TYPE_SPEC);
+
+	if (!nm_streq0 (old_value, new_value))
+		_nm_settings_plugin_emit_signal_unmanaged_specs_changed (NM_SETTINGS_PLUGIN (self));
 }
 
 static GSList *
 get_unmanaged_specs (NMSettingsPlugin *config)
 {
-	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE ((NMSKeyfilePlugin *) config);
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (config);
 	gs_free char *value = NULL;
 
 	value = nm_config_data_get_value (nm_config_get_data (priv->config),
@@ -517,7 +1160,41 @@ nms_keyfile_plugin_init (NMSKeyfilePlugin *plugin)
 	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (plugin);
 
 	priv->config = g_object_ref (nm_config_get ());
-	priv->connections = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free, g_object_unref);
+
+	priv->storages = (NMSettUtilStorages) NM_SETT_UTIL_STORAGES_INIT (priv->storages, nms_keyfile_storage_destroy);
+
+	/* dirname_libs are a set of read-only directories with lower priority than /etc or /run.
+	 * There is nothing complicated about having multiple of such directories, so dirname_libs
+	 * is a list (which currently only has at most one directory). */
+	priv->dirname_libs[0] = nm_sd_utils_path_simplify (g_strdup (NM_KEYFILE_PATH_NAME_LIB), FALSE);
+	priv->dirname_libs[1] = NULL;
+	priv->dirname_run = nm_sd_utils_path_simplify (g_strdup (NM_KEYFILE_PATH_NAME_RUN), FALSE);
+	priv->dirname_etc = nm_config_data_get_value (NM_CONFIG_GET_DATA_ORIG,
+	                                              NM_CONFIG_KEYFILE_GROUP_KEYFILE,
+	                                              NM_CONFIG_KEYFILE_KEY_KEYFILE_PATH,
+	                                              NM_CONFIG_GET_VALUE_STRIP);
+	if (priv->dirname_etc && priv->dirname_etc[0] == '\0') {
+		/* special case: configure an empty keyfile path so that NM has no writable keyfile
+		 * directory. In this case, NM will only honor dirname_libs and dirname_run, meaning
+		 * it cannot persist profile to non-volatile memory. */
+		nm_clear_g_free (&priv->dirname_etc);
+	} else if (!priv->dirname_etc || priv->dirname_etc[0] != '/') {
+		/* either invalid path or unspecified. Use the default. */
+		g_free (priv->dirname_etc);
+		priv->dirname_etc = nm_sd_utils_path_simplify (g_strdup (NM_KEYFILE_PATH_NAME_ETC_DEFAULT), FALSE);
+	} else
+		nm_sd_utils_path_simplify (priv->dirname_etc, FALSE);
+
+	/* no duplicates */
+	if (NM_IN_STRSET (priv->dirname_libs[0], priv->dirname_etc,
+	                                         priv->dirname_run))
+		nm_clear_g_free (&priv->dirname_libs[0]);
+	if (NM_IN_STRSET (priv->dirname_etc, priv->dirname_run))
+		nm_clear_g_free (&priv->dirname_etc);
+
+	nm_assert (!priv->dirname_libs[0] || priv->dirname_libs[0][0] == '/');
+	nm_assert (!priv->dirname_etc     || priv->dirname_etc[0]     == '/');
+	nm_assert ( priv->dirname_run     && priv->dirname_run[0]     == '/');
 }
 
 static void
@@ -555,17 +1232,19 @@ nms_keyfile_plugin_new (void)
 static void
 dispose (GObject *object)
 {
-	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE ((NMSKeyfilePlugin *) object);
-
-	if (priv->connections) {
-		g_hash_table_destroy (priv->connections);
-		priv->connections = NULL;
-	}
+	NMSKeyfilePlugin *self = NMS_KEYFILE_PLUGIN (object);
+	NMSKeyfilePluginPrivate *priv = NMS_KEYFILE_PLUGIN_GET_PRIVATE (self);
 
-	if (priv->config) {
+	if (priv->config)
 		g_signal_handlers_disconnect_by_func (priv->config, config_changed_cb, object);
-		g_clear_object (&priv->config);
-	}
+
+	nm_sett_util_storages_clear (&priv->storages);
+
+	nm_clear_g_free (&priv->dirname_libs[0]);
+	nm_clear_g_free (&priv->dirname_etc);
+	nm_clear_g_free (&priv->dirname_run);
+
+	g_clear_object (&priv->config);
 
 	G_OBJECT_CLASS (nms_keyfile_plugin_parent_class)->dispose (object);
 }
@@ -579,9 +1258,11 @@ nms_keyfile_plugin_class_init (NMSKeyfilePluginClass *klass)
 	object_class->constructed = constructed;
 	object_class->dispose     = dispose;
 
-	plugin_class->get_connections     = get_connections;
-	plugin_class->load_connection     = load_connection;
+	plugin_class->plugin_name         = "keyfile";
+	plugin_class->get_unmanaged_specs = get_unmanaged_specs;
 	plugin_class->reload_connections  = reload_connections;
+	plugin_class->load_connections    = load_connections;
 	plugin_class->add_connection      = add_connection;
-	plugin_class->get_unmanaged_specs = get_unmanaged_specs;
+	plugin_class->update_connection   = update_connection;
+	plugin_class->delete_connection   = delete_connection;
 }
diff --git a/src/settings/plugins/keyfile/nms-keyfile-plugin.h b/src/settings/plugins/keyfile/nms-keyfile-plugin.h
index 95403c779b..f3e6870861 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-plugin.h
+++ b/src/settings/plugins/keyfile/nms-keyfile-plugin.h
@@ -21,6 +21,11 @@
 #ifndef __NMS_KEYFILE_PLUGIN_H__
 #define __NMS_KEYFILE_PLUGIN_H__
 
+#include "settings/nm-settings-plugin.h"
+#include "settings/nm-settings-storage.h"
+
+#include "nms-keyfile-utils.h"
+
 #define NMS_TYPE_KEYFILE_PLUGIN            (nms_keyfile_plugin_get_type ())
 #define NMS_KEYFILE_PLUGIN(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NMS_TYPE_KEYFILE_PLUGIN, NMSKeyfilePlugin))
 #define NMS_KEYFILE_PLUGIN_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NMS_TYPE_KEYFILE_PLUGIN, NMSKeyfilePluginClass))
@@ -35,4 +40,31 @@ GType nms_keyfile_plugin_get_type (void);
 
 NMSKeyfilePlugin *nms_keyfile_plugin_new (void);
 
+gboolean nms_keyfile_plugin_add_connection (NMSKeyfilePlugin *self,
+                                            NMConnection *connection,
+                                            gboolean is_nm_generated,
+                                            gboolean is_volatile,
+                                            gboolean in_memory,
+                                            NMSettingsStorage **out_storage,
+                                            NMConnection **out_connection,
+                                            GError **error);
+
+gboolean nms_keyfile_plugin_update_connection (NMSKeyfilePlugin *self,
+                                               NMSettingsStorage *storage,
+                                               NMConnection *connection,
+                                               gboolean is_nm_generated,
+                                               gboolean is_volatile,
+                                               gboolean force_rename,
+                                               NMSettingsStorage **out_storage,
+                                               NMConnection **out_connection,
+                                               GError **error);
+
+gboolean nms_keyfile_plugin_set_nmmeta_tombstone (NMSKeyfilePlugin *self,
+                                                  gboolean simulate,
+                                                  const char *uuid,
+                                                  gboolean in_memory,
+                                                  gboolean set,
+                                                  NMSettingsStorage **out_storage,
+                                                  gboolean *out_hard_failure);
+
 #endif /* __NMS_KEYFILE_PLUGIN_H__ */
diff --git a/src/settings/plugins/keyfile/nms-keyfile-reader.c b/src/settings/plugins/keyfile/nms-keyfile-reader.c
index 76b4828b42..a6562a04dc 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-reader.c
+++ b/src/settings/plugins/keyfile/nms-keyfile-reader.c
@@ -161,6 +161,9 @@ nms_keyfile_reader_from_keyfile (GKeyFile *key_file,
 NMConnection *
 nms_keyfile_reader_from_file (const char *full_filename,
                               const char *profile_dir,
+                              struct stat *out_stat,
+                              NMTernary *out_is_nm_generated,
+                              NMTernary *out_is_volatile,
                               GError **error)
 {
 	gs_unref_keyfile GKeyFile *key_file = NULL;
@@ -170,9 +173,12 @@ nms_keyfile_reader_from_file (const char *full_filename,
 	nm_assert (full_filename && full_filename[0] == '/');
 	nm_assert (!profile_dir || profile_dir[0] == '/');
 
+	NM_SET_OUT (out_is_nm_generated, NM_TERNARY_DEFAULT);
+	NM_SET_OUT (out_is_volatile, NM_TERNARY_DEFAULT);
+
 	if (!nms_keyfile_utils_check_file_permissions (NMS_KEYFILE_FILETYPE_KEYFILE,
 	                                               full_filename,
-	                                               NULL,
+	                                               out_stat,
 	                                               error))
 		return NULL;
 
@@ -194,6 +200,16 @@ nms_keyfile_reader_from_file (const char *full_filename,
 		connection = NULL;
 	}
 
+	NM_SET_OUT (out_is_nm_generated, nm_key_file_get_boolean (key_file,
+	                                                          NM_KEYFILE_GROUP_NMMETA,
+	                                                          NM_KEYFILE_KEY_NMMETA_NM_GENERATED,
+	                                                          NM_TERNARY_DEFAULT));
+
+	NM_SET_OUT (out_is_volatile, nm_key_file_get_boolean (key_file,
+	                                                      NM_KEYFILE_GROUP_NMMETA,
+	                                                      NM_KEYFILE_KEY_NMMETA_VOLATILE,
+	                                                      NM_TERNARY_DEFAULT));
+
 	return connection;
 }
 
diff --git a/src/settings/plugins/keyfile/nms-keyfile-reader.h b/src/settings/plugins/keyfile/nms-keyfile-reader.h
index 430096ebb7..b17b6dd77b 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-reader.h
+++ b/src/settings/plugins/keyfile/nms-keyfile-reader.h
@@ -30,8 +30,13 @@ NMConnection *nms_keyfile_reader_from_keyfile (GKeyFile *key_file,
                                                gboolean verbose,
                                                GError **error);
 
+struct stat;
+
 NMConnection *nms_keyfile_reader_from_file (const char *full_filename,
                                             const char *profile_dir,
+                                            struct stat *out_stat,
+                                            NMTernary *out_is_nm_generated,
+                                            NMTernary *out_is_volatile,
                                             GError **error);
 
 #endif /* __NMS_KEYFILE_READER_H__ */
diff --git a/src/settings/plugins/keyfile/nms-keyfile-storage.c b/src/settings/plugins/keyfile/nms-keyfile-storage.c
new file mode 100644
index 0000000000..885553a86c
--- /dev/null
+++ b/src/settings/plugins/keyfile/nms-keyfile-storage.c
@@ -0,0 +1,236 @@
+/* NetworkManager system settings service - keyfile plugin
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Copyright (C) 2018 Red Hat, Inc.
+ */
+
+#include "nm-default.h"
+
+#include "nms-keyfile-storage.h"
+
+#include "nm-utils.h"
+#include "nm-core-internal.h"
+#include "nms-keyfile-plugin.h"
+
+/*****************************************************************************/
+
+struct _NMSKeyfileStorageClass {
+	NMSettingsStorageClass parent;
+};
+
+G_DEFINE_TYPE (NMSKeyfileStorage, nms_keyfile_storage, NM_TYPE_SETTINGS_STORAGE)
+
+/*****************************************************************************/
+
+void
+nms_keyfile_storage_copy_content (NMSKeyfileStorage *dst,
+                                  const NMSKeyfileStorage *src)
+{
+	nm_assert (src != dst);
+	nm_assert (nm_streq (nms_keyfile_storage_get_uuid (dst), nms_keyfile_storage_get_uuid (src)));
+	nm_assert (nms_keyfile_storage_get_filename (dst) && nm_streq (nms_keyfile_storage_get_filename (dst), nms_keyfile_storage_get_filename (src)));
+
+	nm_g_object_ref_set (&dst->connection, src->connection);
+	dst->storage_type     = src->storage_type;
+	dst->stat_mtime       = src->stat_mtime;
+	dst->is_nm_generated  = src->is_nm_generated;
+	dst->is_volatile      = src->is_volatile;
+	dst->is_tombstone     = src->is_tombstone;
+}
+
+NMConnection *
+nms_keyfile_storage_steal_connection (NMSKeyfileStorage *self)
+{
+	nm_assert (NMS_IS_KEYFILE_STORAGE (self));
+	nm_assert (NM_IS_CONNECTION (self->connection));
+
+	return g_steal_pointer (&self->connection);
+}
+
+/*****************************************************************************/
+
+static int
+cmp_fcn (const NMSKeyfileStorage *a,
+         const NMSKeyfileStorage *b)
+{
+	nm_assert (NMS_IS_KEYFILE_STORAGE (a));
+	nm_assert (NMS_IS_KEYFILE_STORAGE (b));
+	nm_assert (a != b);
+
+	/* sort by storage-type, which also has a numeric value according to their
+	 * (inverse) priority. */
+	NM_CMP_FIELD_UNSAFE (b, a, storage_type);
+
+	/* tombstones are more important. */
+	nm_assert (a->is_tombstone == nm_settings_storage_is_keyfile_tombstone (NM_SETTINGS_STORAGE (a)));
+	nm_assert (b->is_tombstone == nm_settings_storage_is_keyfile_tombstone (NM_SETTINGS_STORAGE (b)));
+	NM_CMP_FIELD_UNSAFE (a, b, is_tombstone);
+
+	/* newer files are more important. */
+	NM_CMP_FIELD (b, a, stat_mtime.tv_sec);
+	NM_CMP_FIELD (b, a, stat_mtime.tv_nsec);
+
+	NM_CMP_DIRECT_STRCMP (nms_keyfile_storage_get_filename (a), nms_keyfile_storage_get_filename (b));
+
+	return 0;
+}
+
+/*****************************************************************************/
+
+static void
+nms_keyfile_storage_init (NMSKeyfileStorage *self)
+{
+}
+
+static NMSKeyfileStorage *
+_storage_new (NMSKeyfilePlugin *plugin,
+              const char *uuid,
+              const char *filename)
+{
+	nm_assert (NMS_IS_KEYFILE_PLUGIN (plugin));
+	nm_assert (nm_utils_is_uuid (uuid));
+	nm_assert (filename && filename[0] == '/');
+
+	return g_object_new (NMS_TYPE_KEYFILE_STORAGE,
+	                     NM_SETTINGS_STORAGE_PLUGIN, plugin,
+	                     NM_SETTINGS_STORAGE_UUID, uuid,
+	                     NM_SETTINGS_STORAGE_FILENAME, filename,
+	                     NULL);
+}
+
+NMSKeyfileStorage *
+nms_keyfile_storage_new_tombstone (NMSKeyfilePlugin *plugin,
+                                   const char *uuid,
+                                   const char *filename,
+                                   NMSKeyfileStorageType storage_type)
+{
+	NMSKeyfileStorage *self;
+
+	nm_assert (nm_utils_is_uuid (uuid));
+	nm_assert (filename && filename[0] == '/');
+	nm_assert (nms_keyfile_nmmeta_check_filename (filename, NULL));
+	nm_assert (NM_IN_SET (storage_type, NMS_KEYFILE_STORAGE_TYPE_ETC,
+	                                    NMS_KEYFILE_STORAGE_TYPE_RUN));
+
+	self = _storage_new (plugin, uuid, filename);
+
+	self->is_tombstone = TRUE;
+
+	self->storage_type = storage_type;
+
+	return self;
+}
+
+NMSKeyfileStorage *
+nms_keyfile_storage_new_connection (NMSKeyfilePlugin *plugin,
+                                    NMConnection *connection_take /* pass reference */,
+                                    const char *filename,
+                                    NMSKeyfileStorageType storage_type,
+                                    NMTernary is_nm_generated_opt,
+                                    NMTernary is_volatile_opt,
+                                    const struct timespec *stat_mtime)
+{
+	NMSKeyfileStorage *self;
+
+	nm_assert (NMS_IS_KEYFILE_PLUGIN (plugin));
+	nm_assert (NM_IS_CONNECTION (connection_take));
+	nm_assert (_nm_connection_verify (connection_take, NULL) == NM_SETTING_VERIFY_SUCCESS);
+	nm_assert (filename && filename[0] == '/');
+	nm_assert (   storage_type >= NMS_KEYFILE_STORAGE_TYPE_RUN
+	           && storage_type <= _NMS_KEYFILE_STORAGE_TYPE_LIB_LAST);
+	nmtst_connection_assert_unchanging (connection_take);
+
+	self = _storage_new (plugin, nm_connection_get_uuid (connection_take), filename);
+
+	self->connection = connection_take; /* take reference. */
+
+	if (storage_type == NMS_KEYFILE_STORAGE_TYPE_RUN) {
+		self->is_nm_generated = (is_nm_generated_opt == NM_TERNARY_TRUE);
+		self->is_volatile     = (is_volatile_opt == NM_TERNARY_TRUE);
+	}
+
+	if (stat_mtime)
+		self->stat_mtime = *stat_mtime;
+
+	self->storage_type = storage_type;
+
+	return self;
+}
+
+static void
+_storage_clear (NMSKeyfileStorage *self)
+{
+	c_list_unlink (&self->parent._storage_lst);
+	c_list_unlink (&self->parent._storage_by_uuid_lst);
+	g_clear_object (&self->connection);
+}
+
+static void
+dispose (GObject *object)
+{
+	NMSKeyfileStorage *self = NMS_KEYFILE_STORAGE (object);
+
+	_storage_clear (self);
+
+	G_OBJECT_CLASS (nms_keyfile_storage_parent_class)->dispose (object);
+}
+
+void
+nms_keyfile_storage_destroy (NMSKeyfileStorage *self)
+{
+	_storage_clear (self);
+	g_object_unref (self);
+}
+
+static void
+nms_keyfile_storage_class_init (NMSKeyfileStorageClass *klass)
+{
+	GObjectClass *object_class = G_OBJECT_CLASS (klass);
+	NMSettingsStorageClass *storage_class = NM_SETTINGS_STORAGE_CLASS (klass);
+
+	object_class->dispose = dispose;
+
+	storage_class->cmp_fcn = (int (*) (NMSettingsStorage *, NMSettingsStorage *)) cmp_fcn;
+}
+
+/*****************************************************************************/
+
+#include "settings/nm-settings-connection.h"
+
+void
+nm_settings_storage_load_sett_flags (NMSettingsStorage *self,
+                                     NMSettingsConnectionIntFlags *sett_flags,
+                                     NMSettingsConnectionIntFlags *sett_mask)
+{
+	NMSKeyfileStorage *s;
+
+	*sett_flags = NM_SETTINGS_CONNECTION_INT_FLAGS_NONE;
+	*sett_mask =   NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED
+	             | NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE;
+
+	if (!NMS_IS_KEYFILE_STORAGE (self))
+		return;
+
+	s = NMS_KEYFILE_STORAGE (self);
+	if (s->storage_type != NMS_KEYFILE_STORAGE_TYPE_RUN)
+		return;
+
+	if (s->is_nm_generated)
+		*sett_flags |= NM_SETTINGS_CONNECTION_INT_FLAGS_NM_GENERATED;
+
+	if (s->is_volatile)
+		*sett_flags |= NM_SETTINGS_CONNECTION_INT_FLAGS_VOLATILE;
+}
diff --git a/src/settings/plugins/keyfile/nms-keyfile-storage.h b/src/settings/plugins/keyfile/nms-keyfile-storage.h
new file mode 100644
index 0000000000..0cc537b7bf
--- /dev/null
+++ b/src/settings/plugins/keyfile/nms-keyfile-storage.h
@@ -0,0 +1,157 @@
+/* NetworkManager system settings service - keyfile plugin
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Copyright (C) 2018 Red Hat, Inc.
+ */
+
+#ifndef __NMS_KEYFILE_STORAGE_H__
+#define __NMS_KEYFILE_STORAGE_H__
+
+#include "c-list/src/c-list.h"
+#include "settings/nm-settings-storage.h"
+#include "nms-keyfile-utils.h"
+
+/*****************************************************************************/
+
+#define NMS_TYPE_KEYFILE_STORAGE            (nms_keyfile_storage_get_type ())
+#define NMS_KEYFILE_STORAGE(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), NMS_TYPE_KEYFILE_STORAGE, NMSKeyfileStorage))
+#define NMS_KEYFILE_STORAGE_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), NMS_TYPE_KEYFILE_STORAGE, NMSKeyfileStorageClass))
+#define NMS_IS_KEYFILE_STORAGE(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), NMS_TYPE_KEYFILE_STORAGE))
+#define NMS_IS_KEYFILE_STORAGE_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), NMS_TYPE_KEYFILE_STORAGE))
+#define NMS_KEYFILE_STORAGE_GET_CLASS(obj)  (G_TYPE_INSTANCE_GET_CLASS ((obj), NMS_TYPE_KEYFILE_STORAGE, NMSKeyfileStorageClass))
+
+typedef struct {
+	NMSettingsStorage parent;
+
+	/* The connection. Note that there are tombstones (loaded-uuid files to /dev/null)
+	 * that don't have a connection.
+	 *
+	 * Also, we don't actually remember the loaded connection after returning it
+	 * to NMSettings. So, also for regular storages (non-tombstones) this field
+	 * is often cleared. */
+	NMConnection *connection;
+
+	NMSKeyfileStorageType storage_type;
+
+	/* the timestamp (stat's mtime) of the keyfile. For tombstones this
+	 * is irrelevant. The purpose is that if the same storage type (directory) has
+	 * multiple files with the same UUID, then the newer file gets preferred. */
+	struct timespec stat_mtime;
+
+	/* these flags are only relevant for storages with %NMS_KEYFILE_STORAGE_TYPE_RUN
+	 * (and non-tombstones). This is to persist and reload these settings flags to
+	 * /run. */
+	bool is_nm_generated:1;
+	bool is_volatile:1;
+
+	/* whether this is a tombstone to hide a UUID (via the loaded uuid symlinks).
+	 * If this is falls, the storage contains a profile, though note that
+	 * the connection field will be cleared when it's not used. So, a non-tombstone
+	 * has a connection in principle, but the connection field may still be %NULL.
+	 *
+	 * Note that a tombstone instance doesn't have a connection, but NMSettings
+	 * considers it alive because is_tombstone is %TRUE. That means, once a tombstone
+	 * gets removed, this flag is cleared. Then the storage instance has no connnection
+	 * and is no longer a tombstone, and NMSettings considers it ready for deletion.
+	 */
+	bool is_tombstone:1;
+
+	/* this flag is only used during reload to mark and prune old entries. */
+	bool dirty:1;
+
+} NMSKeyfileStorage;
+
+typedef struct _NMSKeyfileStorageClass NMSKeyfileStorageClass;
+
+GType nms_keyfile_storage_get_type (void);
+
+struct _NMSKeyfilePlugin;
+
+NMSKeyfileStorage *nms_keyfile_storage_new_tombstone (struct _NMSKeyfilePlugin *self,
+                                                      const char *uuid,
+                                                      const char *filename,
+                                                      NMSKeyfileStorageType storage_type);
+
+NMSKeyfileStorage *nms_keyfile_storage_new_connection (struct _NMSKeyfilePlugin *self,
+                                                       NMConnection *connection_take /* pass reference */,
+                                                       const char *filename,
+                                                       NMSKeyfileStorageType storage_type,
+                                                       NMTernary is_nm_generated_opt,
+                                                       NMTernary is_volatile_opt,
+                                                       const struct timespec *stat_mtime);
+
+void nms_keyfile_storage_destroy (NMSKeyfileStorage *storage);
+
+/*****************************************************************************/
+
+void nms_keyfile_storage_copy_content (NMSKeyfileStorage *dst,
+                                       const NMSKeyfileStorage *src);
+
+NMConnection *nms_keyfile_storage_steal_connection (NMSKeyfileStorage *storage);
+
+/*****************************************************************************/
+
+static inline const char *
+nms_keyfile_storage_get_uuid (const NMSKeyfileStorage *self)
+{
+	return nm_settings_storage_get_uuid ((const NMSettingsStorage *) self);
+}
+
+static inline const char *
+nms_keyfile_storage_get_filename (const NMSKeyfileStorage *self)
+{
+	return nm_settings_storage_get_filename ((const NMSettingsStorage *) self);
+}
+
+/*****************************************************************************/
+
+static inline gboolean
+nm_settings_storage_is_keyfile_run (const NMSettingsStorage *self)
+{
+	return    NMS_IS_KEYFILE_STORAGE (self)
+	       && (((NMSKeyfileStorage *) self)->storage_type == NMS_KEYFILE_STORAGE_TYPE_RUN);
+}
+
+static inline gboolean
+nm_settings_storage_is_keyfile_lib (const NMSettingsStorage *self)
+{
+	return    NMS_IS_KEYFILE_STORAGE (self)
+	       && (((NMSKeyfileStorage *) self)->storage_type >= NMS_KEYFILE_STORAGE_TYPE_LIB_BASE);
+}
+
+static inline gboolean
+nm_settings_storage_is_keyfile_tombstone (const NMSettingsStorage *self)
+{
+	/* Only keyfile storage supports tombstones. They indicate that a uuid
+	 * is shadowed via a symlink to /dev/null.
+	 *
+	 * Note that tombstones don't have a NMConnection instead they shadow
+	 * a UUID. As such, NMSettings considers them alive also if they have
+	 * not profile. That means, when a tombstone gets removed for good,
+	 * the is_tombstone must be cleared (so that it becomes truly dead). */
+	return    NMS_IS_KEYFILE_STORAGE (self)
+	       && ((NMSKeyfileStorage *) self)->is_tombstone;
+}
+
+/*****************************************************************************/
+
+enum _NMSettingsConnectionIntFlags;
+
+void nm_settings_storage_load_sett_flags (NMSettingsStorage *self,
+                                          enum _NMSettingsConnectionIntFlags *sett_flags,
+                                          enum _NMSettingsConnectionIntFlags *sett_mask);
+
+#endif /* __NMS_KEYFILE_STORAGE_H__ */
diff --git a/src/settings/plugins/keyfile/nms-keyfile-utils.c b/src/settings/plugins/keyfile/nms-keyfile-utils.c
index faa91debae..0d03132793 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-utils.c
+++ b/src/settings/plugins/keyfile/nms-keyfile-utils.c
@@ -14,7 +14,7 @@
  * with this program; if not, write to the Free Software Foundation, Inc.,
  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  *
- * (C) Copyright 2010 Red Hat, Inc.
+ * (C) Copyright 2010 - 2018 Red Hat, Inc.
  */
 
 #include "nm-default.h"
@@ -305,22 +305,3 @@ nms_keyfile_utils_check_file_permissions (NMSKeyfileFiletype filetype,
 	NM_SET_OUT (out_st, st);
 	return TRUE;
 }
-
-/*****************************************************************************/
-
-const char *
-nms_keyfile_utils_get_path (void)
-{
-	static char *path = NULL;
-
-	if (G_UNLIKELY (!path)) {
-		path = nm_config_data_get_value (NM_CONFIG_GET_DATA_ORIG,
-		                                 NM_CONFIG_KEYFILE_GROUP_KEYFILE,
-		                                 NM_CONFIG_KEYFILE_KEY_KEYFILE_PATH,
-		                                 NM_CONFIG_GET_VALUE_STRIP | NM_CONFIG_GET_VALUE_NO_EMPTY);
-		if (!path)
-			path = g_strdup (""NM_KEYFILE_PATH_NAME_ETC_DEFAULT"");
-	}
-	return path;
-}
-
diff --git a/src/settings/plugins/keyfile/nms-keyfile-utils.h b/src/settings/plugins/keyfile/nms-keyfile-utils.h
index 8fa2e3914c..1f5280f727 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-utils.h
+++ b/src/settings/plugins/keyfile/nms-keyfile-utils.h
@@ -14,7 +14,7 @@
  * with this program; if not, write to the Free Software Foundation, Inc.,
  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  *
- * (C) Copyright 2010-2016 Red Hat, Inc.
+ * (C) Copyright 2010 - 2018 Red Hat, Inc.
  */
 
 #ifndef __NMS_KEYFILE_UTILS_H__
@@ -22,18 +22,25 @@
 
 #include "NetworkManagerUtils.h"
 
-#define NMS_KEYFILE_CONNECTION_LOG_PATH(path)  ((path) ?: "in-memory")
-#define NMS_KEYFILE_CONNECTION_LOG_FMT         "%s (%s,\"%s\")"
-#define NMS_KEYFILE_CONNECTION_LOG_ARG(con)    NMS_KEYFILE_CONNECTION_LOG_PATH (nm_settings_connection_get_filename ((NMSettingsConnection *) (con))), nm_settings_connection_get_uuid ((NMSettingsConnection *) (con)), nm_settings_connection_get_id ((NMSettingsConnection *) (con))
-#define NMS_KEYFILE_CONNECTION_LOG_FMTD        "%s (%s,\"%s\",%p)"
-#define NMS_KEYFILE_CONNECTION_LOG_ARGD(con)   NMS_KEYFILE_CONNECTION_LOG_PATH (nm_settings_connection_get_filename ((NMSettingsConnection *) (con))), nm_settings_connection_get_uuid ((NMSettingsConnection *) (con)), nm_settings_connection_get_id ((NMSettingsConnection *) (con)), (con)
-
 typedef enum {
 	NMS_KEYFILE_FILETYPE_KEYFILE,
 	NMS_KEYFILE_FILETYPE_NMLOADED,
 } NMSKeyfileFiletype;
 
-const char *nms_keyfile_utils_get_path (void);
+typedef enum {
+	NMS_KEYFILE_STORAGE_TYPE_RUN       = 1, /* read-write, runtime only, e.g. /run */
+	NMS_KEYFILE_STORAGE_TYPE_ETC       = 2, /* read-write, persistent,   e.g. /etc     */
+	NMS_KEYFILE_STORAGE_TYPE_LIB_BASE  = 3, /* read-only,                e.g. /usr/lib */
+
+	_NMS_KEYFILE_STORAGE_TYPE_LIB_LAST = 1000,
+} NMSKeyfileStorageType;
+
+static inline NMSKeyfileStorageType
+NMS_KEYFILE_STORAGE_TYPE_LIB (guint run_idx)
+{
+	nm_assert (run_idx <= (_NMS_KEYFILE_STORAGE_TYPE_LIB_LAST - NMS_KEYFILE_STORAGE_TYPE_LIB_BASE));
+	return NMS_KEYFILE_STORAGE_TYPE_LIB_BASE + run_idx;
+}
 
 /*****************************************************************************/
 
diff --git a/src/settings/plugins/keyfile/nms-keyfile-writer.c b/src/settings/plugins/keyfile/nms-keyfile-writer.c
index a0c3c17b98..54e62b7373 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-writer.c
+++ b/src/settings/plugins/keyfile/nms-keyfile-writer.c
@@ -168,6 +168,8 @@ _handler_write (NMConnection *connection,
 
 static gboolean
 _internal_write_connection (NMConnection *connection,
+                            gboolean is_nm_generated,
+                            gboolean is_volatile,
                             const char *keyfile_dir,
                             const char *profile_dir,
                             gboolean with_extension,
@@ -212,6 +214,21 @@ _internal_write_connection (NMConnection *connection,
 	kf_file = nm_keyfile_write (connection, _handler_write, &info, error);
 	if (!kf_file)
 		return FALSE;
+
+	if (is_nm_generated) {
+		g_key_file_set_boolean (kf_file,
+		                        NM_KEYFILE_GROUP_NMMETA,
+		                        NM_KEYFILE_KEY_NMMETA_NM_GENERATED,
+		                        TRUE);
+	}
+
+	if (is_volatile) {
+		g_key_file_set_boolean (kf_file,
+		                        NM_KEYFILE_GROUP_NMMETA,
+		                        NM_KEYFILE_KEY_NMMETA_VOLATILE,
+		                        TRUE);
+	}
+
 	kf_content_buf = g_key_file_to_data (kf_file, &kf_content_len, error);
 	if (!kf_content_buf)
 		return FALSE;
@@ -337,8 +354,12 @@ _internal_write_connection (NMConnection *connection,
 
 gboolean
 nms_keyfile_writer_connection (NMConnection *connection,
-                               gboolean save_to_disk,
+                               gboolean is_nm_generated,
+                               gboolean is_volatile,
+                               const char *keyfile_dir,
+                               const char *profile_dir,
                                const char *existing_path,
+                               gboolean existing_path_read_only,
                                gboolean force_rename,
                                NMSKeyfileWriterAllowFilenameCb allow_filename_cb,
                                gpointer allow_filename_user_data,
@@ -347,21 +368,16 @@ nms_keyfile_writer_connection (NMConnection *connection,
                                gboolean *out_reread_same,
                                GError **error)
 {
-	const char *keyfile_dir;
-
-	if (save_to_disk)
-		keyfile_dir = nms_keyfile_utils_get_path ();
-	else
-		keyfile_dir = NM_KEYFILE_PATH_NAME_RUN;
-
 	return _internal_write_connection (connection,
+	                                   is_nm_generated,
+	                                   is_volatile,
 	                                   keyfile_dir,
-	                                   nms_keyfile_utils_get_path (),
+	                                   profile_dir,
 	                                   TRUE,
 	                                   0,
 	                                   0,
 	                                   existing_path,
-	                                   FALSE,
+	                                   existing_path_read_only,
 	                                   force_rename,
 	                                   allow_filename_cb,
 	                                   allow_filename_user_data,
@@ -382,6 +398,8 @@ nms_keyfile_writer_test_connection (NMConnection *connection,
                                     GError **error)
 {
 	return _internal_write_connection (connection,
+	                                   FALSE,
+	                                   FALSE,
 	                                   keyfile_dir,
 	                                   keyfile_dir,
 	                                   FALSE,
diff --git a/src/settings/plugins/keyfile/nms-keyfile-writer.h b/src/settings/plugins/keyfile/nms-keyfile-writer.h
index db41b81c50..4fb9a20638 100644
--- a/src/settings/plugins/keyfile/nms-keyfile-writer.h
+++ b/src/settings/plugins/keyfile/nms-keyfile-writer.h
@@ -27,8 +27,12 @@ typedef gboolean (*NMSKeyfileWriterAllowFilenameCb) (const char *check_filename,
                                                      gpointer allow_filename_user_data);
 
 gboolean nms_keyfile_writer_connection (NMConnection *connection,
-                                        gboolean save_to_disk,
+                                        gboolean is_nm_generated,
+                                        gboolean is_volatile,
+                                        const char *keyfile_dir,
+                                        const char *profile_dir,
                                         const char *existing_path,
+                                        gboolean existing_path_read_only,
                                         gboolean force_rename,
                                         NMSKeyfileWriterAllowFilenameCb allow_filename_cb,
                                         gpointer allow_filename_user_data,
diff --git a/src/settings/plugins/keyfile/tests/test-keyfile-settings.c b/src/settings/plugins/keyfile/tests/test-keyfile-settings.c
index 112c92b73e..5942f04bb1 100644
--- a/src/settings/plugins/keyfile/tests/test-keyfile-settings.c
+++ b/src/settings/plugins/keyfile/tests/test-keyfile-settings.c
@@ -72,6 +72,9 @@ check_ip_route (NMSettingIPConfig *config, int idx, const char *destination, int
 	\
 	_connection = nms_keyfile_reader_from_file (full_filename, \
 	                                            NULL, \
+	                                            NULL, \
+	                                            NULL, \
+	                                            NULL, \
 	                                            (nmtst_get_rand_uint32 () % 2) ? &_error : NULL); \
 	nmtst_assert_success (_connection, _error); \
 	nmtst_assert_connection_verifies_without_normalization (_connection); \
diff --git a/tools/meson-post-install.sh b/tools/meson-post-install.sh
index d2474f475d..76b44371b1 100755
--- a/tools/meson-post-install.sh
+++ b/tools/meson-post-install.sh
@@ -30,6 +30,7 @@ for dir in "${pkgconfdir}/conf.d" \
            "${pkglibdir}/dispatcher.d/no-wait.d" \
            "${pkglibdir}/dispatcher.d/pre-down.d" \
            "${pkglibdir}/dispatcher.d/pre-up.d" \
+           "${pkglibdir}/system-connections" \
            "${pkglibdir}/VPN"; do
     mkdir -p "${DESTDIR}${dir}"
     chmod 0755 "${DESTDIR}${dir}"