==> Synchronizing chroot copy [/home/alhp/workspace/chroot/root] -> [build_32987531-4dcf-4cc4-8040-fb36af4c4755]...done ==> Making package: seaweedfs 4.40-1.1 (Mon Jul 20 23:26:36 2026) ==> Retrieving sources... -> Downloading seaweedfs-4.40.tar.gz... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 6.09M 0 6.09M 0 0 4.00M 0 00:01 6.08M 100 17.09M 0 17.09M 0 0 6.75M 0 00:02 8.51M 100 27.05M 0 27.05M 0 0 7.64M 0 00:03 8.97M 100 32.51M 0 32.51M 0 0 7.15M 0 00:04 8.08M 100 38.34M 0 38.34M 0 0 6.88M 0 00:05 7.59M 100 45.70M 0 45.70M 0 0 6.95M 0 00:06 7.85M 100 57.75M 0 57.75M 0 0 7.62M 0 00:07 8.06M 100 68.57M 0 68.57M 0 0 7.99M 0 00:08 8.23M 100 76.34M 0 76.34M 0 0 7.93M 0 00:09 8.63M 100 81.39M 0 81.39M 0 0 8.08M 0 00:10 8.63M 100 81.39M 0 81.39M 0 0 8.08M 0 00:10 8.63M 100 81.39M 0 81.39M 0 0 8.08M 0 00:10 8.63M ==> Validating source files with sha256sums... seaweedfs-4.40.tar.gz ... Passed ==> Making package: seaweedfs 4.40-1.1 (Mon Jul 20 21:26:55 2026) ==> Checking runtime dependencies... ==> Installing missing dependencies... resolving dependencies... looking for conflicting packages... Package (1) New Version Net Change extra/mailcap 2.1.54-2 0.11 MiB Total Installed Size: 0.11 MiB :: Proceed with installation? [Y/n] checking keyring... checking package integrity... loading package files... checking for file conflicts... :: Processing package changes... installing mailcap... :: Running post-transaction hooks... (1/1) Arming ConditionNeedsUpdate... ==> Checking buildtime dependencies... ==> Installing missing dependencies... resolving dependencies... looking for conflicting packages... Package (1) New Version Net Change extra/go 2:1.26.5-1 215.65 MiB Total Installed Size: 215.65 MiB :: Proceed with installation? [Y/n] checking keyring... checking package integrity... loading package files... checking for file conflicts... :: Processing package changes... installing go... :: Running post-transaction hooks... (1/1) Arming ConditionNeedsUpdate... ==> Retrieving sources... -> Found seaweedfs-4.40.tar.gz ==> WARNING: Skipping all source file integrity checks. ==> Extracting sources... -> Extracting seaweedfs-4.40.tar.gz with bsdtar ==> Starting prepare()... ==> Starting build()... ==> Starting check()... ? github.com/seaweedfs/seaweedfs/weed [no test files] === RUN TestStaticGzMirror --- PASS: TestStaticGzMirror (0.03s) === RUN TestStaticHandlerGzip --- PASS: TestStaticHandlerGzip (0.02s) === RUN TestStaticHandlerIdentity --- PASS: TestStaticHandlerIdentity (0.01s) === RUN TestStaticHandlerNotModified --- PASS: TestStaticHandlerNotModified (0.00s) === RUN TestStaticHandlerVariantETags --- PASS: TestStaticHandlerVariantETags (0.00s) === RUN TestStaticHandlerNotFound --- PASS: TestStaticHandlerNotFound (0.00s) === RUN TestAcceptsGzip --- PASS: TestAcceptsGzip (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin 0.083s === RUN TestApplyDefaults_WithEmbeddedStruct --- PASS: TestApplyDefaults_WithEmbeddedStruct (0.00s) === RUN TestApplyDefaults_PartiallySet --- PASS: TestApplyDefaults_PartiallySet (0.00s) === RUN TestApplyDefaults_NonPointer --- PASS: TestApplyDefaults_NonPointer (0.00s) === RUN TestApplyDefaults_NonStruct --- PASS: TestApplyDefaults_NonStruct (0.00s) === RUN TestApplyDefaults_EmptySchema --- PASS: TestApplyDefaults_EmptySchema (0.00s) === RUN TestApplyDefaults_MissingSchemaField --- PASS: TestApplyDefaults_MissingSchemaField (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/config 0.009s === RUN TestAdminLockManagerRefCountsOverlappingHolds === PAUSE TestAdminLockManagerRefCountsOverlappingHolds === RUN TestAdminLockManagerYieldsBeforeReacquire === PAUSE TestAdminLockManagerYieldsBeforeReacquire === RUN TestAdminLockManagerFairnessWindowForcesFullRelease === PAUSE TestAdminLockManagerFairnessWindowForcesFullRelease === RUN TestAdminLockManagerAcquireWaitsForReleaseInFlight === PAUSE TestAdminLockManagerAcquireWaitsForReleaseInFlight === RUN TestListClusterNodesRequestIncludesFilerGroup --- PASS: TestListClusterNodesRequestIncludesFilerGroup (0.00s) === RUN TestListClusterNodesRequestKeepsDefaultFilerGroupEmpty --- PASS: TestListClusterNodesRequestKeepsDefaultFilerGroupEmpty (0.00s) === RUN TestCleanMaintenanceScript === RUN TestCleanMaintenanceScript/empty === RUN TestCleanMaintenanceScript/only_lock_unlock === RUN TestCleanMaintenanceScript/strips_lock_and_unlock === RUN TestCleanMaintenanceScript/case_insensitive_lock === RUN TestCleanMaintenanceScript/preserves_comments_removal === RUN TestCleanMaintenanceScript/no_lock_unlock_present === RUN TestCleanMaintenanceScript/windows_line_endings === RUN TestCleanMaintenanceScript/lock_with_inline_comment === RUN TestCleanMaintenanceScript/command_with_inline_comment_preserved === RUN TestCleanMaintenanceScript/only_inline_comment_after_stripping === RUN TestCleanMaintenanceScript/typical_master_default --- PASS: TestCleanMaintenanceScript (0.00s) --- PASS: TestCleanMaintenanceScript/empty (0.00s) --- PASS: TestCleanMaintenanceScript/only_lock_unlock (0.00s) --- PASS: TestCleanMaintenanceScript/strips_lock_and_unlock (0.00s) --- PASS: TestCleanMaintenanceScript/case_insensitive_lock (0.00s) --- PASS: TestCleanMaintenanceScript/preserves_comments_removal (0.00s) --- PASS: TestCleanMaintenanceScript/no_lock_unlock_present (0.00s) --- PASS: TestCleanMaintenanceScript/windows_line_endings (0.00s) --- PASS: TestCleanMaintenanceScript/lock_with_inline_comment (0.00s) --- PASS: TestCleanMaintenanceScript/command_with_inline_comment_preserved (0.00s) --- PASS: TestCleanMaintenanceScript/only_inline_comment_after_stripping (0.00s) --- PASS: TestCleanMaintenanceScript/typical_master_default (0.00s) === RUN TestCollectCollectionStatsECUnevenShards --- PASS: TestCollectCollectionStatsECUnevenShards (0.00s) === RUN TestCollectCollectionStatsECEmptyCollection --- PASS: TestCollectCollectionStatsECEmptyCollection (0.00s) === RUN TestCollectCollectionStatsECFileAndDeleteCountAggregation --- PASS: TestCollectCollectionStatsECFileAndDeleteCountAggregation (0.00s) === RUN TestCollectCollectionStatsECFileCountMaxDedupe --- PASS: TestCollectCollectionStatsECFileCountMaxDedupe (0.00s) === RUN TestApplyMaintenanceConfigFromToml I0720 21:29:49.082944 config_toml.go:95 Applied [maintenance.vacuum] settings from admin.toml I0720 21:29:49.083239 config_toml.go:107 Applied [maintenance.erasure_coding] settings from admin.toml --- PASS: TestApplyMaintenanceConfigFromToml (0.00s) === RUN TestApplyMaintenanceConfigFromTomlNoKeys --- PASS: TestApplyMaintenanceConfigFromTomlNoKeys (0.00s) === RUN TestApplyMaintenanceConfigFromTomlRequiresDataDir --- PASS: TestApplyMaintenanceConfigFromTomlRequiresDataDir (0.00s) === RUN TestGenerateBreadcrumbs === RUN TestGenerateBreadcrumbs/root_path === RUN TestGenerateBreadcrumbs/simple_path === RUN TestGenerateBreadcrumbs/nested_path === RUN TestGenerateBreadcrumbs/bucket_path === RUN TestGenerateBreadcrumbs/bucket_nested_path === RUN TestGenerateBreadcrumbs/path_with_trailing_slash --- PASS: TestGenerateBreadcrumbs (0.00s) --- PASS: TestGenerateBreadcrumbs/root_path (0.00s) --- PASS: TestGenerateBreadcrumbs/simple_path (0.00s) --- PASS: TestGenerateBreadcrumbs/nested_path (0.00s) --- PASS: TestGenerateBreadcrumbs/bucket_path (0.00s) --- PASS: TestGenerateBreadcrumbs/bucket_nested_path (0.00s) --- PASS: TestGenerateBreadcrumbs/path_with_trailing_slash (0.00s) === RUN TestPathHandlingWithForwardSlashes === RUN TestPathHandlingWithForwardSlashes/root === RUN TestPathHandlingWithForwardSlashes/single_level === RUN TestPathHandlingWithForwardSlashes/multiple_levels === RUN TestPathHandlingWithForwardSlashes/bucket_path --- PASS: TestPathHandlingWithForwardSlashes (0.00s) --- PASS: TestPathHandlingWithForwardSlashes/root (0.00s) --- PASS: TestPathHandlingWithForwardSlashes/single_level (0.00s) --- PASS: TestPathHandlingWithForwardSlashes/multiple_levels (0.00s) --- PASS: TestPathHandlingWithForwardSlashes/bucket_path (0.00s) === RUN TestParentPathCalculationLogic === RUN TestParentPathCalculationLogic/root_path === RUN TestParentPathCalculationLogic/single_level === RUN TestParentPathCalculationLogic/two_levels === RUN TestParentPathCalculationLogic/deep_nesting === RUN TestParentPathCalculationLogic/bucket_root === RUN TestParentPathCalculationLogic/bucket_directory --- PASS: TestParentPathCalculationLogic (0.00s) --- PASS: TestParentPathCalculationLogic/root_path (0.00s) --- PASS: TestParentPathCalculationLogic/single_level (0.00s) --- PASS: TestParentPathCalculationLogic/two_levels (0.00s) --- PASS: TestParentPathCalculationLogic/deep_nesting (0.00s) --- PASS: TestParentPathCalculationLogic/bucket_root (0.00s) --- PASS: TestParentPathCalculationLogic/bucket_directory (0.00s) === RUN TestFileExtensionHandlingLogic === RUN TestFileExtensionHandlingLogic/file.txt === RUN TestFileExtensionHandlingLogic/file.log === RUN TestFileExtensionHandlingLogic/archive.tar.gz === RUN TestFileExtensionHandlingLogic/image.jpg === RUN TestFileExtensionHandlingLogic/document.pdf === RUN TestFileExtensionHandlingLogic/data.json === RUN TestFileExtensionHandlingLogic/noextension === RUN TestFileExtensionHandlingLogic/.hidden === RUN TestFileExtensionHandlingLogic/file.TXT === RUN TestFileExtensionHandlingLogic/file.JPG --- PASS: TestFileExtensionHandlingLogic (0.00s) --- PASS: TestFileExtensionHandlingLogic/file.txt (0.00s) --- PASS: TestFileExtensionHandlingLogic/file.log (0.00s) --- PASS: TestFileExtensionHandlingLogic/archive.tar.gz (0.00s) --- PASS: TestFileExtensionHandlingLogic/image.jpg (0.00s) --- PASS: TestFileExtensionHandlingLogic/document.pdf (0.00s) --- PASS: TestFileExtensionHandlingLogic/data.json (0.00s) --- PASS: TestFileExtensionHandlingLogic/noextension (0.00s) --- PASS: TestFileExtensionHandlingLogic/.hidden (0.00s) --- PASS: TestFileExtensionHandlingLogic/file.TXT (0.00s) --- PASS: TestFileExtensionHandlingLogic/file.JPG (0.00s) === RUN TestBucketPathDetectionLogic === RUN TestBucketPathDetectionLogic/root_is_not_a_bucket_path === RUN TestBucketPathDetectionLogic/buckets_root === RUN TestBucketPathDetectionLogic/single_bucket === RUN TestBucketPathDetectionLogic/bucket_with_nested_path === RUN TestBucketPathDetectionLogic/non-bucket_path --- PASS: TestBucketPathDetectionLogic (0.00s) --- PASS: TestBucketPathDetectionLogic/root_is_not_a_bucket_path (0.00s) --- PASS: TestBucketPathDetectionLogic/buckets_root (0.00s) --- PASS: TestBucketPathDetectionLogic/single_bucket (0.00s) --- PASS: TestBucketPathDetectionLogic/bucket_with_nested_path (0.00s) --- PASS: TestBucketPathDetectionLogic/non-bucket_path (0.00s) === RUN TestPathJoinHandlesEdgeCases === RUN TestPathJoinHandlesEdgeCases/root_directory === RUN TestPathJoinHandlesEdgeCases/simple_directory === RUN TestPathJoinHandlesEdgeCases/nested_directory === RUN TestPathJoinHandlesEdgeCases/handles_trailing_slash === RUN TestPathJoinHandlesEdgeCases/handles_empty_name --- PASS: TestPathJoinHandlesEdgeCases (0.00s) --- PASS: TestPathJoinHandlesEdgeCases/root_directory (0.00s) --- PASS: TestPathJoinHandlesEdgeCases/simple_directory (0.00s) --- PASS: TestPathJoinHandlesEdgeCases/nested_directory (0.00s) --- PASS: TestPathJoinHandlesEdgeCases/handles_trailing_slash (0.00s) --- PASS: TestPathJoinHandlesEdgeCases/handles_empty_name (0.00s) === RUN TestWindowsPathNormalizationBehavior === RUN TestWindowsPathNormalizationBehavior/backslash_separator === RUN TestWindowsPathNormalizationBehavior/mixed_separators === RUN TestWindowsPathNormalizationBehavior/already_normalized === RUN TestWindowsPathNormalizationBehavior/simple_backslash_path === RUN TestWindowsPathNormalizationBehavior/deep_nested_path --- PASS: TestWindowsPathNormalizationBehavior (0.00s) --- PASS: TestWindowsPathNormalizationBehavior/backslash_separator (0.00s) --- PASS: TestWindowsPathNormalizationBehavior/mixed_separators (0.00s) --- PASS: TestWindowsPathNormalizationBehavior/already_normalized (0.00s) --- PASS: TestWindowsPathNormalizationBehavior/simple_backslash_path (0.00s) --- PASS: TestWindowsPathNormalizationBehavior/deep_nested_path (0.00s) === RUN TestBreadcrumbPathFormatting === RUN TestBreadcrumbPathFormatting/breadcrumbs_for_/ === RUN TestBreadcrumbPathFormatting/breadcrumbs_for_/folder === RUN TestBreadcrumbPathFormatting/breadcrumbs_for_/folder/subfolder === RUN TestBreadcrumbPathFormatting/breadcrumbs_for_/buckets/mybucket === RUN TestBreadcrumbPathFormatting/breadcrumbs_for_/buckets/mybucket/data --- PASS: TestBreadcrumbPathFormatting (0.00s) --- PASS: TestBreadcrumbPathFormatting/breadcrumbs_for_/ (0.00s) --- PASS: TestBreadcrumbPathFormatting/breadcrumbs_for_/folder (0.00s) --- PASS: TestBreadcrumbPathFormatting/breadcrumbs_for_/folder/subfolder (0.00s) --- PASS: TestBreadcrumbPathFormatting/breadcrumbs_for_/buckets/mybucket (0.00s) --- PASS: TestBreadcrumbPathFormatting/breadcrumbs_for_/buckets/mybucket/data (0.00s) === RUN TestDirectoryNavigation === RUN TestDirectoryNavigation/navigate_from_root === RUN TestDirectoryNavigation/navigate_from_single_folder === RUN TestDirectoryNavigation/navigate_from_nested_folder === RUN TestDirectoryNavigation/navigate_bucket_contents --- PASS: TestDirectoryNavigation (0.00s) --- PASS: TestDirectoryNavigation/navigate_from_root (0.00s) --- PASS: TestDirectoryNavigation/navigate_from_single_folder (0.00s) --- PASS: TestDirectoryNavigation/navigate_from_nested_folder (0.00s) --- PASS: TestDirectoryNavigation/navigate_bucket_contents (0.00s) === RUN TestResolveEntryMimePrefersStoredMime --- PASS: TestResolveEntryMimePrefersStoredMime (0.00s) === RUN TestResolveEntryMimePrefersStoredMimeMalformedParameter --- PASS: TestResolveEntryMimePrefersStoredMimeMalformedParameter (0.00s) === RUN TestResolveEntryMimeFallsBackToFilename --- PASS: TestResolveEntryMimeFallsBackToFilename (0.00s) === RUN TestResolveEntryMimeReturnsDirectoryMime --- PASS: TestResolveEntryMimeReturnsDirectoryMime (0.00s) === RUN TestResolveEntryMimeWhitespaceMimeFallsBackToFilename --- PASS: TestResolveEntryMimeWhitespaceMimeFallsBackToFilename (0.00s) === RUN TestIcebergLocationToFilerPath --- PASS: TestIcebergLocationToFilerPath (0.00s) === RUN TestFormatParquetRow --- PASS: TestFormatParquetRow (0.01s) === RUN TestTruncateCell --- PASS: TestTruncateCell (0.00s) === RUN TestExpirePluginJobAPI === RUN TestExpirePluginJobAPI/empty_job_id === RUN TestExpirePluginJobAPI/invalid_json === RUN TestExpirePluginJobAPI/job_not_found === RUN TestExpirePluginJobAPI/successful_expire === RUN TestExpirePluginJobAPI/non-active_job --- PASS: TestExpirePluginJobAPI (0.00s) --- PASS: TestExpirePluginJobAPI/empty_job_id (0.00s) --- PASS: TestExpirePluginJobAPI/invalid_json (0.00s) --- PASS: TestExpirePluginJobAPI/job_not_found (0.00s) --- PASS: TestExpirePluginJobAPI/successful_expire (0.00s) --- PASS: TestExpirePluginJobAPI/non-active_job (0.00s) === RUN TestApplyDescriptorDefaultsToPersistedConfigBackfillsAdminDefaults === PAUSE TestApplyDescriptorDefaultsToPersistedConfigBackfillsAdminDefaults === RUN TestApplyDescriptorDefaultsToPersistedConfigReplacesBlankAdminScript === PAUSE TestApplyDescriptorDefaultsToPersistedConfigReplacesBlankAdminScript === RUN TestFilterTrackedJobsByLane === PAUSE TestFilterTrackedJobsByLane === RUN TestFilterActivitiesByLane === PAUSE TestFilterActivitiesByLane === RUN TestGetObjectStoreUserDetailsIncludesInlinePolicies I0720 21:29:49.092832 config_loader.go:73 Using explicit credential store: memory --- PASS: TestGetObjectStoreUserDetailsIncludesInlinePolicies (0.00s) === RUN TestFilerAddressFunctionInterface user_management_test.go:46: FilerEtcStore correctly implements SetFilerAddressFunc interface --- PASS: TestFilerAddressFunctionInterface (0.00s) === RUN TestGenerateAccessKey --- PASS: TestGenerateAccessKey (0.00s) === RUN TestGenerateSecretKey --- PASS: TestGenerateSecretKey (0.00s) === RUN TestGenerateAccountId --- PASS: TestGenerateAccountId (0.00s) === RUN TestVacuumMonitorStateTransitions === PAUSE TestVacuumMonitorStateTransitions === RUN TestSyncVacuumState === PAUSE TestSyncVacuumState === RUN TestBuildExportVolumeDerivedFields --- PASS: TestBuildExportVolumeDerivedFields (0.00s) === RUN TestBuildExportDiskFilterAndTotals --- PASS: TestBuildExportDiskFilterAndTotals (0.00s) === RUN TestBuildExportDiskPrunesWhenNoMatch --- PASS: TestBuildExportDiskPrunesWhenNoMatch (0.00s) === RUN TestFindDuplicateVolumeIdsForExport --- PASS: TestFindDuplicateVolumeIdsForExport (0.00s) === RUN TestSendToWorkerDropsOnCancelledContext --- PASS: TestSendToWorkerDropsOnCancelledContext (0.00s) === RUN TestHeartbeatDuringUnregisterDoesNotPanic --- PASS: TestHeartbeatDuringUnregisterDoesNotPanic (0.01s) === CONT TestAdminLockManagerRefCountsOverlappingHolds --- PASS: TestAdminLockManagerRefCountsOverlappingHolds (0.00s) === CONT TestSyncVacuumState === RUN TestSyncVacuumState/no_change_when_state_matches === RUN TestSyncVacuumState/worker_connects_triggers_disable I0720 21:29:49.104579 admin_server.go:368 Vacuum plugin worker connected, disabling master automatic vacuum === RUN TestSyncVacuumState/worker_disconnects_triggers_enable I0720 21:29:49.104594 admin_server.go:377 Vacuum plugin worker disconnected, re-enabling master automatic vacuum --- PASS: TestSyncVacuumState (0.00s) --- PASS: TestSyncVacuumState/no_change_when_state_matches (0.00s) --- PASS: TestSyncVacuumState/worker_connects_triggers_disable (0.00s) --- PASS: TestSyncVacuumState/worker_disconnects_triggers_enable (0.00s) === CONT TestVacuumMonitorStateTransitions === CONT TestFilterActivitiesByLane --- PASS: TestFilterActivitiesByLane (0.00s) === CONT TestFilterTrackedJobsByLane --- PASS: TestFilterTrackedJobsByLane (0.00s) === CONT TestApplyDescriptorDefaultsToPersistedConfigReplacesBlankAdminScript --- PASS: TestApplyDescriptorDefaultsToPersistedConfigReplacesBlankAdminScript (0.00s) === CONT TestApplyDescriptorDefaultsToPersistedConfigBackfillsAdminDefaults --- PASS: TestApplyDescriptorDefaultsToPersistedConfigBackfillsAdminDefaults (0.00s) === CONT TestAdminLockManagerAcquireWaitsForReleaseInFlight === CONT TestAdminLockManagerFairnessWindowForcesFullRelease === CONT TestAdminLockManagerYieldsBeforeReacquire --- PASS: TestVacuumMonitorStateTransitions (0.01s) --- PASS: TestAdminLockManagerFairnessWindowForcesFullRelease (0.08s) --- PASS: TestAdminLockManagerYieldsBeforeReacquire (0.10s) --- PASS: TestAdminLockManagerAcquireWaitsForReleaseInFlight (0.12s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/dash 0.269s === RUN TestSetupRoutes_RegistersPluginSchedulerStatesAPI_NoAuth --- PASS: TestSetupRoutes_RegistersPluginSchedulerStatesAPI_NoAuth (0.02s) === RUN TestSetupRoutes_RegistersPluginSchedulerStatesAPI_WithAuth --- PASS: TestSetupRoutes_RegistersPluginSchedulerStatesAPI_WithAuth (0.00s) === RUN TestSetupRoutes_RegistersPluginPages_NoAuth --- PASS: TestSetupRoutes_RegistersPluginPages_NoAuth (0.00s) === RUN TestSetupRoutes_RegistersPluginPages_WithAuth --- PASS: TestSetupRoutes_RegistersPluginPages_WithAuth (0.01s) === RUN TestValidateAndCleanFilePath_AllowsControlChars --- PASS: TestValidateAndCleanFilePath_AllowsControlChars (0.00s) === RUN TestValidateAndCleanFilePath_RejectsEmpty --- PASS: TestValidateAndCleanFilePath_RejectsEmpty (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/handlers 0.078s ? github.com/seaweedfs/seaweedfs/weed/admin/internal/httputil [no test files] === RUN TestMaintenanceManager_ErrorHandling E0720 21:29:49.007778 maintenance_manager.go:393 Maintenance scan failed: dial tcp [::1]:19333: connect: connection refused (will retry with backoff) E0720 21:29:49.008078 maintenance_manager.go:395 Maintenance scan still failing after 2 attempts: dial tcp [::1]:19333: connect: connection refused (backoff: 2s) E0720 21:29:49.008085 maintenance_manager.go:395 Maintenance scan still failing after 3 attempts: dial tcp [::1]:19333: connect: connection refused (backoff: 4s) E0720 21:29:49.008088 maintenance_manager.go:395 Maintenance scan still failing after 6 attempts: dial tcp [::1]:19333: connect: connection refused (backoff: 32s) E0720 21:29:49.008091 maintenance_manager.go:395 Maintenance scan still failing after 9 attempts: dial tcp [::1]:19333: connect: connection refused (backoff: 4m16s) E0720 21:29:49.008093 maintenance_manager.go:395 Maintenance scan still failing after 10 attempts: dial tcp [::1]:19333: connect: connection refused (backoff: 5m0s) --- PASS: TestMaintenanceManager_ErrorHandling (0.00s) === RUN TestIsConnectionError --- PASS: TestIsConnectionError (0.00s) === RUN TestMaintenanceManager_GetErrorState E0720 21:29:49.008146 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008149 maintenance_manager.go:399 Maintenance scan failed: test error --- PASS: TestMaintenanceManager_GetErrorState (0.00s) === RUN TestMaintenanceManager_LogThrottling E0720 21:29:49.008170 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008172 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008174 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008176 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008179 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008181 maintenance_manager.go:399 Maintenance scan failed: test error E0720 21:29:49.008183 maintenance_manager.go:399 Maintenance scan failed: test error --- PASS: TestMaintenanceManager_LogThrottling (0.00s) === RUN TestCanScheduleTaskNow_FallbackLogic --- PASS: TestCanScheduleTaskNow_FallbackLogic (0.00s) === RUN TestCanScheduleTaskNow_FallbackWithRunningTasks --- PASS: TestCanScheduleTaskNow_FallbackWithRunningTasks (0.00s) === RUN TestCanScheduleTaskNow_DifferentTaskTypes --- PASS: TestCanScheduleTaskNow_DifferentTaskTypes (0.00s) === RUN TestCanScheduleTaskNow_WithIntegration --- PASS: TestCanScheduleTaskNow_WithIntegration (0.00s) === RUN TestGetRunningTaskCount --- PASS: TestGetRunningTaskCount (0.00s) === RUN TestCanExecuteTaskType --- PASS: TestCanExecuteTaskType (0.00s) === RUN TestGetMaxConcurrentForTaskType_DefaultBehavior --- PASS: TestGetMaxConcurrentForTaskType_DefaultBehavior (0.00s) === RUN TestCanScheduleTaskNow_NilTask --- PASS: TestCanScheduleTaskNow_NilTask (0.00s) === RUN TestCanScheduleTaskNow_EmptyTaskType --- PASS: TestCanScheduleTaskNow_EmptyTaskType (0.00s) === RUN TestCanScheduleTaskNow_WithPolicy --- PASS: TestCanScheduleTaskNow_WithPolicy (0.00s) === RUN TestMaintenanceQueue_TaskIDPreservation I0720 21:29:49.008322 maintenance_queue.go:205 Task queued: ec_task_123 (erasure_coding) volume 100 on server1, priority 1, reason: I0720 21:29:49.008332 maintenance_queue.go:205 Task queued: manual_id_456 (vacuum) volume 0 on , priority 0, reason: --- PASS: TestMaintenanceQueue_TaskIDPreservation (0.00s) === RUN TestMaintenanceQueue_ActiveTopologySync I0720 21:29:49.008389 maintenance_queue.go:205 Task queued: sync_test_123 (balance) volume 100 on server1, priority 0, reason: I0720 21:29:49.008402 maintenance_queue.go:503 Task assigned: sync_test_123 (balance) → worker worker1 (volume 100, server server1) I0720 21:29:49.008423 maintenance_queue.go:610 Task completed: sync_test_123 (balance) worker worker1, duration 13.566µs, volume 100 --- PASS: TestMaintenanceQueue_ActiveTopologySync (0.00s) === RUN TestMaintenanceQueue_StaleWorkerCapacityRelease I0720 21:29:49.008448 maintenance_queue.go:205 Task queued: stale_test_123 (balance) volume 100 on server1, priority 0, reason: I0720 21:29:49.008468 maintenance_queue.go:503 Task assigned: stale_test_123 (balance) → worker worker1 (volume 100, server server1) W0720 21:29:49.008473 maintenance_queue.go:984 Removed stale maintenance worker worker1 --- PASS: TestMaintenanceQueue_StaleWorkerCapacityRelease (0.00s) === RUN TestMaintenanceManager_CancelTaskCapacityRelease I0720 21:29:49.008496 maintenance_queue.go:205 Task queued: cancel_test_123 (balance) volume 100 on server1, priority 0, reason: --- PASS: TestMaintenanceManager_CancelTaskCapacityRelease (0.00s) === RUN TestMaintenanceQueue_LoadRequeuesInFlightTasks I0720 21:29:49.008514 maintenance_queue.go:96 Task queue initialized: re-queued 2 in-flight task(s) from persistence; scanner will re-detect the rest --- PASS: TestMaintenanceQueue_LoadRequeuesInFlightTasks (0.00s) === RUN TestMaintenanceQueue_RetryCapacitySync I0720 21:29:49.008546 maintenance_queue.go:205 Task queued: retry_test_123 (balance) volume 100 on server1, priority 0, reason: I0720 21:29:49.008557 maintenance_queue.go:503 Task assigned: retry_test_123 (balance) → worker worker1 (volume 100, server server1) W0720 21:29:49.008564 maintenance_queue.go:583 Task failed, scheduling retry: retry_test_123 (balance) attempt 1/3, worker worker1, duration 12.154µs, error: simulated failure --- PASS: TestMaintenanceQueue_RetryCapacitySync (0.00s) === RUN TestMaintenanceQueue_AssignTaskRollback I0720 21:29:49.008586 maintenance_queue.go:205 Task queued: rollback_test_123 (balance) volume 100 on server1, priority 0, reason: W0720 21:29:49.008592 maintenance_queue.go:470 Failed to update ActiveTopology for task assignment rollback_test_123: pending task rollback_test_123 not found. Rolling back assignment. --- PASS: TestMaintenanceQueue_AssignTaskRollback (0.00s) === RUN TestGetNextTask_SkipsVolumeConflictsAcrossTypes I0720 21:29:49.008606 maintenance_queue.go:205 Task queued: t1 (balance) volume 100 on server1, priority 2, reason: I0720 21:29:49.008611 maintenance_queue.go:205 Task queued: t3 (vacuum) volume 200 on server1, priority 1, reason: I0720 21:29:49.008617 maintenance_queue.go:503 Task assigned: t1 (balance) → worker worker1 (volume 100, server server1) I0720 21:29:49.008620 maintenance_queue.go:503 Task assigned: t3 (vacuum) → worker worker2 (volume 200, server server1) --- PASS: TestGetNextTask_SkipsVolumeConflictsAcrossTypes (0.00s) === RUN TestAddTask_OnePendingTaskPerVolume I0720 21:29:49.008633 maintenance_queue.go:205 Task queued: t1 (balance) volume 100 on server1, priority 0, reason: --- PASS: TestAddTask_OnePendingTaskPerVolume (0.00s) === RUN TestAddTask_RejectsWhenVolumeHasRunningTask I0720 21:29:49.008646 maintenance_queue.go:205 Task queued: t1 (balance) volume 100 on server1, priority 0, reason: --- PASS: TestAddTask_RejectsWhenVolumeHasRunningTask (0.00s) === RUN TestPendingOperations_ConflictDetection --- PASS: TestPendingOperations_ConflictDetection (0.00s) === RUN TestPendingOperations_CapacityProjection --- PASS: TestPendingOperations_CapacityProjection (0.00s) === RUN TestPendingOperations_VolumeFiltering --- PASS: TestPendingOperations_VolumeFiltering (0.00s) === RUN TestPendingOperations_OperationLifecycle --- PASS: TestPendingOperations_OperationLifecycle (0.00s) === RUN TestPendingOperations_StaleCleanup W0720 21:29:49.008707 pending_operations.go:265 Removed stale pending operation: volume 301, task task-stale, age 24h0m0.000001543s --- PASS: TestPendingOperations_StaleCleanup (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/maintenance 0.054s === RUN TestDecorateClusterContext_NonS3LifecycleIsPassThrough --- PASS: TestDecorateClusterContext_NonS3LifecycleIsPassThrough (0.00s) === RUN TestDecorateClusterContext_RpsZeroSkipsAllocation --- PASS: TestDecorateClusterContext_RpsZeroSkipsAllocation (0.00s) === RUN TestDecorateClusterContext_NoExecutorsSkipsAllocation --- PASS: TestDecorateClusterContext_NoExecutorsSkipsAllocation (0.00s) === RUN TestDecorateClusterContext_SingletonJobGetsFullBudget --- PASS: TestDecorateClusterContext_SingletonJobGetsFullBudget (0.00s) === RUN TestDecorateClusterContext_SharedEvenlyWhenParallelLimited --- PASS: TestDecorateClusterContext_SharedEvenlyWhenParallelLimited (0.00s) === RUN TestDecorateClusterContext_MaxJobsExceedsExecutors --- PASS: TestDecorateClusterContext_MaxJobsExceedsExecutors (0.00s) === RUN TestDecorateClusterContext_BurstSharedWhenParallel --- PASS: TestDecorateClusterContext_BurstSharedWhenParallel (0.00s) === RUN TestDecorateClusterContext_BurstZeroOmitsKey --- PASS: TestDecorateClusterContext_BurstZeroOmitsKey (0.00s) === RUN TestDecorateClusterContext_BurstFloorIsOneWhenDividesBelowOne --- PASS: TestDecorateClusterContext_BurstFloorIsOneWhenDividesBelowOne (0.00s) === RUN TestDecorateClusterContext_DoesNotMutateInput --- PASS: TestDecorateClusterContext_DoesNotMutateInput (0.00s) === RUN TestDecorateClusterContext_NilInputPassesThrough --- PASS: TestDecorateClusterContext_NilInputPassesThrough (0.00s) === RUN TestConfigStoreDescriptorRoundTrip === PAUSE TestConfigStoreDescriptorRoundTrip === RUN TestConfigStoreRunHistoryRetention === PAUSE TestConfigStoreRunHistoryRetention === RUN TestConfigStoreListJobTypes === PAUSE TestConfigStoreListJobTypes === RUN TestConfigStoreMonitorStateRoundTrip === PAUSE TestConfigStoreMonitorStateRoundTrip === RUN TestConfigStoreSaveJobTypeConfigIfNotExists === PAUSE TestConfigStoreSaveJobTypeConfigIfNotExists === RUN TestConfigStoreJobDetailRoundTrip === PAUSE TestConfigStoreJobDetailRoundTrip === RUN TestConfigStoreDeleteJobDetail === PAUSE TestConfigStoreDeleteJobDetail === RUN TestRunDetectionSendsCancelOnContextDone === PAUSE TestRunDetectionSendsCancelOnContextDone === RUN TestExecuteJobSendsCancelOnContextDone === PAUSE TestExecuteJobSendsCancelOnContextDone === RUN TestAdminScriptExecutionBlocksOtherDetection --- PASS: TestAdminScriptExecutionBlocksOtherDetection (0.10s) === RUN TestAdminScriptExecutionBlocksOtherExecution --- PASS: TestAdminScriptExecutionBlocksOtherExecution (0.10s) === RUN TestEnsureJobTypeConfigFromDescriptorBootstrapsDefaults === PAUSE TestEnsureJobTypeConfigFromDescriptorBootstrapsDefaults === RUN TestEnsureJobTypeConfigFromDescriptorDoesNotOverwriteExisting === PAUSE TestEnsureJobTypeConfigFromDescriptorDoesNotOverwriteExisting === RUN TestRunDetectionIncludesLatestSuccessfulRun --- PASS: TestRunDetectionIncludesLatestSuccessfulRun (0.00s) === RUN TestRunDetectionOmitsLastSuccessfulRunWhenNoSuccessHistory --- PASS: TestRunDetectionOmitsLastSuccessfulRunWhenNoSuccessHistory (0.00s) === RUN TestRunDetectionWithReportCapturesDetectionActivities --- PASS: TestRunDetectionWithReportCapturesDetectionActivities (0.01s) === RUN TestRunDetectionAdminScriptUsesLastCompletedRun --- PASS: TestRunDetectionAdminScriptUsesLastCompletedRun (0.00s) === RUN TestPluginLoadsPersistedMonitorStateOnStart === PAUSE TestPluginLoadsPersistedMonitorStateOnStart === RUN TestPluginPersistsMonitorStateAfterJobUpdates === PAUSE TestPluginPersistsMonitorStateAfterJobUpdates === RUN TestTrackExecutionQueuedMarksPendingState === PAUSE TestTrackExecutionQueuedMarksPendingState === RUN TestHandleJobProgressUpdateCarriesWorkerIDInActivities === PAUSE TestHandleJobProgressUpdateCarriesWorkerIDInActivities === RUN TestHandleJobProgressUpdateWithoutJobIDTracksDetectionActivities === PAUSE TestHandleJobProgressUpdateWithoutJobIDTracksDetectionActivities === RUN TestHandleJobCompletedCarriesWorkerIDInActivitiesAndRunHistory === PAUSE TestHandleJobCompletedCarriesWorkerIDInActivitiesAndRunHistory === RUN TestTrackExecutionStartStoresJobPayloadDetails === PAUSE TestTrackExecutionStartStoresJobPayloadDetails === RUN TestTrackExecutionStartStoresErasureCodingExecutionPlan === PAUSE TestTrackExecutionStartStoresErasureCodingExecutionPlan === RUN TestBuildJobDetailIncludesActivitiesAndRunRecord === PAUSE TestBuildJobDetailIncludesActivitiesAndRunRecord === RUN TestBuildJobDetailLoadsFromDiskWhenMemoryCleared === PAUSE TestBuildJobDetailLoadsFromDiskWhenMemoryCleared === RUN TestLoadSchedulerPolicyUsesAdminConfig === PAUSE TestLoadSchedulerPolicyUsesAdminConfig === RUN TestLoadSchedulerPolicyUsesDescriptorDefaultsWhenConfigMissing === PAUSE TestLoadSchedulerPolicyUsesDescriptorDefaultsWhenConfigMissing === RUN TestReserveScheduledExecutorRespectsPerWorkerLimit === PAUSE TestReserveScheduledExecutorRespectsPerWorkerLimit === RUN TestFilterScheduledProposalsDedupe === PAUSE TestFilterScheduledProposalsDedupe === RUN TestBuildScheduledJobSpecDoesNotReuseProposalID === PAUSE TestBuildScheduledJobSpecDoesNotReuseProposalID === RUN TestFilterProposalsWithActiveJobs === PAUSE TestFilterProposalsWithActiveJobs === RUN TestReserveScheduledExecutorTimesOutWhenNoExecutor === PAUSE TestReserveScheduledExecutorTimesOutWhenNoExecutor === RUN TestReserveScheduledExecutorWaitsForWorkerCapacity === PAUSE TestReserveScheduledExecutorWaitsForWorkerCapacity === RUN TestShouldSkipDetectionForWaitingJobs === PAUSE TestShouldSkipDetectionForWaitingJobs === RUN TestWaitingBacklogThresholdHonorsMaxResultsCap === PAUSE TestWaitingBacklogThresholdHonorsMaxResultsCap === RUN TestListSchedulerStatesIncludesPolicyAndState === PAUSE TestListSchedulerStatesIncludesPolicyAndState === RUN TestListSchedulerStatesShowsDisabledWhenNoPolicy === PAUSE TestListSchedulerStatesShowsDisabledWhenNoPolicy === RUN TestPickDetectorPrefersLeasedWorker === PAUSE TestPickDetectorPrefersLeasedWorker === RUN TestPickDetectorReassignsWhenLeaseIsStale === PAUSE TestPickDetectorReassignsWhenLeaseIsStale === RUN TestRunLaneSchedulerIterationLockBehavior === PAUSE TestRunLaneSchedulerIterationLockBehavior === RUN TestDispatchScheduledProposalsLocksPerJob === PAUSE TestDispatchScheduledProposalsLocksPerJob === RUN TestPruneSchedulerState_DefaultLaneKeepsForeignLanesAndPrunesOwnStale --- PASS: TestPruneSchedulerState_DefaultLaneKeepsForeignLanesAndPrunesOwnStale (0.00s) === RUN TestPruneSchedulerState_LifecycleLaneLeavesDefaultLane --- PASS: TestPruneSchedulerState_LifecycleLaneLeavesDefaultLane (0.00s) === RUN TestPruneDetectorLeases_IsLaneScoped --- PASS: TestPruneDetectorLeases_IsLaneScoped (0.00s) === RUN TestLaneStatus_LifecycleNextDetectionSurvivesDefaultLanePrune --- PASS: TestLaneStatus_LifecycleNextDetectionSurvivesDefaultLanePrune (0.00s) === RUN TestRegistryPickDetectorPrefersMoreFreeSlots === PAUSE TestRegistryPickDetectorPrefersMoreFreeSlots === RUN TestRegistryPickExecutorAllowsSameWorker === PAUSE TestRegistryPickExecutorAllowsSameWorker === RUN TestRegistryDetectableJobTypes === PAUSE TestRegistryDetectableJobTypes === RUN TestRegistryJobTypes === PAUSE TestRegistryJobTypes === RUN TestRegistryListExecutorsSortedBySlots === PAUSE TestRegistryListExecutorsSortedBySlots === RUN TestRegistryPickExecutorRoundRobinForTopTie === PAUSE TestRegistryPickExecutorRoundRobinForTopTie === RUN TestRegistryListExecutorsRoundRobinForTopTie === PAUSE TestRegistryListExecutorsRoundRobinForTopTie === RUN TestRegistrySkipsStaleWorkersForSelectionAndListing === PAUSE TestRegistrySkipsStaleWorkersForSelectionAndListing === RUN TestRegistryReturnsNoDetectorWhenAllWorkersStale === PAUSE TestRegistryReturnsNoDetectorWhenAllWorkersStale === RUN TestRegistryHasCapableWorkerWithDetectCapability === PAUSE TestRegistryHasCapableWorkerWithDetectCapability === RUN TestRegistryHasCapableWorkerWithExecuteCapability === PAUSE TestRegistryHasCapableWorkerWithExecuteCapability === RUN TestRegistryHasCapableWorkerReturnsFalseWhenNoWorkers === PAUSE TestRegistryHasCapableWorkerReturnsFalseWhenNoWorkers === RUN TestRegistryHasCapableWorkerSkipsStaleWorkers === PAUSE TestRegistryHasCapableWorkerSkipsStaleWorkers === RUN TestRegistryHasCapableWorkerIgnoresNilCapability === PAUSE TestRegistryHasCapableWorkerIgnoresNilCapability === RUN TestJobTypeLaneMapCoversKnownTypes --- PASS: TestJobTypeLaneMapCoversKnownTypes (0.00s) === RUN TestJobTypeLaneFallsBackToDefault --- PASS: TestJobTypeLaneFallsBackToDefault (0.00s) === RUN TestAllLanesHaveIdleSleep --- PASS: TestAllLanesHaveIdleSleep (0.00s) === RUN TestLaneRequiresLock === RUN TestLaneRequiresLock/Default === RUN TestLaneRequiresLock/Iceberg === RUN TestLaneRequiresLock/Lifecycle === RUN TestLaneRequiresLock/Unknown --- PASS: TestLaneRequiresLock (0.00s) --- PASS: TestLaneRequiresLock/Default (0.00s) --- PASS: TestLaneRequiresLock/Iceberg (0.00s) --- PASS: TestLaneRequiresLock/Lifecycle (0.00s) --- PASS: TestLaneRequiresLock/Unknown (0.00s) === RUN TestKnownJobTypesInMap --- PASS: TestKnownJobTypesInMap (0.00s) === RUN TestGetSchedulerStatusIncludesInProcessJobs --- PASS: TestGetSchedulerStatusIncludesInProcessJobs (0.00s) === RUN TestGetSchedulerStatusIncludesLastDetectionCount --- PASS: TestGetSchedulerStatusIncludesLastDetectionCount (0.00s) === RUN TestGetLaneSchedulerStatusShowsActiveConcurrentLaneWork --- PASS: TestGetLaneSchedulerStatusShowsActiveConcurrentLaneWork (0.00s) === CONT TestConfigStoreDescriptorRoundTrip --- PASS: TestConfigStoreDescriptorRoundTrip (0.00s) === CONT TestRegistryHasCapableWorkerIgnoresNilCapability --- PASS: TestRegistryHasCapableWorkerIgnoresNilCapability (0.00s) === CONT TestRegistryHasCapableWorkerSkipsStaleWorkers --- PASS: TestRegistryHasCapableWorkerSkipsStaleWorkers (0.00s) === CONT TestRegistryHasCapableWorkerReturnsFalseWhenNoWorkers --- PASS: TestRegistryHasCapableWorkerReturnsFalseWhenNoWorkers (0.00s) === CONT TestRegistryHasCapableWorkerWithExecuteCapability --- PASS: TestRegistryHasCapableWorkerWithExecuteCapability (0.00s) === CONT TestRegistryHasCapableWorkerWithDetectCapability --- PASS: TestRegistryHasCapableWorkerWithDetectCapability (0.00s) === CONT TestRegistryReturnsNoDetectorWhenAllWorkersStale --- PASS: TestRegistryReturnsNoDetectorWhenAllWorkersStale (0.00s) === CONT TestRegistrySkipsStaleWorkersForSelectionAndListing --- PASS: TestRegistrySkipsStaleWorkersForSelectionAndListing (0.00s) === CONT TestRegistryListExecutorsRoundRobinForTopTie --- PASS: TestRegistryListExecutorsRoundRobinForTopTie (0.00s) === CONT TestRegistryPickExecutorRoundRobinForTopTie --- PASS: TestRegistryPickExecutorRoundRobinForTopTie (0.00s) === CONT TestRegistryListExecutorsSortedBySlots --- PASS: TestRegistryListExecutorsSortedBySlots (0.00s) === CONT TestRegistryJobTypes --- PASS: TestRegistryJobTypes (0.00s) === CONT TestRegistryDetectableJobTypes --- PASS: TestRegistryDetectableJobTypes (0.00s) === CONT TestRegistryPickExecutorAllowsSameWorker --- PASS: TestRegistryPickExecutorAllowsSameWorker (0.00s) === CONT TestRegistryPickDetectorPrefersMoreFreeSlots --- PASS: TestRegistryPickDetectorPrefersMoreFreeSlots (0.00s) === CONT TestDispatchScheduledProposalsLocksPerJob === RUN TestDispatchScheduledProposalsLocksPerJob/DefaultLaneLocksEachJob === PAUSE TestDispatchScheduledProposalsLocksPerJob/DefaultLaneLocksEachJob === RUN TestDispatchScheduledProposalsLocksPerJob/LifecycleLaneNeedsNoLock === PAUSE TestDispatchScheduledProposalsLocksPerJob/LifecycleLaneNeedsNoLock === CONT TestDispatchScheduledProposalsLocksPerJob/DefaultLaneLocksEachJob === CONT TestRunLaneSchedulerIterationLockBehavior === RUN TestRunLaneSchedulerIterationLockBehavior/Default === PAUSE TestRunLaneSchedulerIterationLockBehavior/Default === RUN TestRunLaneSchedulerIterationLockBehavior/Iceberg === PAUSE TestRunLaneSchedulerIterationLockBehavior/Iceberg === RUN TestRunLaneSchedulerIterationLockBehavior/Lifecycle === PAUSE TestRunLaneSchedulerIterationLockBehavior/Lifecycle === CONT TestRunLaneSchedulerIterationLockBehavior/Default === CONT TestPickDetectorReassignsWhenLeaseIsStale --- PASS: TestPickDetectorReassignsWhenLeaseIsStale (0.00s) === CONT TestPickDetectorPrefersLeasedWorker --- PASS: TestPickDetectorPrefersLeasedWorker (0.00s) === CONT TestListSchedulerStatesShowsDisabledWhenNoPolicy --- PASS: TestListSchedulerStatesShowsDisabledWhenNoPolicy (0.00s) === CONT TestListSchedulerStatesIncludesPolicyAndState --- PASS: TestListSchedulerStatesIncludesPolicyAndState (0.00s) === CONT TestWaitingBacklogThresholdHonorsMaxResultsCap --- PASS: TestWaitingBacklogThresholdHonorsMaxResultsCap (0.00s) === CONT TestShouldSkipDetectionForWaitingJobs --- PASS: TestShouldSkipDetectionForWaitingJobs (0.00s) === CONT TestReserveScheduledExecutorWaitsForWorkerCapacity === CONT TestReserveScheduledExecutorTimesOutWhenNoExecutor --- PASS: TestReserveScheduledExecutorTimesOutWhenNoExecutor (0.00s) === CONT TestFilterProposalsWithActiveJobs --- PASS: TestFilterProposalsWithActiveJobs (0.00s) === CONT TestBuildScheduledJobSpecDoesNotReuseProposalID --- PASS: TestBuildScheduledJobSpecDoesNotReuseProposalID (0.00s) === CONT TestFilterScheduledProposalsDedupe --- PASS: TestFilterScheduledProposalsDedupe (0.00s) === CONT TestReserveScheduledExecutorRespectsPerWorkerLimit --- PASS: TestReserveScheduledExecutorRespectsPerWorkerLimit (0.00s) === CONT TestLoadSchedulerPolicyUsesDescriptorDefaultsWhenConfigMissing --- PASS: TestLoadSchedulerPolicyUsesDescriptorDefaultsWhenConfigMissing (0.00s) === CONT TestLoadSchedulerPolicyUsesAdminConfig --- PASS: TestLoadSchedulerPolicyUsesAdminConfig (0.00s) === CONT TestBuildJobDetailLoadsFromDiskWhenMemoryCleared === CONT TestConfigStoreDeleteJobDetail === CONT TestHandleJobProgressUpdateCarriesWorkerIDInActivities --- PASS: TestHandleJobProgressUpdateCarriesWorkerIDInActivities (0.00s) === CONT TestBuildJobDetailIncludesActivitiesAndRunRecord --- PASS: TestConfigStoreDeleteJobDetail (0.00s) === CONT TestTrackExecutionQueuedMarksPendingState --- PASS: TestTrackExecutionQueuedMarksPendingState (0.00s) === CONT TestPluginPersistsMonitorStateAfterJobUpdates --- PASS: TestPluginPersistsMonitorStateAfterJobUpdates (0.00s) === CONT TestPluginLoadsPersistedMonitorStateOnStart --- PASS: TestBuildJobDetailIncludesActivitiesAndRunRecord (0.00s) === CONT TestTrackExecutionStartStoresErasureCodingExecutionPlan === CONT TestTrackExecutionStartStoresJobPayloadDetails === CONT TestHandleJobProgressUpdateWithoutJobIDTracksDetectionActivities === CONT TestHandleJobCompletedCarriesWorkerIDInActivitiesAndRunHistory === CONT TestDispatchScheduledProposalsLocksPerJob/LifecycleLaneNeedsNoLock === CONT TestRunLaneSchedulerIterationLockBehavior/Lifecycle === CONT TestRunLaneSchedulerIterationLockBehavior/Iceberg --- PASS: TestBuildJobDetailLoadsFromDiskWhenMemoryCleared (0.01s) --- PASS: TestPluginLoadsPersistedMonitorStateOnStart (0.01s) --- PASS: TestHandleJobProgressUpdateWithoutJobIDTracksDetectionActivities (0.01s) --- PASS: TestHandleJobCompletedCarriesWorkerIDInActivitiesAndRunHistory (0.01s) === CONT TestEnsureJobTypeConfigFromDescriptorDoesNotOverwriteExisting === CONT TestEnsureJobTypeConfigFromDescriptorBootstrapsDefaults === CONT TestExecuteJobSendsCancelOnContextDone === CONT TestRunDetectionSendsCancelOnContextDone === CONT TestConfigStoreJobDetailRoundTrip === CONT TestConfigStoreSaveJobTypeConfigIfNotExists === RUN TestConfigStoreSaveJobTypeConfigIfNotExists/in-memory === PAUSE TestConfigStoreSaveJobTypeConfigIfNotExists/in-memory === RUN TestConfigStoreSaveJobTypeConfigIfNotExists/on-disk === PAUSE TestConfigStoreSaveJobTypeConfigIfNotExists/on-disk === CONT TestConfigStoreSaveJobTypeConfigIfNotExists/in-memory === CONT TestConfigStoreMonitorStateRoundTrip === CONT TestConfigStoreListJobTypes === CONT TestConfigStoreRunHistoryRetention === CONT TestConfigStoreSaveJobTypeConfigIfNotExists/on-disk --- PASS: TestTrackExecutionStartStoresJobPayloadDetails (0.01s) --- PASS: TestTrackExecutionStartStoresErasureCodingExecutionPlan (0.01s) --- PASS: TestDispatchScheduledProposalsLocksPerJob (0.00s) --- PASS: TestDispatchScheduledProposalsLocksPerJob/DefaultLaneLocksEachJob (0.01s) --- PASS: TestDispatchScheduledProposalsLocksPerJob/LifecycleLaneNeedsNoLock (0.01s) --- PASS: TestRunLaneSchedulerIterationLockBehavior (0.00s) --- PASS: TestRunLaneSchedulerIterationLockBehavior/Default (0.00s) --- PASS: TestRunLaneSchedulerIterationLockBehavior/Lifecycle (0.01s) --- PASS: TestRunLaneSchedulerIterationLockBehavior/Iceberg (0.01s) --- PASS: TestEnsureJobTypeConfigFromDescriptorDoesNotOverwriteExisting (0.00s) --- PASS: TestEnsureJobTypeConfigFromDescriptorBootstrapsDefaults (0.00s) --- PASS: TestExecuteJobSendsCancelOnContextDone (0.00s) --- PASS: TestRunDetectionSendsCancelOnContextDone (0.00s) --- PASS: TestConfigStoreMonitorStateRoundTrip (0.00s) --- PASS: TestConfigStoreListJobTypes (0.00s) --- PASS: TestConfigStoreJobDetailRoundTrip (0.00s) --- PASS: TestConfigStoreSaveJobTypeConfigIfNotExists (0.00s) --- PASS: TestConfigStoreSaveJobTypeConfigIfNotExists/in-memory (0.00s) --- PASS: TestConfigStoreSaveJobTypeConfigIfNotExists/on-disk (0.00s) --- PASS: TestConfigStoreRunHistoryRetention (0.00s) --- PASS: TestReserveScheduledExecutorWaitsForWorkerCapacity (0.03s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/plugin 0.295s === RUN TestActiveTopologyBasicOperations --- PASS: TestActiveTopologyBasicOperations (0.00s) === RUN TestActiveTopologyUpdate --- PASS: TestActiveTopologyUpdate (0.00s) === RUN TestTaskLifecycle --- PASS: TestTaskLifecycle (0.00s) === RUN TestTaskDetectionScenarios === RUN TestTaskDetectionScenarios/Empty_cluster_-_no_tasks_needed === RUN TestTaskDetectionScenarios/Unbalanced_cluster_-_balance_task_needed === RUN TestTaskDetectionScenarios/High_garbage_ratio_-_vacuum_task_needed === RUN TestTaskDetectionScenarios/Large_volumes_-_EC_task_needed === RUN TestTaskDetectionScenarios/Recent_tasks_-_no_immediate_re-detection --- PASS: TestTaskDetectionScenarios (0.00s) --- PASS: TestTaskDetectionScenarios/Empty_cluster_-_no_tasks_needed (0.00s) --- PASS: TestTaskDetectionScenarios/Unbalanced_cluster_-_balance_task_needed (0.00s) --- PASS: TestTaskDetectionScenarios/High_garbage_ratio_-_vacuum_task_needed (0.00s) --- PASS: TestTaskDetectionScenarios/Large_volumes_-_EC_task_needed (0.00s) --- PASS: TestTaskDetectionScenarios/Recent_tasks_-_no_immediate_re-detection (0.00s) === RUN TestTargetSelectionScenarios === RUN TestTargetSelectionScenarios/Balance_task_-_find_least_loaded_disk === RUN TestTargetSelectionScenarios/EC_task_-_find_multiple_available_disks === RUN TestTargetSelectionScenarios/Vacuum_task_-_cross-type_tasks_do_not_block_per_disk --- PASS: TestTargetSelectionScenarios (0.00s) --- PASS: TestTargetSelectionScenarios/Balance_task_-_find_least_loaded_disk (0.00s) --- PASS: TestTargetSelectionScenarios/EC_task_-_find_multiple_available_disks (0.00s) --- PASS: TestTargetSelectionScenarios/Vacuum_task_-_cross-type_tasks_do_not_block_per_disk (0.00s) === RUN TestDiskLoadCalculation --- PASS: TestDiskLoadCalculation (0.00s) === RUN TestCrossTypeTasksDoNotBlockPerDisk --- PASS: TestCrossTypeTasksDoNotBlockPerDisk (0.00s) === RUN TestECPlanningNotBlockedByUnrelatedBalance --- PASS: TestECPlanningNotBlockedByUnrelatedBalance (0.00s) === RUN TestECPlannerSeesEachPhysicalDisk --- PASS: TestECPlannerSeesEachPhysicalDisk (0.00s) === RUN TestVolumeIndexResolvesPhysicalDiskZero --- PASS: TestVolumeIndexResolvesPhysicalDiskZero (0.00s) === RUN TestECShardIndexResolvesPhysicalDiskZero --- PASS: TestECShardIndexResolvesPhysicalDiskZero (0.00s) === RUN TestPublicInterfaces --- PASS: TestPublicInterfaces (0.00s) === RUN TestDestinationPlanning === RUN TestDestinationPlanning/GetAvailableDisks_functionality === RUN TestDestinationPlanning/Topology_provides_planning_information --- PASS: TestDestinationPlanning (0.00s) --- PASS: TestDestinationPlanning/GetAvailableDisks_functionality (0.00s) --- PASS: TestDestinationPlanning/Topology_provides_planning_information (0.00s) === RUN TestGetDisksWithEffectiveCapacityNotCappedAtTenByLoad === PAUSE TestGetDisksWithEffectiveCapacityNotCappedAtTenByLoad === RUN TestCountTopologyResources_multiDiskPerNode --- PASS: TestCountTopologyResources_multiDiskPerNode (0.00s) === RUN TestStorageSlotChangeArithmetic --- PASS: TestStorageSlotChangeArithmetic (0.00s) === RUN TestStorageSlotChange --- PASS: TestStorageSlotChange (0.00s) === RUN TestStorageSlotChangeCapacityCalculation --- PASS: TestStorageSlotChangeCapacityCalculation (0.00s) === RUN TestGetDisksWithEffectiveCapacity_UnknownEmptyDiskFallback --- PASS: TestGetDisksWithEffectiveCapacity_UnknownEmptyDiskFallback (0.00s) === RUN TestECMultipleTargets storage_slot_test.go:512: EC operation distributed 14 shards across 3 targets storage_slot_test.go:513: Capacity impacts: EC source reserves with zero impact, Targets minimal (shards < 10) --- PASS: TestECMultipleTargets (0.00s) === RUN TestCapacityReservationCycle storage_slot_test.go:611: Capacity lifecycle with StorageSlotChange: Pending -> Assigned -> Released -> Applied storage_slot_test.go:612: Source: 10 -> 11 -> 11 -> 10 -> 11 (freed by pending balance, then applied) storage_slot_test.go:613: Target: 10 -> 9 -> 9 -> 10 -> 9 (reserved by pending, then applied) --- PASS: TestCapacityReservationCycle (0.00s) === RUN TestReplicatedVolumeECOperations storage_slot_test.go:766: Replicated volume EC operation: 3 source replicas, 14 EC shards distributed across 3 destinations storage_slot_test.go:768: Each source replica reserves with zero capacity impact, destinations receive EC shards --- PASS: TestReplicatedVolumeECOperations (0.00s) === RUN TestECWithOldShardCleanup storage_slot_test.go:919: EC operation with cleanup: 2 volume replicas + 2 old EC shard locations → 14 new EC shards storage_slot_test.go:921: Volume sources have zero impact, old EC shard sources free capacity, new destinations consume shard slots --- PASS: TestECWithOldShardCleanup (0.00s) === RUN TestDetailedCapacityCalculations storage_slot_test.go:997: Detailed capacity calculation: VolumeSlots=80, ShardSlots=-5 storage_slot_test.go:999: Capacity impact: VolumeSlots=0, ShardSlots=5 storage_slot_test.go:1001: Simple capacity (backward compatible): 80 --- PASS: TestDetailedCapacityCalculations (0.00s) === RUN TestStorageSlotChangeConversions storage_slot_test.go:1052: Conversion tests passed: 10 shards = 1 volume slot storage_slot_test.go:1053: Mixed capacity (2 volumes + 15 shards) = 3 equivalent volume slots storage_slot_test.go:1055: Available capacity (10 volumes) = 100 total shard slots storage_slot_test.go:1057: NOTE: This test adapts automatically to erasure_coding.DataShardsCount = 10 --- PASS: TestStorageSlotChangeConversions (0.00s) === CONT TestGetDisksWithEffectiveCapacityNotCappedAtTenByLoad --- PASS: TestGetDisksWithEffectiveCapacityNotCappedAtTenByLoad (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/topology 0.051s === RUN TestFileBrowserPathURLEscapesQueryPath --- PASS: TestFileBrowserPathURLEscapesQueryPath (0.00s) === RUN TestFileBrowserPageURLEscapesCursor --- PASS: TestFileBrowserPageURLEscapesCursor (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/admin/view/app 0.125s ? github.com/seaweedfs/seaweedfs/weed/admin/view/components [no test files] ? github.com/seaweedfs/seaweedfs/weed/admin/view/layout [no test files] === RUN TestConcurrentAddRemoveNodes --- PASS: TestConcurrentAddRemoveNodes (0.00s) === RUN TestIsKnownNode --- PASS: TestIsKnownNode (0.00s) === RUN TestLockClientHostMatchesFilerRing --- PASS: TestLockClientHostMatchesFilerRing (0.00s) === RUN TestLockClientHostFallsBackToSeed --- PASS: TestLockClientHostFallsBackToSeed (0.00s) === RUN TestLockClientSetRingVersionGuard --- PASS: TestLockClientSetRingVersionGuard (0.00s) === RUN TestLockClientPrimaryForKey --- PASS: TestLockClientPrimaryForKey (0.00s) === RUN TestLockClientPriorOwnerForKey --- PASS: TestLockClientPriorOwnerForKey (0.00s) === RUN TestLockClientPriorOwnerForKeyExpires --- PASS: TestLockClientPriorOwnerForKeyExpires (0.04s) === RUN TestLockRingManager_BatchesRapidChanges I0720 21:29:49.167766 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989167725372: [filer1:8888 filer2:8888 filer3:8888] --- PASS: TestLockRingManager_BatchesRapidChanges (0.20s) === RUN TestLockRingManager_DropAndJoinBatched I0720 21:29:49.270702 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989270692713: [filer1:8888 filer2:8888 filer3:8888] I0720 21:29:49.373963 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989373931070: [filer2:8888 filer4:8888 filer1:8888] --- PASS: TestLockRingManager_DropAndJoinBatched (0.20s) === RUN TestLockRingManager_VersionIncrements I0720 21:29:49.527306 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989527277565: [filer1:8888] I0720 21:29:49.630630 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989630600474: [filer1:8888 filer2:8888] --- PASS: TestLockRingManager_VersionIncrements (0.20s) === RUN TestLockRingManager_FlushPending I0720 21:29:49.677558 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989677528008: [filer1:8888 filer2:8888] --- PASS: TestLockRingManager_FlushPending (0.00s) === RUN TestLockRingManager_MultipleGroups I0720 21:29:49.730644 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "group1" version 1784582989730636416: [filer1:8888] I0720 21:29:49.730725 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "group2" version 1784582989730598423: [filer2:8888] --- PASS: TestLockRingManager_MultipleGroups (0.10s) === RUN TestLockRingManager_GetServers --- PASS: TestLockRingManager_GetServers (0.00s) === RUN TestLockRingManager_NoBroadcastWithoutFn I0720 21:29:49.790635 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989790599572: [filer1:8888] --- PASS: TestLockRingManager_NoBroadcastWithoutFn (0.06s) === RUN TestLockRingManager_GetLastUpdateReturnsBroadcastState I0720 21:29:49.834162 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "default" version 1784582989834149356: [filer1:8888 filer2:8888] --- PASS: TestLockRingManager_GetLastUpdateReturnsBroadcastState (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/cluster 0.885s === RUN TestDLM_PrimaryCrash_BackupPromotes --- PASS: TestDLM_PrimaryCrash_BackupPromotes (0.02s) === RUN TestDLM_BackupCrash_PrimaryContinues --- PASS: TestDLM_BackupCrash_PrimaryContinues (0.03s) === RUN TestDLM_BothPrimaryAndBackupCrash --- PASS: TestDLM_BothPrimaryAndBackupCrash (0.03s) === RUN TestDLM_RollingRestart distributed_lock_manager_test.go:337: Locks survived rolling restart: 2 / 5 --- PASS: TestDLM_RollingRestart (0.13s) === RUN TestDLM_GenerationIncrementsOnNewAcquisition --- PASS: TestDLM_GenerationIncrementsOnNewAcquisition (3.05s) === RUN TestDLM_ReplicationFailure_PrimaryStillWorks --- PASS: TestDLM_ReplicationFailure_PrimaryStillWorks (0.02s) === RUN TestDLM_UnlockReplicatesToBackup --- PASS: TestDLM_UnlockReplicatesToBackup (0.03s) === RUN TestDLM_TopologyChange_LockSurvivesServerAddition --- PASS: TestDLM_TopologyChange_LockSurvivesServerAddition (0.04s) === RUN TestDLM_ConsistentHashing_MinimalDisruption distributed_lock_manager_test.go:477: Lock distribution before: map[filer1:8888:7 filer2:8888:22 filer3:8888:21] --- PASS: TestDLM_ConsistentHashing_MinimalDisruption (0.52s) === RUN TestDLM_NodeDropAndJoin_OwnershipDisruption distributed_lock_manager_test.go:593: Locks disrupted from departed filer3: 44 / 100 distributed_lock_manager_test.go:594: Locks disrupted from surviving filer1/filer2: 14 / 100 distributed_lock_manager_test.go:595: Locks now on new filer4: 24 / 100 --- PASS: TestDLM_NodeDropAndJoin_OwnershipDisruption (1.12s) === RUN TestDLM_RenewalDuringTransferWindow distributed_lock_manager_test.go:671: Key transfer-window-lock-0: primary changed from filer3:8888 to filer4:8888, but renewal on old primary succeeded --- PASS: TestDLM_RenewalDuringTransferWindow (0.03s) === RUN TestDLM_StaleReplicationRejected --- PASS: TestDLM_StaleReplicationRejected (0.00s) === RUN TestHashRing_BasicOperations --- PASS: TestHashRing_BasicOperations (0.00s) === RUN TestHashRing_DuplicateAddRemove --- PASS: TestHashRing_DuplicateAddRemove (0.00s) === RUN TestHashRing_SetServers --- PASS: TestHashRing_SetServers (0.00s) === RUN TestHashRing_ConsistencyOnRemoval hash_ring_test.go:101: Keys that moved: 380 / 1000 --- PASS: TestHashRing_ConsistencyOnRemoval (0.00s) === RUN TestHashRing_BackupIsSuccessor --- PASS: TestHashRing_BackupIsSuccessor (0.04s) === RUN TestHashRing_Distribution hash_ring_test.go:150: Server filer1:8888: 677 keys (32.3% deviation) hash_ring_test.go:150: Server filer3:8888: 1250 keys (25.0% deviation) hash_ring_test.go:150: Server filer2:8888: 1073 keys (7.3% deviation) --- PASS: TestHashRing_Distribution (0.00s) === RUN TestHashRing_GetPrimary --- PASS: TestHashRing_GetPrimary (0.00s) === RUN TestHashRing_OrderIndependent --- PASS: TestHashRing_OrderIndependent (0.01s) === RUN TestLockManager_GetLockOwnerIgnoresExpiredLock --- PASS: TestLockManager_GetLockOwnerIgnoresExpiredLock (0.00s) === RUN TestLockManager_InsertLockRejectsStaleTransfer --- PASS: TestLockManager_InsertLockRejectsStaleTransfer (0.00s) === RUN TestLockManager_InsertLockAdvancesGenerationCounter --- PASS: TestLockManager_InsertLockAdvancesGenerationCounter (0.00s) === RUN TestLockManager_InsertBackupLockRejectsOlderGeneration --- PASS: TestLockManager_InsertBackupLockRejectsOlderGeneration (0.00s) === RUN TestLockManager_InsertBackupLockKeepsPrimaryRole --- PASS: TestLockManager_InsertBackupLockKeepsPrimaryRole (0.00s) === RUN TestLockManager_RemoveBackupLockRejectsOlderGeneration --- PASS: TestLockManager_RemoveBackupLockRejectsOlderGeneration (0.00s) === RUN TestLockRing_PriorOwner --- PASS: TestLockRing_PriorOwner (5.00s) === RUN TestLockRing_PriorOwnerExpires --- PASS: TestLockRing_PriorOwnerExpires (0.04s) === RUN TestLockRing_SetSnapshot --- PASS: TestLockRing_SetSnapshot (0.22s) === RUN TestLockRing_SnapshotCompaction --- PASS: TestLockRing_SnapshotCompaction (0.23s) === RUN TestLockRing_VersionRejectsStale I0720 21:29:59.560837 lock_ring.go:53 LockRing: rejecting stale update v2 (current v3) --- PASS: TestLockRing_VersionRejectsStale (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/cluster/lock_manager 10.611s ? github.com/seaweedfs/seaweedfs/weed/cluster/maintenance [no test files] === RUN TestIsIgnorable404_WrappedErrNotFound --- PASS: TestIsIgnorable404_WrappedErrNotFound (0.00s) === RUN TestIsIgnorable404_BrokenUnwrapChain --- PASS: TestIsIgnorable404_BrokenUnwrapChain (0.00s) === RUN TestIsIgnorable404_NonIgnorableError --- PASS: TestIsIgnorable404_NonIgnorableError (0.00s) === RUN TestIsIgnorable404_TransientLookupNotSwallowed === RUN TestIsIgnorable404_TransientLookupNotSwallowed/lookup_file_id_race === RUN TestIsIgnorable404_TransientLookupNotSwallowed/volume_id_not_found_race --- PASS: TestIsIgnorable404_TransientLookupNotSwallowed (0.00s) --- PASS: TestIsIgnorable404_TransientLookupNotSwallowed/lookup_file_id_race (0.00s) --- PASS: TestIsIgnorable404_TransientLookupNotSwallowed/volume_id_not_found_race (0.00s) === RUN TestErrorClassifiersNilSafe --- PASS: TestErrorClassifiersNilSafe (0.00s) === RUN TestIsSourceLookupError_NonLookupErrors === RUN TestIsSourceLookupError_NonLookupErrors/genuine_S3_404 === RUN TestIsSourceLookupError_NonLookupErrors/network_error === RUN TestIsSourceLookupError_NonLookupErrors/plain_not_found_without_volume_id --- PASS: TestIsSourceLookupError_NonLookupErrors (0.00s) --- PASS: TestIsSourceLookupError_NonLookupErrors/genuine_S3_404 (0.00s) --- PASS: TestIsSourceLookupError_NonLookupErrors/network_error (0.00s) --- PASS: TestIsSourceLookupError_NonLookupErrors/plain_not_found_without_volume_id (0.00s) === RUN TestEventSupersessionProbe_PathDerivation === RUN TestEventSupersessionProbe_PathDerivation/legacy_event_without_NewParentPath === RUN TestEventSupersessionProbe_PathDerivation/rename_event_with_NewParentPath --- PASS: TestEventSupersessionProbe_PathDerivation (0.00s) --- PASS: TestEventSupersessionProbe_PathDerivation/legacy_event_without_NewParentPath (0.00s) --- PASS: TestEventSupersessionProbe_PathDerivation/rename_event_with_NewParentPath (0.00s) === RUN TestEventSourceSuperseded_Guards --- PASS: TestEventSourceSuperseded_Guards (0.00s) === RUN TestInitialSnapshotTargetKey --- PASS: TestInitialSnapshotTargetKey (0.00s) === RUN TestVersionedFilePathRewrittenForRemote --- PASS: TestVersionedFilePathRewrittenForRemote (0.00s) === RUN TestVersionsDirectoryFilteredByHasData --- PASS: TestVersionsDirectoryFilteredByHasData (0.00s) === RUN TestIsVersionedPath === RUN TestIsVersionedPath/version_file_in_.versions_dir === RUN TestIsVersionedPath/regular_file === RUN TestIsVersionedPath/.versions_directory_entry === RUN TestIsVersionedPath/non-version_file_in_.versions_dir === RUN TestIsVersionedPath/file_in_user_dir_ending_with_.versions_but_no_v__prefix === RUN TestIsVersionedPath/regular_directory === RUN TestIsVersionedPath/file_named_like_.versions_dir === RUN TestIsVersionedPath/.versions_dir_outside_/buckets/ === RUN TestIsVersionedPath/v__file_outside_/buckets/ --- PASS: TestIsVersionedPath (0.00s) --- PASS: TestIsVersionedPath/version_file_in_.versions_dir (0.00s) --- PASS: TestIsVersionedPath/regular_file (0.00s) --- PASS: TestIsVersionedPath/.versions_directory_entry (0.00s) --- PASS: TestIsVersionedPath/non-version_file_in_.versions_dir (0.00s) --- PASS: TestIsVersionedPath/file_in_user_dir_ending_with_.versions_but_no_v__prefix (0.00s) --- PASS: TestIsVersionedPath/regular_directory (0.00s) --- PASS: TestIsVersionedPath/file_named_like_.versions_dir (0.00s) --- PASS: TestIsVersionedPath/.versions_dir_outside_/buckets/ (0.00s) --- PASS: TestIsVersionedPath/v__file_outside_/buckets/ (0.00s) === RUN TestDeleteMarkerDetectedBeforeHasDataFilter --- PASS: TestDeleteMarkerDetectedBeforeHasDataFilter (0.00s) === RUN TestRewriteVersionedSourcePath === RUN TestRewriteVersionedSourcePath/version_file_in_.versions_dir === RUN TestRewriteVersionedSourcePath/regular_file === RUN TestRewriteVersionedSourcePath/version_file_at_bucket_root === RUN TestRewriteVersionedSourcePath/non-bucket_path_not_rewritten === RUN TestRewriteVersionedSourcePath/non-bucket_mount_not_rewritten === RUN TestRewriteVersionedSourcePath/non-version_file_in_.versions_dir === RUN TestRewriteVersionedSourcePath/dir_not_ending_in_.versions --- PASS: TestRewriteVersionedSourcePath (0.00s) --- PASS: TestRewriteVersionedSourcePath/version_file_in_.versions_dir (0.00s) --- PASS: TestRewriteVersionedSourcePath/regular_file (0.00s) --- PASS: TestRewriteVersionedSourcePath/version_file_at_bucket_root (0.00s) --- PASS: TestRewriteVersionedSourcePath/non-bucket_path_not_rewritten (0.00s) --- PASS: TestRewriteVersionedSourcePath/non-bucket_mount_not_rewritten (0.00s) --- PASS: TestRewriteVersionedSourcePath/non-version_file_in_.versions_dir (0.00s) --- PASS: TestRewriteVersionedSourcePath/dir_not_ending_in_.versions (0.00s) === RUN TestPathAncestors --- PASS: TestPathAncestors (0.00s) === RUN TestFileVsFileConflict --- PASS: TestFileVsFileConflict (0.00s) === RUN TestFileUnderActiveDirConflict --- PASS: TestFileUnderActiveDirConflict (0.00s) === RUN TestDirWithActiveFileUnder --- PASS: TestDirWithActiveFileUnder (0.00s) === RUN TestDirVsDirConflict --- PASS: TestDirVsDirConflict (0.00s) === RUN TestRenameConflict --- PASS: TestRenameConflict (0.00s) === RUN TestActiveRenameConflict --- PASS: TestActiveRenameConflict (0.00s) === RUN TestRootDirConflict --- PASS: TestRootDirConflict (0.00s) === RUN TestIndexCleanup --- PASS: TestIndexCleanup (0.00s) === RUN TestWatermarkWithHeap --- PASS: TestWatermarkWithHeap (0.00s) === RUN TestNonBarrierDirUpdateDoesNotBlockDescendants === RUN TestNonBarrierDirUpdateDoesNotBlockDescendants/attribute_update_on_/dir1_does_not_block_file_under_it === RUN TestNonBarrierDirUpdateDoesNotBlockDescendants/barrier_dir_create_at_the_same_path_still_blocks_descendants === RUN TestNonBarrierDirUpdateDoesNotBlockDescendants/barrier_dir_delete_still_waits_for_in-flight_descendants === RUN TestNonBarrierDirUpdateDoesNotBlockDescendants/non-barrier_dir_update_still_keeps_ancestor_barrier_waiting --- PASS: TestNonBarrierDirUpdateDoesNotBlockDescendants (0.00s) --- PASS: TestNonBarrierDirUpdateDoesNotBlockDescendants/attribute_update_on_/dir1_does_not_block_file_under_it (0.00s) --- PASS: TestNonBarrierDirUpdateDoesNotBlockDescendants/barrier_dir_create_at_the_same_path_still_blocks_descendants (0.00s) --- PASS: TestNonBarrierDirUpdateDoesNotBlockDescendants/barrier_dir_delete_still_waits_for_in-flight_descendants (0.00s) --- PASS: TestNonBarrierDirUpdateDoesNotBlockDescendants/non-barrier_dir_update_still_keeps_ancestor_barrier_waiting (0.00s) === RUN TestSamePathBarrierSerialization === RUN TestSamePathBarrierSerialization/barrier_dir_at_p_blocks_same-path_file === RUN TestSamePathBarrierSerialization/barrier_dir_at_p_blocks_another_same-path_barrier_dir === RUN TestSamePathBarrierSerialization/barrier_dir_at_p_blocks_non-barrier_update_at_same_path === RUN TestSamePathBarrierSerialization/file_at_p_blocks_same-path_barrier_dir === RUN TestSamePathBarrierSerialization/non-barrier_update_at_p_blocks_incoming_barrier_dir_at_same_path === RUN TestSamePathBarrierSerialization/non-barrier_update_at_p_does_NOT_block_same-path_non-barrier_update --- PASS: TestSamePathBarrierSerialization (0.00s) --- PASS: TestSamePathBarrierSerialization/barrier_dir_at_p_blocks_same-path_file (0.00s) --- PASS: TestSamePathBarrierSerialization/barrier_dir_at_p_blocks_another_same-path_barrier_dir (0.00s) --- PASS: TestSamePathBarrierSerialization/barrier_dir_at_p_blocks_non-barrier_update_at_same_path (0.00s) --- PASS: TestSamePathBarrierSerialization/file_at_p_blocks_same-path_barrier_dir (0.00s) --- PASS: TestSamePathBarrierSerialization/non-barrier_update_at_p_blocks_incoming_barrier_dir_at_same_path (0.00s) --- PASS: TestSamePathBarrierSerialization/non-barrier_update_at_p_does_NOT_block_same-path_non-barrier_update (0.00s) === RUN TestMetadataProcessorEmptyMarkerKeepsWatermarkStale filer_sync_jobs_test.go:611: marker carried fresh ts 3601000000000 but watermark stayed stale at 1000000000 --- PASS: TestMetadataProcessorEmptyMarkerKeepsWatermarkStale (0.00s) === RUN TestDestKeyPreservesColonForNonLocalSink === RUN TestDestKeyPreservesColonForNonLocalSink/non-incremental === RUN TestDestKeyPreservesColonForNonLocalSink/incremental --- PASS: TestDestKeyPreservesColonForNonLocalSink (0.00s) --- PASS: TestDestKeyPreservesColonForNonLocalSink/non-incremental (0.00s) --- PASS: TestDestKeyPreservesColonForNonLocalSink/incremental (0.00s) === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/equal === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/descendant === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/sibling_prefix === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/root === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/empty --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/equal (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/descendant (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/sibling_prefix (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/root (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/empty (0.00s) === RUN TestMatchesExcludePathUsesDirectoryBoundaries --- PASS: TestMatchesExcludePathUsesDirectoryBoundaries (0.00s) === RUN TestGenProcessFunctionRenameCreatesThenDeletes --- PASS: TestGenProcessFunctionRenameCreatesThenDeletes (0.00s) === RUN TestGenProcessFunctionRenameUsesMoveEntryWhenSupported --- PASS: TestGenProcessFunctionRenameUsesMoveEntryWhenSupported (0.00s) === RUN TestGenProcessFunctionRenameMoverKeepsOldWhenDeletesDisabled --- PASS: TestGenProcessFunctionRenameMoverKeepsOldWhenDeletesDisabled (0.00s) === RUN TestGenProcessFunctionInPlaceUpdateUsesUpdateEntry --- PASS: TestGenProcessFunctionInPlaceUpdateUsesUpdateEntry (0.00s) === RUN TestGenProcessFunctionInPlaceUpdateFallbackDeletesBeforeCreate --- PASS: TestGenProcessFunctionInPlaceUpdateFallbackDeletesBeforeCreate (0.00s) === RUN TestGenProcessFunctionRenameCreateOnlyWhenDeletesDisabled --- PASS: TestGenProcessFunctionRenameCreateOnlyWhenDeletesDisabled (0.00s) === RUN TestGenProcessFunctionRenameToSiblingPrefixBecomesDelete --- PASS: TestGenProcessFunctionRenameToSiblingPrefixBecomesDelete (0.00s) === RUN TestGenProcessFunctionRenameFromExcludedDirBecomesCreate --- PASS: TestGenProcessFunctionRenameFromExcludedDirBecomesCreate (0.00s) === RUN TestStreamDeliveryBottleneck filer_sync_subscription_test.go:131: 1 stream: 2000 events 5.001s 400 events/sec filer_sync_subscription_test.go:136: 10 streams: 2000 events 286ms 6999 events/sec filer_sync_subscription_test.go:140: Speedup: 17.5x (10 parallel streams vs 1 stream) --- PASS: TestStreamDeliveryBottleneck (5.29s) === RUN TestConcurrencyIneffectiveOnStreamBottleneck filer_sync_subscription_test.go:174: concurrency= 32: 1000 events 2.177s 459 events/sec filer_sync_subscription_test.go:174: concurrency=128: 1000 events 2.029s 493 events/sec filer_sync_subscription_test.go:174: concurrency=512: 1000 events 1.319s 758 events/sec filer_sync_subscription_test.go:180: concurrency 512 vs 32: 1.65x (expected ~1.0x when stream-limited) filer_sync_subscription_test.go:183: unexpected ratio 1.65: concurrency should not affect stream-limited throughput --- FAIL: TestConcurrencyIneffectiveOnStreamBottleneck (5.53s) === RUN TestLogBufferSubscriptionThroughput === RUN TestLogBufferSubscriptionThroughput/single_subscriber_root filer_sync_subscription_test.go:259: 1 subscriber (/): 2000 events 2.269s 881 events/sec === RUN TestLogBufferSubscriptionThroughput/parallel_subscribers_per_dir filer_sync_subscription_test.go:315: 10 subscribers: 2000 events 219ms 9130 events/sec === NAME TestLogBufferSubscriptionThroughput filer_sync_subscription_test.go:325: LogBuffer speedup: 10.4x (10 parallel subscribers vs 1) --- PASS: TestLogBufferSubscriptionThroughput (2.49s) --- PASS: TestLogBufferSubscriptionThroughput/single_subscriber_root (2.27s) --- PASS: TestLogBufferSubscriptionThroughput/parallel_subscribers_per_dir (0.22s) === RUN TestVerifySyncMissingFile [MISSING] /root/file.txt (size=100, etag=d41d8cd98f00b204e9800998ecf8427e-0) --- PASS: TestVerifySyncMissingFile (0.00s) === RUN TestVerifySyncOnlyInB === RUN TestVerifySyncOnlyInB/bidirectional [ONLY_IN_B] /root/extra.txt === RUN TestVerifySyncOnlyInB/active-passive_ignores_onlyInB --- PASS: TestVerifySyncOnlyInB (0.00s) --- PASS: TestVerifySyncOnlyInB/bidirectional (0.00s) --- PASS: TestVerifySyncOnlyInB/active-passive_ignores_onlyInB (0.00s) === RUN TestVerifySyncSizeMismatch [SIZE_MISMATCH] /root/data.bin (a=1024, b=512, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncSizeMismatch (0.00s) === RUN TestVerifySyncConcurrencyBound --- PASS: TestVerifySyncConcurrencyBound (0.03s) === RUN TestVerifySyncETagMismatch [ETAG_MISMATCH] /root/data.bin (a=112233, b=445566, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncETagMismatch (0.00s) === RUN TestVerifySyncChunkReorder --- PASS: TestVerifySyncChunkReorder (0.00s) === RUN TestVerifySyncGenuineChunkDivergence [ETAG_MISMATCH] /root/model.bin (a=bc86f2b2489529a978fb72f5c7924a06-3, b=835086ae0fedc4b8173fed690601b0d0-3, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncGenuineChunkDivergence (0.00s) === RUN TestVerifySyncChunkCountDiffStaysEtagMismatch [ETAG_MISMATCH] /root/data.bin (a=4054c6f7787ecb7b323323155b0c0583-3, b=be6996cb706f34d65b9fee8745325abf-2, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncChunkCountDiffStaysEtagMismatch (0.00s) === RUN TestVerifySyncDuplicateOffsetStaysEtagMismatch [ETAG_MISMATCH] /root/data.bin (a=cb662da895d59679c42914df7a97a8f8-2, b=5ea5b42ba5a8f329a95a02fbb277546d-2, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncDuplicateOffsetStaysEtagMismatch (0.00s) === RUN TestVerifySyncManifestChunkStaysEtagMismatch [ETAG_MISMATCH] /root/data.bin (a=0a5ffa6ef1e4da9211ba94feda315660-2, b=278c4fdf68ec7ad5e134331f06ea61f4-2, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncManifestChunkStaysEtagMismatch (0.00s) === RUN TestVerifySyncEmptyChunkETagStaysEtagMismatch [ETAG_MISMATCH] /root/data.bin (a=2abb7e6461a0bdb2a4a2a2d16bbd4ae2-3, b=a356dbadc2bbb35f128fb27178f8a89b-3, mtime equal [chunk-level issue]) --- PASS: TestVerifySyncEmptyChunkETagStaysEtagMismatch (0.00s) === RUN TestVerifySyncChunkReorderJSONEmittedByDefault --- PASS: TestVerifySyncChunkReorderJSONEmittedByDefault (0.00s) === RUN TestVerifySyncCutoffTime === RUN TestVerifySyncCutoffTime/A-only_recent_file_is_skipped,_not_reported_missing === RUN TestVerifySyncCutoffTime/A-only_old_file_is_reported_missing [MISSING] /old.txt (size=10, etag=d41d8cd98f00b204e9800998ecf8427e-0) === RUN TestVerifySyncCutoffTime/B-only_recent_file_is_skipped,_not_reported_as_ONLY_IN_B === RUN TestVerifySyncCutoffTime/B-only_old_file_is_reported_as_ONLY_IN_B [ONLY_IN_B] /old.txt --- PASS: TestVerifySyncCutoffTime (0.00s) --- PASS: TestVerifySyncCutoffTime/A-only_recent_file_is_skipped,_not_reported_missing (0.00s) --- PASS: TestVerifySyncCutoffTime/A-only_old_file_is_reported_missing (0.00s) --- PASS: TestVerifySyncCutoffTime/B-only_recent_file_is_skipped,_not_reported_as_ONLY_IN_B (0.00s) --- PASS: TestVerifySyncCutoffTime/B-only_old_file_is_reported_as_ONLY_IN_B (0.00s) === RUN TestVerifySyncCutoffMatchedFileBSideRecent --- PASS: TestVerifySyncCutoffMatchedFileBSideRecent (0.00s) === RUN TestVerifySyncMissingDirRecursesEvenWithRecentMtime [MISSING] /subdir/ (directory) [MISSING] /subdir/old.txt (size=10, etag=d41d8cd98f00b204e9800998ecf8427e-0) --- PASS: TestVerifySyncMissingDirRecursesEvenWithRecentMtime (0.00s) === RUN TestVerifySyncRootPath --- PASS: TestVerifySyncRootPath (0.00s) === RUN TestVerifySyncNoDeadlockDeepTree --- PASS: TestVerifySyncNoDeadlockDeepTree (0.01s) === RUN TestVerifySyncByteOrderSkew --- PASS: TestVerifySyncByteOrderSkew (0.00s) === RUN TestVerifySyncSourcesListConcurrently --- PASS: TestVerifySyncSourcesListConcurrently (0.00s) === RUN TestVerifySyncListErrorNoBogusDiffs --- PASS: TestVerifySyncListErrorNoBogusDiffs (0.00s) === RUN TestFixEcxFromShards I0720 21:30:07.792351 ec_encoder.go:138 encodeDatFile /tmp/TestFixEcxFromShards2749425022/001/7.dat size:3000 with EC context 10+4 (total: 14) I0720 21:30:07.802913 fix.go:407 volume 7: no .dat size in .vif; reconstructing padded .dat (10485760 bytes) from 10 data shards I0720 21:30:07.806823 fix.go:426 volume 7: wrote /tmp/TestFixEcxFromShards2749425022/001/7.ecx from 10 data shards I0720 21:30:07.807050 fix.go:447 volume 7: wrote /tmp/TestFixEcxFromShards2749425022/001/7.vif (version 3, datFileSize 3000, ec 10+4) --- PASS: TestFixEcxFromShards (0.02s) === RUN TestFixEcxFromShardsWithVif I0720 21:30:07.809767 ec_encoder.go:138 encodeDatFile /tmp/TestFixEcxFromShardsWithVif2644947510/001/9.dat size:3000 with EC context 10+4 (total: 14) I0720 21:30:07.820242 fix.go:426 volume 9: wrote /tmp/TestFixEcxFromShardsWithVif2644947510/001/9.ecx from 10 data shards --- PASS: TestFixEcxFromShardsWithVif (0.01s) === RUN TestFixEcxFromShardsMissingShards I0720 21:30:07.823057 ec_encoder.go:138 encodeDatFile /tmp/TestFixEcxFromShardsMissingShards2929209193/001/11.dat size:3000 with EC context 10+4 (total: 14) I0720 21:30:07.834880 fix.go:384 volume 11: 11/14 shards present; reconstructing missing shards (10+4 (total: 14)) before index rebuild I0720 21:30:07.834986 ec_encoder.go:254 rebuilding /tmp/TestFixEcxFromShardsMissingShards2929209193/001/11: 11 shards present, 3 missing [2 5 11], config 10+4 (total: 14) I0720 21:30:07.839146 fix.go:407 volume 11: no .dat size in .vif; reconstructing padded .dat (10485760 bytes) from 10 data shards I0720 21:30:07.844452 fix.go:426 volume 11: wrote /tmp/TestFixEcxFromShardsMissingShards2929209193/001/11.ecx from 10 data shards I0720 21:30:07.844530 fix.go:447 volume 11: wrote /tmp/TestFixEcxFromShardsMissingShards2929209193/001/11.vif (version 3, datFileSize 3000, ec 10+4) --- PASS: TestFixEcxFromShardsMissingShards (0.03s) === RUN TestSaveToIdxWritesOffsetOrder --- PASS: TestSaveToIdxWritesOffsetOrder (0.00s) === RUN TestRunFuseDoesNotSkipOptionAfterConcurrentWriters --- PASS: TestRunFuseDoesNotSkipOptionAfterConcurrentWriters (0.00s) === RUN TestPeerIndexIgnoresGrpcPort --- PASS: TestPeerIndexIgnoresGrpcPort (0.00s) === RUN TestCheckPeersAddsSelfWhenGrpcPortMismatches I0720 21:30:07.848539 master.go:380 current: 127.0.0.1:9000 peers:127.0.0.1:9002,127.0.0.1:9003 --- PASS: TestCheckPeersAddsSelfWhenGrpcPortMismatches (0.00s) === RUN TestCheckPeersCanonicalizesSelfEntry I0720 21:30:07.848558 master.go:380 current: 127.0.0.1:9000 peers:127.0.0.1:9000,127.0.0.1:9002,127.0.0.1:9003 --- PASS: TestCheckPeersCanonicalizesSelfEntry (0.00s) === RUN TestCheckPeersDeduplicatesAliasPeers I0720 21:30:07.848573 master.go:380 current: 127.0.0.1:9000 peers:127.0.0.1:9002,127.0.0.1:9002.19002,127.0.0.1:9003 --- PASS: TestCheckPeersDeduplicatesAliasPeers (0.00s) === RUN TestMetadataEventDirectoryMembershipUsesDirectoryBoundaries --- PASS: TestMetadataEventDirectoryMembershipUsesDirectoryBoundaries (0.00s) === RUN TestMetadataEventUpdatesAndRemovesDirectory === RUN TestMetadataEventUpdatesAndRemovesDirectory/nil_response === RUN TestMetadataEventUpdatesAndRemovesDirectory/create_event === RUN TestMetadataEventUpdatesAndRemovesDirectory/rename_out === RUN TestMetadataEventUpdatesAndRemovesDirectory/rename_into === RUN TestMetadataEventUpdatesAndRemovesDirectory/rename_within === RUN TestMetadataEventUpdatesAndRemovesDirectory/delete --- PASS: TestMetadataEventUpdatesAndRemovesDirectory (0.00s) --- PASS: TestMetadataEventUpdatesAndRemovesDirectory/nil_response (0.00s) --- PASS: TestMetadataEventUpdatesAndRemovesDirectory/create_event (0.00s) --- PASS: TestMetadataEventUpdatesAndRemovesDirectory/rename_out (0.00s) --- PASS: TestMetadataEventUpdatesAndRemovesDirectory/rename_into (0.00s) --- PASS: TestMetadataEventUpdatesAndRemovesDirectory/rename_within (0.00s) --- PASS: TestMetadataEventUpdatesAndRemovesDirectory/delete (0.00s) === RUN TestApplyMiniAdminCredentialFallbackFromEnv --- PASS: TestApplyMiniAdminCredentialFallbackFromEnv (0.00s) === RUN TestParseBucketList === RUN TestParseBucketList/empty === RUN TestParseBucketList/single === RUN TestParseBucketList/multi === RUN TestParseBucketList/trims_whitespace === RUN TestParseBucketList/drops_empty_entries === RUN TestParseBucketList/dedupes_preserving_order === RUN TestParseBucketList/only_commas --- PASS: TestParseBucketList (0.00s) --- PASS: TestParseBucketList/empty (0.00s) --- PASS: TestParseBucketList/single (0.00s) --- PASS: TestParseBucketList/multi (0.00s) --- PASS: TestParseBucketList/trims_whitespace (0.00s) --- PASS: TestParseBucketList/drops_empty_entries (0.00s) --- PASS: TestParseBucketList/dedupes_preserving_order (0.00s) --- PASS: TestParseBucketList/only_commas (0.00s) === RUN TestMiniDefaultPluginJobTypes --- PASS: TestMiniDefaultPluginJobTypes (0.00s) === RUN Test_bucketPathForMountRoot === RUN Test_bucketPathForMountRoot/bucket_root_mount === RUN Test_bucketPathForMountRoot/bucket_root_with_trailing_slash === RUN Test_bucketPathForMountRoot/subdirectory_mount === RUN Test_bucketPathForMountRoot/non-bucket_mount --- PASS: Test_bucketPathForMountRoot (0.00s) --- PASS: Test_bucketPathForMountRoot/bucket_root_mount (0.00s) --- PASS: Test_bucketPathForMountRoot/bucket_root_with_trailing_slash (0.00s) --- PASS: Test_bucketPathForMountRoot/subdirectory_mount (0.00s) --- PASS: Test_bucketPathForMountRoot/non-bucket_mount (0.00s) === RUN TestBuildPluginWorkerHandlerExplicitTypes --- PASS: TestBuildPluginWorkerHandlerExplicitTypes (0.00s) === RUN TestBuildPluginWorkerHandlerAliases --- PASS: TestBuildPluginWorkerHandlerAliases (0.00s) === RUN TestBuildPluginWorkerHandlerUnknown --- PASS: TestBuildPluginWorkerHandlerUnknown (0.00s) === RUN TestBuildPluginWorkerHandlers --- PASS: TestBuildPluginWorkerHandlers (0.00s) === RUN TestBuildPluginWorkerHandlersCategories --- PASS: TestBuildPluginWorkerHandlersCategories (0.00s) === RUN TestPluginWorkerDefaultJobTypes --- PASS: TestPluginWorkerDefaultJobTypes (0.00s) === RUN TestResolvePluginWorkerID I0720 21:30:07.849000 worker.go:148 Generated new worker ID: w-arch-n-571a I0720 21:30:07.849016 worker.go:155 Saved worker ID to /tmp/TestResolvePluginWorkerID1172700235/001/worker.id I0720 21:30:07.849031 worker.go:110 Loaded existing worker ID from /tmp/TestResolvePluginWorkerID1172700235/001/worker.id: w-arch-n-571a --- PASS: TestResolvePluginWorkerID (0.00s) === RUN TestParsePluginWorkerAdminAddress --- PASS: TestParsePluginWorkerAdminAddress (0.00s) === RUN TestResolvePluginWorkerAdminServerUsesStatusGrpcPort --- PASS: TestResolvePluginWorkerAdminServerUsesStatusGrpcPort (0.00s) === RUN TestResolvePluginWorkerAdminServerKeepsDefaultGrpcOffset --- PASS: TestResolvePluginWorkerAdminServerKeepsDefaultGrpcOffset (0.00s) === RUN TestDeriveS3AdvertisedEndpoint === RUN TestDeriveS3AdvertisedEndpoint/wildcard_bind_IP_with_no_externalUrl_does_not_advertise_an_endpoint === RUN TestDeriveS3AdvertisedEndpoint/empty_bind_IP_with_no_externalUrl_does_not_advertise_an_endpoint === RUN TestDeriveS3AdvertisedEndpoint/explicit_bind_IP_is_kept === RUN TestDeriveS3AdvertisedEndpoint/IPv6_literals_are_bracketed === RUN TestDeriveS3AdvertisedEndpoint/TLS_key_+_https_port_switches_scheme_and_port === RUN TestDeriveS3AdvertisedEndpoint/TLS-only_(no_separate_https_port)_still_uses_https_on_the_HTTP_port === RUN TestDeriveS3AdvertisedEndpoint/externalUrl_wins_and_trailing_slash_is_stripped --- PASS: TestDeriveS3AdvertisedEndpoint (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/wildcard_bind_IP_with_no_externalUrl_does_not_advertise_an_endpoint (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/empty_bind_IP_with_no_externalUrl_does_not_advertise_an_endpoint (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/explicit_bind_IP_is_kept (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/IPv6_literals_are_bracketed (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/TLS_key_+_https_port_switches_scheme_and_port (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/TLS-only_(no_separate_https_port)_still_uses_https_on_the_HTTP_port (0.00s) --- PASS: TestDeriveS3AdvertisedEndpoint/externalUrl_wins_and_trailing_slash_is_stripped (0.00s) === RUN TestReadingTomlConfiguration database is map[connection_max:5000 enabled:true ports:[8001 8001 8002] server:192.168.1.1] servers is map[alpha:map[dc:eqdc10 ip:10.0.0.1] beta:map[dc:eqdc10 ip:10.0.0.2]] alpha ip is 10.0.0.1 --- PASS: TestReadingTomlConfiguration (0.00s) === RUN TestXYZ I0720 21:30:07.850033 volume_test.go:12 Last-Modified Mon, 08 Jul 2013 08:53:16 GMT --- PASS: TestXYZ (0.00s) === RUN TestWorkerDefaultJobTypes --- PASS: TestWorkerDefaultJobTypes (0.00s) FAIL FAIL github.com/seaweedfs/seaweedfs/weed/command 13.716s ? github.com/seaweedfs/seaweedfs/weed/command/scaffold [no test files] === RUN TestCredentialStoreInterface --- PASS: TestCredentialStoreInterface (0.00s) === RUN TestCredentialManagerCreation --- PASS: TestCredentialManagerCreation (0.00s) === RUN TestCredentialInterface --- PASS: TestCredentialInterface (0.00s) === RUN TestCredentialManagerIntegration --- PASS: TestCredentialManagerIntegration (0.00s) === RUN TestErrorTypes --- PASS: TestErrorTypes (0.00s) === RUN TestGetAvailableStores credential_test.go:317: Available stores: [memory filer_etc] (count: 2) --- PASS: TestGetAvailableStores (0.00s) === RUN TestValidateServiceAccountId === RUN TestValidateServiceAccountId/simple === RUN TestValidateServiceAccountId/uppercase_parent === RUN TestValidateServiceAccountId/hyphenated_uuid_suffix === RUN TestValidateServiceAccountId/with_hyphen === RUN TestValidateServiceAccountId/with_underscore === RUN TestValidateServiceAccountId/email-style === RUN TestValidateServiceAccountId/with_dot === RUN TestValidateServiceAccountId/with_plus === RUN TestValidateServiceAccountId/with_equals === RUN TestValidateServiceAccountId/with_comma === RUN TestValidateServiceAccountId/empty === RUN TestValidateServiceAccountId/missing_prefix === RUN TestValidateServiceAccountId/wrong_prefix === RUN TestValidateServiceAccountId/uppercase_uuid === RUN TestValidateServiceAccountId/missing_uuid === RUN TestValidateServiceAccountId/missing_user === RUN TestValidateServiceAccountId/colon_in_user --- PASS: TestValidateServiceAccountId (0.00s) --- PASS: TestValidateServiceAccountId/simple (0.00s) --- PASS: TestValidateServiceAccountId/uppercase_parent (0.00s) --- PASS: TestValidateServiceAccountId/hyphenated_uuid_suffix (0.00s) --- PASS: TestValidateServiceAccountId/with_hyphen (0.00s) --- PASS: TestValidateServiceAccountId/with_underscore (0.00s) --- PASS: TestValidateServiceAccountId/email-style (0.00s) --- PASS: TestValidateServiceAccountId/with_dot (0.00s) --- PASS: TestValidateServiceAccountId/with_plus (0.00s) --- PASS: TestValidateServiceAccountId/with_equals (0.00s) --- PASS: TestValidateServiceAccountId/with_comma (0.00s) --- PASS: TestValidateServiceAccountId/empty (0.00s) --- PASS: TestValidateServiceAccountId/missing_prefix (0.00s) --- PASS: TestValidateServiceAccountId/wrong_prefix (0.00s) --- PASS: TestValidateServiceAccountId/uppercase_uuid (0.00s) --- PASS: TestValidateServiceAccountId/missing_uuid (0.00s) --- PASS: TestValidateServiceAccountId/missing_user (0.00s) --- PASS: TestValidateServiceAccountId/colon_in_user (0.00s) === RUN TestSaveConfigurationDelegatesWithoutMaster --- PASS: TestSaveConfigurationDelegatesWithoutMaster (0.00s) === RUN TestSaveConfigurationDiffSnapshotTakenBeforeWrite --- PASS: TestSaveConfigurationDiffSnapshotTakenBeforeWrite (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/credential 0.082s === RUN TestFilerEtcStoreListPolicyNamesIncludesLegacyPolicies --- PASS: TestFilerEtcStoreListPolicyNamesIncludesLegacyPolicies (0.01s) === RUN TestFilerEtcStoreDeletePolicyRemovesLegacyManagedCopy --- PASS: TestFilerEtcStoreDeletePolicyRemovesLegacyManagedCopy (0.00s) === RUN TestFilerEtcStoreDeletePolicySerializesLegacyUpdates --- PASS: TestFilerEtcStoreDeletePolicySerializesLegacyUpdates (0.31s) === RUN TestFilerEtcStoreLoadManagedPoliciesRespectsReadContext W0720 21:29:54.502764 filer_etc_policy.go:190 Failed to read policy file cancel-me.json: LookupEntry1: rpc error: code = Canceled desc = context canceled --- PASS: TestFilerEtcStoreLoadManagedPoliciesRespectsReadContext (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/credential/filer_etc 0.366s === RUN TestIamGrpcStore_AdminBearerToken === RUN TestIamGrpcStore_AdminBearerToken/matching_key_succeeds === RUN TestIamGrpcStore_AdminBearerToken/wrong_key_unauthenticated === RUN TestIamGrpcStore_AdminBearerToken/no_key_unauthenticated --- PASS: TestIamGrpcStore_AdminBearerToken (0.01s) --- PASS: TestIamGrpcStore_AdminBearerToken/matching_key_succeeds (0.00s) --- PASS: TestIamGrpcStore_AdminBearerToken/wrong_key_unauthenticated (0.00s) --- PASS: TestIamGrpcStore_AdminBearerToken/no_key_unauthenticated (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/credential/grpc 0.088s === RUN TestMemoryStore --- PASS: TestMemoryStore (0.00s) === RUN TestMemoryStoreConcurrency --- PASS: TestMemoryStoreConcurrency (0.00s) === RUN TestMemoryStoreReset --- PASS: TestMemoryStoreReset (0.00s) === RUN TestMemoryStoreConfigurationSaveLoad --- PASS: TestMemoryStoreConfigurationSaveLoad (0.00s) === RUN TestMemoryStoreServiceAccountByAccessKey --- PASS: TestMemoryStoreServiceAccountByAccessKey (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/credential/memory 0.048s ? github.com/seaweedfs/seaweedfs/weed/credential/postgres [no test files] === RUN TestInlinePolicyOperations --- PASS: TestInlinePolicyOperations (0.00s) === RUN TestMemoryRenameUserMovesIdentityAndPolicies --- PASS: TestMemoryRenameUserMovesIdentityAndPolicies (0.00s) === RUN TestStoreRegistration integration_test.go:40: Available stores: [memory postgres filer_etc] --- PASS: TestStoreRegistration (0.00s) === RUN TestMemoryStoreIntegration --- PASS: TestMemoryStoreIntegration (0.00s) === RUN TestPolicyManagement --- PASS: TestPolicyManagement (0.00s) === RUN TestPolicyManagementWithFilerEtc policy_test.go:140: Filer connection required for filer_etc store testing --- SKIP: TestPolicyManagementWithFilerEtc (0.00s) === RUN TestPolicyManagementWithPostgres policy_test.go:146: PostgreSQL connection required for postgres store testing --- SKIP: TestPolicyManagementWithPostgres (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/credential/test 0.052s === RUN TestEntryCodec_AtimeRoundTrip --- PASS: TestEntryCodec_AtimeRoundTrip (0.00s) === RUN TestEntryCodec_AtimeZeroFallsBackToMtime --- PASS: TestEntryCodec_AtimeZeroFallsBackToMtime (0.00s) === RUN TestEntryCodec_AtimeSubSecondEpochPreserved --- PASS: TestEntryCodec_AtimeSubSecondEpochPreserved (0.00s) === RUN TestChunkGroup_ReadDataAt_ErrorHandling === RUN TestChunkGroup_ReadDataAt_ErrorHandling/should_return_immediately_on_error === RUN TestChunkGroup_ReadDataAt_ErrorHandling/should_handle_EOF_correctly === RUN TestChunkGroup_ReadDataAt_ErrorHandling/should_return_EOF_when_offset_exceeds_file_size === RUN TestChunkGroup_ReadDataAt_ErrorHandling/should_demonstrate_the_GitHub_issue_fix_-_errors_should_not_be_masked === RUN TestChunkGroup_ReadDataAt_ErrorHandling/Context_Cancellation filechunk_group_test.go:145: Operation completed before context cancellation was checked - this is expected for empty ChunkGroup === RUN TestChunkGroup_ReadDataAt_ErrorHandling/Context_Cancellation_with_Timeout --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling (0.00s) --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling/should_return_immediately_on_error (0.00s) --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling/should_handle_EOF_correctly (0.00s) --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling/should_return_EOF_when_offset_exceeds_file_size (0.00s) --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling/should_demonstrate_the_GitHub_issue_fix_-_errors_should_not_be_masked (0.00s) --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling/Context_Cancellation (0.00s) --- PASS: TestChunkGroup_ReadDataAt_ErrorHandling/Context_Cancellation_with_Timeout (0.00s) === RUN TestChunkGroup_SearchChunks_Cancellation === RUN TestChunkGroup_SearchChunks_Cancellation/Context_Cancellation_in_SearchChunks filechunk_group_test.go:207: SearchChunks completed with cancelled context - context threading verified === RUN TestChunkGroup_SearchChunks_Cancellation/Context_with_Timeout_in_SearchChunks --- PASS: TestChunkGroup_SearchChunks_Cancellation (0.00s) --- PASS: TestChunkGroup_SearchChunks_Cancellation/Context_Cancellation_in_SearchChunks (0.00s) --- PASS: TestChunkGroup_SearchChunks_Cancellation/Context_with_Timeout_in_SearchChunks (0.00s) === RUN TestChunkGroup_doSearchChunks --- PASS: TestChunkGroup_doSearchChunks (0.00s) === RUN TestDoMaybeManifestize test 0 test 1 test 2 test 3 --- PASS: TestDoMaybeManifestize (0.00s) === RUN TestManifestRoundTripPreservesChunks --- PASS: TestManifestRoundTripPreservesChunks (0.00s) === RUN TestCompactResolvedOverlappingManifests --- PASS: TestCompactResolvedOverlappingManifests (0.00s) === RUN TestDoMinusChunksWithResolvedManifests --- PASS: TestDoMinusChunksWithResolvedManifests (0.00s) === RUN TestManifestizeSmallBatchWithRemainder --- PASS: TestManifestizeSmallBatchWithRemainder (0.00s) === RUN TestCompactMultipleOverlappingManifestGenerations --- PASS: TestCompactMultipleOverlappingManifestGenerations (0.00s) === RUN TestManifestBloatDetection filechunk_manifest_test.go:493: n=1 manifests: totalStored=20000 fileSize=10000 ratio=2.0x expectMerge=false filechunk_manifest_test.go:493: n=2 manifests: totalStored=30000 fileSize=10000 ratio=3.0x expectMerge=true filechunk_manifest_test.go:493: n=3 manifests: totalStored=40000 fileSize=10000 ratio=4.0x expectMerge=true filechunk_manifest_test.go:493: n=4 manifests: totalStored=50000 fileSize=10000 ratio=5.0x expectMerge=true filechunk_manifest_test.go:493: n=5 manifests: totalStored=60000 fileSize=10000 ratio=6.0x expectMerge=true filechunk_manifest_test.go:493: n=6 manifests: totalStored=70000 fileSize=10000 ratio=7.0x expectMerge=true filechunk_manifest_test.go:493: n=7 manifests: totalStored=80000 fileSize=10000 ratio=8.0x expectMerge=true filechunk_manifest_test.go:493: n=8 manifests: totalStored=90000 fileSize=10000 ratio=9.0x expectMerge=true filechunk_manifest_test.go:493: n=9 manifests: totalStored=100000 fileSize=10000 ratio=10.0x expectMerge=true filechunk_manifest_test.go:493: n=10 manifests: totalStored=110000 fileSize=10000 ratio=11.0x expectMerge=true --- PASS: TestManifestBloatDetection (0.00s) === RUN Test_removeGarbageChunks --- PASS: Test_removeGarbageChunks (0.00s) === RUN TestDoMinusChunks 2026/07/20 21:29:54 first deleted chunks: [file_id:"1" size:3 modified_ts_ns:100 source_file_id:"11" file_id:"2" offset:3 size:3 modified_ts_ns:200 file_id:"3" offset:6 size:3 modified_ts_ns:300 source_file_id:"33"] 2026/07/20 21:29:54 clusterA synced empty chunks event result: [] --- PASS: TestDoMinusChunks (0.00s) === RUN TestCompactFileChunksRealCase I0720 21:29:54.235328 filechunks2_test.go:84 before chunk 2,512f31f2c0700a [ 0, 25) I0720 21:29:54.235418 filechunks2_test.go:84 before chunk 6,512f2c2e24e9e8 [ 868352, 917585) I0720 21:29:54.235421 filechunks2_test.go:84 before chunk 7,514468dd5954ca [ 884736, 901120) I0720 21:29:54.235423 filechunks2_test.go:84 before chunk 5,5144463173fe77 [ 917504, 2297856) I0720 21:29:54.235425 filechunks2_test.go:84 before chunk 4,51444c7ab54e2d [ 2301952, 2367488) I0720 21:29:54.235427 filechunks2_test.go:84 before chunk 4,514450e643ad22 [ 2371584, 2420736) I0720 21:29:54.235429 filechunks2_test.go:84 before chunk 6,514456a5e9e4d7 [ 2449408, 2490368) I0720 21:29:54.235431 filechunks2_test.go:84 before chunk 3,51444f8d53eebe [ 2494464, 2555904) I0720 21:29:54.235434 filechunks2_test.go:84 before chunk 4,5144578b097c7e [ 2560000, 2596864) I0720 21:29:54.235436 filechunks2_test.go:84 before chunk 3,51445500b6b4ac [ 2637824, 2678784) I0720 21:29:54.235438 filechunks2_test.go:84 before chunk 1,51446285e52a61 [ 2695168, 2715648) I0720 21:29:54.235446 filechunks2_test.go:84 compacted chunk 2,512f31f2c0700a [ 0, 25) I0720 21:29:54.235448 filechunks2_test.go:84 compacted chunk 6,512f2c2e24e9e8 [ 868352, 917585) I0720 21:29:54.235450 filechunks2_test.go:84 compacted chunk 7,514468dd5954ca [ 884736, 901120) I0720 21:29:54.235452 filechunks2_test.go:84 compacted chunk 5,5144463173fe77 [ 917504, 2297856) I0720 21:29:54.235455 filechunks2_test.go:84 compacted chunk 4,51444c7ab54e2d [ 2301952, 2367488) I0720 21:29:54.235456 filechunks2_test.go:84 compacted chunk 4,514450e643ad22 [ 2371584, 2420736) I0720 21:29:54.235458 filechunks2_test.go:84 compacted chunk 6,514456a5e9e4d7 [ 2449408, 2490368) I0720 21:29:54.235460 filechunks2_test.go:84 compacted chunk 3,51444f8d53eebe [ 2494464, 2555904) I0720 21:29:54.235463 filechunks2_test.go:84 compacted chunk 4,5144578b097c7e [ 2560000, 2596864) I0720 21:29:54.235465 filechunks2_test.go:84 compacted chunk 3,51445500b6b4ac [ 2637824, 2678784) I0720 21:29:54.235467 filechunks2_test.go:84 compacted chunk 1,51446285e52a61 [ 2695168, 2715648) --- PASS: TestCompactFileChunksRealCase (0.00s) === RUN TestReadResolvedChunks resolved to 4 visible intervales [0,50) a 1 [50,150) b 2 [175,275) e 5 [275,300) d 4 --- PASS: TestReadResolvedChunks (0.00s) === RUN TestReadResolvedChunks2 resolved to 2 visible intervales [200,225) e 5 [225,250) c 3 --- PASS: TestReadResolvedChunks2 (0.00s) === RUN TestRandomizedReadResolvedChunks --- PASS: TestRandomizedReadResolvedChunks (0.03s) === RUN TestSequentialReadResolvedChunks visibles 13--- PASS: TestSequentialReadResolvedChunks (0.00s) === RUN TestActualReadResolvedChunks [0,2097152) 5,e7b96fef48 1634447487595823000 [2097152,4194304) 5,e5562640b9 1634447487595826000 [4194304,6291456) 5,df033e0fe4 1634447487595827000 [6291456,8388608) 7,eb08148a9b 1634447487595827000 [8388608,10485760) 7,e0f92d1604 1634447487595828000 [10485760,12582912) 7,e33cb63262 1634447487595828000 [12582912,14680064) 5,ea98e40e93 1634447487595829000 [14680064,16777216) 5,e165661172 1634447487595829000 [16777216,18874368) 3,e692097486 1634447487595830000 [18874368,20971520) 3,e28e2e3cbd 1634447487595830000 [20971520,23068672) 3,e443974d4e 1634447487595830000 [23068672,25165824) 2,e815bed597 1634447487595831000 [25165824,27140560) 5,e94715199e 1634447487595832000 --- PASS: TestActualReadResolvedChunks (0.00s) === RUN TestActualReadResolvedChunks2 [0,184320) 1,e7b96fef48 1 [184320,188416) 2,33562640b9 4 [188416,2285568) 4,df033e0fe4 3 --- PASS: TestActualReadResolvedChunks2 (0.00s) === RUN TestCompactFileChunks --- PASS: TestCompactFileChunks (0.00s) === RUN TestCompactFileChunks2 --- PASS: TestCompactFileChunks2 (0.00s) === RUN TestRandomFileChunksCompact --- PASS: TestRandomFileChunksCompact (0.00s) === RUN TestIntervalMerging 2026/07/20 21:29:54 ++++++++++ merged test case 0 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 0, interval start=0, stop=100, fileId=abc 2026/07/20 21:29:54 test case 0, interval start=100, stop=200, fileId=asdf 2026/07/20 21:29:54 test case 0, interval start=200, stop=300, fileId=fsad 2026/07/20 21:29:54 ++++++++++ merged test case 1 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 1, interval start=0, stop=200, fileId=asdf 2026/07/20 21:29:54 ++++++++++ merged test case 2 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 2, interval start=0, stop=70, fileId=b 2026/07/20 21:29:54 test case 2, interval start=70, stop=100, fileId=a 2026/07/20 21:29:54 ++++++++++ merged test case 3 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 3, interval start=0, stop=50, fileId=asdf 2026/07/20 21:29:54 test case 3, interval start=50, stop=300, fileId=xxxx 2026/07/20 21:29:54 ++++++++++ merged test case 4 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 4, interval start=0, stop=200, fileId=asdf 2026/07/20 21:29:54 test case 4, interval start=250, stop=500, fileId=xxxx 2026/07/20 21:29:54 ++++++++++ merged test case 5 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 5, interval start=0, stop=200, fileId=d 2026/07/20 21:29:54 test case 5, interval start=200, stop=220, fileId=c 2026/07/20 21:29:54 ++++++++++ merged test case 6 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 6, interval start=0, stop=100, fileId=xyz 2026/07/20 21:29:54 ++++++++++ merged test case 7 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 7, interval start=0, stop=2097152, fileId=3,029565bf3092 2026/07/20 21:29:54 test case 7, interval start=2097152, stop=5242880, fileId=6,029632f47ae2 2026/07/20 21:29:54 test case 7, interval start=5242880, stop=8388608, fileId=2,029734c5aa10 2026/07/20 21:29:54 test case 7, interval start=8388608, stop=11534336, fileId=5,02982f80de50 2026/07/20 21:29:54 test case 7, interval start=11534336, stop=14376529, fileId=7,0299ad723803 2026/07/20 21:29:54 ++++++++++ merged test case 8 ++++++++++++++++++++ 2026/07/20 21:29:54 test case 8, interval start=0, stop=77824, fileId=4,0b3df938e301 2026/07/20 21:29:54 test case 8, interval start=77824, stop=208896, fileId=4,0b3f0c7202f0 2026/07/20 21:29:54 test case 8, interval start=208896, stop=339968, fileId=2,0b4031a72689 2026/07/20 21:29:54 test case 8, interval start=339968, stop=471040, fileId=3,0b416a557362 2026/07/20 21:29:54 test case 8, interval start=471040, stop=472225, fileId=6,0b3e0650019c --- PASS: TestIntervalMerging (0.00s) === RUN TestChunksReading 2026/07/20 21:29:54 ++++++++++ read test case 0 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 0, chunk 0, offset=0, size=100, fileId=abc 2026/07/20 21:29:54 read case 0, chunk 1, offset=0, size=100, fileId=asdf 2026/07/20 21:29:54 read case 0, chunk 2, offset=0, size=50, fileId=fsad 2026/07/20 21:29:54 ++++++++++ read test case 1 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 1, chunk 0, offset=50, size=100, fileId=asdf 2026/07/20 21:29:54 ++++++++++ read test case 2 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 2, chunk 0, offset=20, size=30, fileId=b 2026/07/20 21:29:54 read case 2, chunk 1, offset=57, size=10, fileId=a 2026/07/20 21:29:54 ++++++++++ read test case 3 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 3, chunk 0, offset=0, size=50, fileId=asdf 2026/07/20 21:29:54 read case 3, chunk 1, offset=0, size=150, fileId=xxxx 2026/07/20 21:29:54 ++++++++++ read test case 4 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 4, chunk 0, offset=0, size=200, fileId=asdf 2026/07/20 21:29:54 read case 4, chunk 1, offset=0, size=150, fileId=xxxx 2026/07/20 21:29:54 ++++++++++ read test case 5 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 5, chunk 0, offset=0, size=200, fileId=c 2026/07/20 21:29:54 read case 5, chunk 1, offset=130, size=20, fileId=b 2026/07/20 21:29:54 ++++++++++ read test case 6 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 6, chunk 0, offset=0, size=100, fileId=xyz 2026/07/20 21:29:54 ++++++++++ read test case 7 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 7, chunk 0, offset=0, size=100, fileId=abc 2026/07/20 21:29:54 read case 7, chunk 1, offset=0, size=100, fileId=asdf 2026/07/20 21:29:54 ++++++++++ read test case 8 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 8, chunk 0, offset=0, size=90, fileId=abc 2026/07/20 21:29:54 read case 8, chunk 1, offset=0, size=100, fileId=asdf 2026/07/20 21:29:54 read case 8, chunk 2, offset=0, size=110, fileId=fsad 2026/07/20 21:29:54 ++++++++++ read test case 9 ++++++++++++++++++++ 2026/07/20 21:29:54 read case 9, chunk 0, offset=0, size=43175936, fileId=2,111fc2cbfac1 2026/07/20 21:29:54 read case 9, chunk 1, offset=0, size=9805824, fileId=2,112a36ea7f85 2026/07/20 21:29:54 read case 9, chunk 2, offset=0, size=19582976, fileId=4,112d5f31c5e7 2026/07/20 21:29:54 read case 9, chunk 3, offset=0, size=60690432, fileId=1,113245f0cdb6 2026/07/20 21:29:54 read case 9, chunk 4, offset=0, size=4014080, fileId=3,1141a70733b5 2026/07/20 21:29:54 read case 9, chunk 5, offset=0, size=16309588, fileId=1,114201d5bbdb --- PASS: TestChunksReading (0.00s) === RUN TestViewFromVisibleIntervals --- PASS: TestViewFromVisibleIntervals (0.00s) === RUN TestViewFromVisibleIntervals2 --- PASS: TestViewFromVisibleIntervals2 (0.00s) === RUN TestViewFromVisibleIntervals3 --- PASS: TestViewFromVisibleIntervals3 (0.00s) === RUN TestCompactFileChunks3 --- PASS: TestCompactFileChunks3 (0.00s) === RUN TestFilerConf --- PASS: TestFilerConf (0.00s) === RUN TestClonePathConf --- PASS: TestClonePathConf (0.00s) === RUN TestClonePathConfNil --- PASS: TestClonePathConfNil (0.00s) === RUN TestApplyBucketQuotaReadOnly --- PASS: TestApplyBucketQuotaReadOnly (0.00s) === RUN TestClearReadOnly --- PASS: TestClearReadOnly (0.00s) === RUN TestDeletionRetryQueue_AddAndRetrieve --- PASS: TestDeletionRetryQueue_AddAndRetrieve (0.00s) === RUN TestDeletionRetryQueue_ExponentialBackoff filer_deletion_test.go:80: Exponential backoff delays: [5m0.00000007s 10m0.00000005s 20m0.00000004s 40m0.000000041s 1h20m0.00000005s] --- PASS: TestDeletionRetryQueue_ExponentialBackoff (0.00s) === RUN TestDeletionRetryQueue_OverflowProtection --- PASS: TestDeletionRetryQueue_OverflowProtection (0.00s) === RUN TestDeletionRetryQueue_MaxAttemptsReached W0720 21:29:54.261615 filer_deletion.go:248 max retry attempts (10) reached for file1, last error: final error --- PASS: TestDeletionRetryQueue_MaxAttemptsReached (0.00s) === RUN TestCalculateBackoff --- PASS: TestCalculateBackoff (0.00s) === RUN TestIsRetryableError --- PASS: TestIsRetryableError (0.00s) === RUN TestDeletionRetryQueue_HeapOrdering --- PASS: TestDeletionRetryQueue_HeapOrdering (0.00s) === RUN TestDeletionRetryQueue_DuplicateFileIds --- PASS: TestDeletionRetryQueue_DuplicateFileIds (0.00s) === RUN TestEnsureEntryInodeMatchesFuseDerivation --- PASS: TestEnsureEntryInodeMatchesFuseDerivation (0.00s) === RUN TestEnsureEntryInodeSharesAcrossHardLinks --- PASS: TestEnsureEntryInodeSharesAcrossHardLinks (0.00s) === RUN TestCreateEntryAssignsInodeWhenMissing --- PASS: TestCreateEntryAssignsInodeWhenMissing (0.02s) === RUN TestCreateEntryAssignsInodesToAutoCreatedParents --- PASS: TestCreateEntryAssignsInodesToAutoCreatedParents (0.02s) === RUN TestUpdateEntryPreservesExistingInode E0720 21:29:54.314572 filer.go:385 existing /doc.txt is a file --- PASS: TestUpdateEntryPreservesExistingInode (0.01s) === RUN TestUpdateEntryBackfillsMissingLegacyInode --- PASS: TestUpdateEntryBackfillsMissingLegacyInode (0.00s) === RUN TestMaybeLazyFetchFromRemote_HitsRemoteAndPersists --- PASS: TestMaybeLazyFetchFromRemote_HitsRemoteAndPersists (0.01s) === RUN TestMaybeLazyFetchFromRemote_NotUnderMount --- PASS: TestMaybeLazyFetchFromRemote_NotUnderMount (0.00s) === RUN TestMaybeLazyFetchFromRemote_RemoteObjectNotFound --- PASS: TestMaybeLazyFetchFromRemote_RemoteObjectNotFound (0.00s) === RUN TestMaybeLazyFetchFromRemote_CreateEntryFailureReturnsInMemoryEntry E0720 21:29:54.322989 filer.go:260 insert entry /buckets/mybucket/failfile.txt: simulated store failure W0720 21:29:54.323011 filer_lazy_remote.go:102 maybeLazyFetchFromRemote: failed to persist filer entry for /buckets/mybucket/failfile.txt: insert entry /buckets/mybucket/failfile.txt: simulated store failure --- PASS: TestMaybeLazyFetchFromRemote_CreateEntryFailureReturnsInMemoryEntry (0.00s) === RUN TestMaybeLazyFetchFromRemote_LongestPrefixMount --- PASS: TestMaybeLazyFetchFromRemote_LongestPrefixMount (0.00s) === RUN TestMaybeLazyFetchFromRemote_ContextGuardPreventsRecursion --- PASS: TestMaybeLazyFetchFromRemote_ContextGuardPreventsRecursion (0.00s) === RUN TestFindEntry_LazyFetchOnMiss --- PASS: TestFindEntry_LazyFetchOnMiss (0.00s) === RUN TestDeleteEntryMetaAndData_RemoteOnlyFileDeletesRemoteAndMetadata --- PASS: TestDeleteEntryMetaAndData_RemoteOnlyFileDeletesRemoteAndMetadata (0.00s) === RUN TestDeleteEntryMetaAndData_IsFromOtherClusterSkipsRemoteDelete --- PASS: TestDeleteEntryMetaAndData_IsFromOtherClusterSkipsRemoteDelete (0.00s) === RUN TestDeleteEntryMetaAndData_RemoteOnlyFileNotUnderMountSkipsRemoteDelete --- PASS: TestDeleteEntryMetaAndData_RemoteOnlyFileNotUnderMountSkipsRemoteDelete (0.00s) === RUN TestDeleteEntryMetaAndData_RemoteMountWithoutClientResolutionKeepsMetadata --- PASS: TestDeleteEntryMetaAndData_RemoteMountWithoutClientResolutionKeepsMetadata (0.00s) === RUN TestDeleteEntryMetaAndData_LocalDeleteFailurePreservesMetadata --- PASS: TestDeleteEntryMetaAndData_LocalDeleteFailurePreservesMetadata (0.00s) === RUN TestDeleteEntryMetaAndData_RemoteDeleteNotFoundStillDeletesMetadata --- PASS: TestDeleteEntryMetaAndData_RemoteDeleteNotFoundStillDeletesMetadata (0.00s) === RUN TestDeleteEntryMetaAndData_RemoteDeleteErrorKeepsMetadata --- PASS: TestDeleteEntryMetaAndData_RemoteDeleteErrorKeepsMetadata (0.00s) === RUN TestDeleteEntryMetaAndData_DirectoryUnderMountDeletesRemoteDirectory --- PASS: TestDeleteEntryMetaAndData_DirectoryUnderMountDeletesRemoteDirectory (0.00s) === RUN TestDeleteEntryMetaAndData_RecursiveFolderDeleteRemotesChildren --- PASS: TestDeleteEntryMetaAndData_RecursiveFolderDeleteRemotesChildren (0.04s) === RUN TestMaybeLazyListFromRemote_PopulatesStoreFromRemote --- PASS: TestMaybeLazyListFromRemote_PopulatesStoreFromRemote (0.00s) === RUN TestMaybeLazyListFromRemote_DisabledWhenTTLZero --- PASS: TestMaybeLazyListFromRemote_DisabledWhenTTLZero (0.00s) === RUN TestMaybeLazyListFromRemote_TTLCachePreventsSecondCall --- PASS: TestMaybeLazyListFromRemote_TTLCachePreventsSecondCall (0.00s) === RUN TestMaybeLazyListFromRemote_NotUnderMount --- PASS: TestMaybeLazyListFromRemote_NotUnderMount (0.00s) === RUN TestMaybeLazyListFromRemote_SkipsLocalOnlyEntries --- PASS: TestMaybeLazyListFromRemote_SkipsLocalOnlyEntries (0.00s) === RUN TestMaybeLazyListFromRemote_MergesExistingRemoteEntry --- PASS: TestMaybeLazyListFromRemote_MergesExistingRemoteEntry (0.00s) === RUN TestMaybeLazyListFromRemote_ContextGuardPreventsRecursion --- PASS: TestMaybeLazyListFromRemote_ContextGuardPreventsRecursion (0.00s) === RUN TestResolveMetadataLogAssignDiskTypeUsesPathRule --- PASS: TestResolveMetadataLogAssignDiskTypeUsesPathRule (0.00s) === RUN TestResolveMetadataLogAssignDiskTypeFallsBackToFilerDefault --- PASS: TestResolveMetadataLogAssignDiskTypeFallsBackToFilerDefault (0.00s) === RUN TestResolveMetadataLogAssignDiskTypeNilFilerConf --- PASS: TestResolveMetadataLogAssignDiskTypeNilFilerConf (0.00s) === RUN TestProtoMarshal x to: 234,2423423422 * 2342342354223234,2342342342"6Ҧ 0Ø: text/jsonPØϮ --- PASS: TestProtoMarshal (0.00s) === RUN TestOnBucketEventsRenameIntoBucketsRootCreatesBucket --- PASS: TestOnBucketEventsRenameIntoBucketsRootCreatesBucket (0.00s) === RUN TestDetectBucket === RUN TestDetectBucket/object_in_bucket === RUN TestDetectBucket/bucket_root === RUN TestDetectBucket/not_under_buckets_path === RUN TestDetectBucket/buckets_path_itself === RUN TestDetectBucket/custom_buckets_path === RUN TestDetectBucket/nested_object --- PASS: TestDetectBucket (0.00s) --- PASS: TestDetectBucket/object_in_bucket (0.00s) --- PASS: TestDetectBucket/bucket_root (0.00s) --- PASS: TestDetectBucket/not_under_buckets_path (0.00s) --- PASS: TestDetectBucket/buckets_path_itself (0.00s) --- PASS: TestDetectBucket/custom_buckets_path (0.00s) --- PASS: TestDetectBucket/nested_object (0.00s) === RUN TestFilerStoreWrapperMimeNormalization === RUN TestFilerStoreWrapperMimeNormalization/files_strip_octet-stream/insert === RUN TestFilerStoreWrapperMimeNormalization/files_strip_octet-stream/update === RUN TestFilerStoreWrapperMimeNormalization/directories_keep_octet-stream/insert === RUN TestFilerStoreWrapperMimeNormalization/directories_keep_octet-stream/update --- PASS: TestFilerStoreWrapperMimeNormalization (0.00s) --- PASS: TestFilerStoreWrapperMimeNormalization/files_strip_octet-stream/insert (0.00s) --- PASS: TestFilerStoreWrapperMimeNormalization/files_strip_octet-stream/update (0.00s) --- PASS: TestFilerStoreWrapperMimeNormalization/directories_keep_octet-stream/insert (0.00s) --- PASS: TestFilerStoreWrapperMimeNormalization/directories_keep_octet-stream/update (0.00s) === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntry/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntry/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntryKnownAbsent/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntryKnownAbsent/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/UpdateEntry/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/UpdateEntry/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteEntry/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteEntry/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteOneEntry/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteOneEntry/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteFolderChildren/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteFolderChildren/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/BeginTransaction/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/BeginTransaction/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/CommitTransaction/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/CommitTransaction/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvPut/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvPut/deadline_exceeded === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvDelete/cancelled === RUN TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvDelete/deadline_exceeded --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntry/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntry/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntryKnownAbsent/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/InsertEntryKnownAbsent/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/UpdateEntry/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/UpdateEntry/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteEntry/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteEntry/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteOneEntry/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteOneEntry/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteFolderChildren/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/DeleteFolderChildren/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/BeginTransaction/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/BeginTransaction/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/CommitTransaction/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/CommitTransaction/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvPut/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvPut/deadline_exceeded (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvDelete/cancelled (0.00s) --- PASS: TestFilerStoreWrapperWriteOpsRejectCancelledContext/KvDelete/deadline_exceeded (0.00s) === RUN TestFilerStoreWrapperWriteOpsSucceedWithActiveContext --- PASS: TestFilerStoreWrapperWriteOpsSucceedWithActiveContext (0.00s) === RUN TestFilerStoreWrapperReadOpsSucceedWithCancelledContext --- PASS: TestFilerStoreWrapperReadOpsSucceedWithCancelledContext (0.00s) === RUN TestFilerStoreWrapperRollbackSucceedsWithCancelledContext --- PASS: TestFilerStoreWrapperRollbackSucceedsWithCancelledContext (0.00s) === RUN TestIntervalList_Overlay [0,25) 6 6 [25,50) 1 1 [50,150) 2 2 [175,210) 5 5 [210,225) 3 3 [225,250) 4 4 [0,25) 6 6 [25,50) 1 1 [50,150) 7 7 [175,210) 5 5 [210,225) 3 3 [225,250) 4 4 --- PASS: TestIntervalList_Overlay (0.00s) === RUN TestIntervalList_Overlay2 [0,50) 2 2 [50,100) 1 1 --- PASS: TestIntervalList_Overlay2 (0.00s) === RUN TestIntervalList_Overlay3 [0,60) 2 2 [60,100) 1 1 --- PASS: TestIntervalList_Overlay3 (0.00s) === RUN TestIntervalList_Overlay4 [0,100) 2 2 --- PASS: TestIntervalList_Overlay4 (0.00s) === RUN TestIntervalList_Overlay5 [0,110) 2 2 --- PASS: TestIntervalList_Overlay5 (0.00s) === RUN TestIntervalList_Overlay6 [50,110) 2 2 --- PASS: TestIntervalList_Overlay6 (0.00s) === RUN TestIntervalList_Overlay7 [50,90) 2 2 [90,100) 1 1 --- PASS: TestIntervalList_Overlay7 (0.00s) === RUN TestIntervalList_Overlay8 [50,60) 1 1 [60,90) 2 2 [90,100) 1 1 --- PASS: TestIntervalList_Overlay8 (0.00s) === RUN TestIntervalList_Overlay9 [50,60) 1 1 [60,100) 2 2 --- PASS: TestIntervalList_Overlay9 (0.00s) === RUN TestIntervalList_Overlay10 [50,60) 1 1 [60,110) 2 2 --- PASS: TestIntervalList_Overlay10 (0.00s) === RUN TestIntervalList_Overlay11 [0,90) 5 5 [90,100) 1 1 [100,110) 2 2 --- PASS: TestIntervalList_Overlay11 (0.00s) === RUN TestIntervalList_insertInterval1 [50,150) 2 2 [200,250) 3 3 --- PASS: TestIntervalList_insertInterval1 (0.00s) === RUN TestIntervalList_insertInterval2 [0,25) 3 3 [50,150) 2 2 --- PASS: TestIntervalList_insertInterval2 (0.00s) === RUN TestIntervalList_insertInterval3 [0,75) 3 3 [75,150) 2 2 [200,250) 4 4 --- PASS: TestIntervalList_insertInterval3 (0.00s) === RUN TestIntervalList_insertInterval4 [0,200) 3 3 [200,250) 4 4 --- PASS: TestIntervalList_insertInterval4 (0.00s) === RUN TestIntervalList_insertInterval5 [0,225) 5 5 [225,250) 4 4 --- PASS: TestIntervalList_insertInterval5 (0.00s) === RUN TestIntervalList_insertInterval6 [0,50) 1 1 [50,150) 2 2 [150,200) 1 1 [200,250) 4 4 [250,275) 1 1 --- PASS: TestIntervalList_insertInterval6 (0.00s) === RUN TestIntervalList_insertInterval7 [50,150) 2 2 [150,200) 1 1 [200,250) 4 4 [250,275) 1 1 --- PASS: TestIntervalList_insertInterval7 (0.00s) === RUN TestIntervalList_insertInterval8 [50,75) 2 2 [75,200) 3 3 [200,250) 4 4 [250,275) 3 3 --- PASS: TestIntervalList_insertInterval8 (0.00s) === RUN TestIntervalList_insertInterval9 [50,150) 3 3 [200,250) 4 4 --- PASS: TestIntervalList_insertInterval9 (0.00s) === RUN TestIntervalList_insertInterval10 [50,100) 2 2 [100,200) 5 5 [200,300) 4 4 --- PASS: TestIntervalList_insertInterval10 (0.00s) === RUN TestIntervalList_insertInterval11 [0,64) 1 1 [64,68) 2 2 [68,72) 4 4 [72,136) 3 3 --- PASS: TestIntervalList_insertInterval11 (0.00s) === RUN TestIntervalList_insertIntervalStruct [0,64) 1 {1 0 0} [64,68) 4 {4 0 0} [68,72) 2 {2 0 0} [72,136) 3 {3 0 0} --- PASS: TestIntervalList_insertIntervalStruct (0.00s) === RUN TestNotifyUpdateEventRecordsRequestMetadataEvent --- PASS: TestNotifyUpdateEventRecordsRequestMetadataEvent (0.00s) === RUN TestMountPeerRegistry_RegisterAndList --- PASS: TestMountPeerRegistry_RegisterAndList (0.00s) === RUN TestMountPeerRegistry_RenewExtendsExpiry --- PASS: TestMountPeerRegistry_RenewExtendsExpiry (0.00s) === RUN TestMountPeerRegistry_ExpirationDropsEntry --- PASS: TestMountPeerRegistry_ExpirationDropsEntry (0.00s) === RUN TestMountPeerRegistry_SweepCountsEvictions --- PASS: TestMountPeerRegistry_SweepCountsEvictions (0.00s) === RUN TestMountPeerRegistry_NegativeTTLFallsBackToDefault --- PASS: TestMountPeerRegistry_NegativeTTLFallsBackToDefault (0.00s) === RUN TestMountPeerRegistry_EmptyList --- PASS: TestMountPeerRegistry_EmptyList (0.00s) === RUN TestMountPeerRegistry_EmptyPeerAddrRejected --- PASS: TestMountPeerRegistry_EmptyPeerAddrRejected (0.00s) === RUN TestMountPeerRegistry_TTLCapped --- PASS: TestMountPeerRegistry_TTLCapped (0.00s) === RUN TestMountPeerRegistry_CapacityLimit --- PASS: TestMountPeerRegistry_CapacityLimit (0.00s) === RUN TestPersistedLogCacheHitMiss --- PASS: TestPersistedLogCacheHitMiss (0.00s) === RUN TestPersistedLogCacheNotCachedWhenUncacheable --- PASS: TestPersistedLogCacheNotCachedWhenUncacheable (0.00s) === RUN TestPersistedLogCacheSingleFlight --- PASS: TestPersistedLogCacheSingleFlight (0.01s) === RUN TestPersistedLogCacheEviction --- PASS: TestPersistedLogCacheEviction (0.00s) === RUN TestDecodeLogRecords --- PASS: TestDecodeLogRecords (0.00s) === RUN TestDecodeLogRecordsIncomplete --- PASS: TestDecodeLogRecordsIncomplete (0.00s) === RUN TestLogFileIteratorChunkedFiltering --- PASS: TestLogFileIteratorChunkedFiltering (0.00s) === RUN TestLogFileIteratorSkipsColdChunks --- PASS: TestLogFileIteratorSkipsColdChunks (0.00s) === RUN TestLogFileIteratorSharesDecodedChunks --- PASS: TestLogFileIteratorSharesDecodedChunks (0.00s) === RUN TestLogFileIteratorStreamsWhenRecordsSpanChunks --- PASS: TestLogFileIteratorStreamsWhenRecordsSpanChunks (0.00s) === RUN TestLogFileIteratorStreamFallbackResumesAfterYielded --- PASS: TestLogFileIteratorStreamFallbackResumesAfterYielded (0.00s) === RUN TestLogFileIteratorLoadErrorPropagates --- PASS: TestLogFileIteratorLoadErrorPropagates (0.00s) === RUN TestPersistedLogCacheIdleEviction --- PASS: TestPersistedLogCacheIdleEviction (0.00s) === RUN TestDecodeLogRecordsRejectsImplausibleRecords --- PASS: TestDecodeLogRecordsRejectsImplausibleRecords (0.00s) === RUN TestPersistedLogCacheLoadLargerThanBudget --- PASS: TestPersistedLogCacheLoadLargerThanBudget (0.00s) === RUN TestMergeRemoteContentEncoding --- PASS: TestMergeRemoteContentEncoding (0.00s) === RUN TestReaderAt --- PASS: TestReaderAt (0.00s) === RUN TestReaderAt0 --- PASS: TestReaderAt0 (0.00s) === RUN TestReaderAt1 --- PASS: TestReaderAt1 (0.00s) === RUN TestReaderAtGappedChunksDoNotLeak --- PASS: TestReaderAtGappedChunksDoNotLeak (0.00s) === RUN TestReaderAtSparseFileDoesNotLeak --- PASS: TestReaderAtSparseFileDoesNotLeak (0.00s) === RUN TestReaderCacheRetryAfterCacheInvalidation I0720 21:29:54.383142 stream.go:134 read chunk 425141,ef8914e9bdbbe8cb6838191f failed, invalidating cache and retrying: 404 Not Found: not found I0720 21:29:54.383153 stream.go:151 retrying read chunk 425141,ef8914e9bdbbe8cb6838191f with 1 new locations --- PASS: TestReaderCacheRetryAfterCacheInvalidation (0.00s) === RUN TestReaderCacheRemovesFailedDownloader --- PASS: TestReaderCacheRemovesFailedDownloader (0.00s) === RUN TestReaderCacheContextCancellation --- PASS: TestReaderCacheContextCancellation (0.00s) === RUN TestReaderCacheFallbackToChunkCache --- PASS: TestReaderCacheFallbackToChunkCache (0.00s) === RUN TestReaderCacheMultipleReadersWaitForSameChunk --- PASS: TestReaderCacheMultipleReadersWaitForSameChunk (0.00s) === RUN TestReaderCachePartialRead === RUN TestReaderCachePartialRead/read_from_start === RUN TestReaderCachePartialRead/read_from_middle === RUN TestReaderCachePartialRead/read_to_end === RUN TestReaderCachePartialRead/read_single_byte --- PASS: TestReaderCachePartialRead (0.00s) --- PASS: TestReaderCachePartialRead/read_from_start (0.00s) --- PASS: TestReaderCachePartialRead/read_from_middle (0.00s) --- PASS: TestReaderCachePartialRead/read_to_end (0.00s) --- PASS: TestReaderCachePartialRead/read_single_byte (0.00s) === RUN TestReaderCacheCleanup --- PASS: TestReaderCacheCleanup (0.00s) === RUN TestSingleChunkCacherDoneSignal --- PASS: TestSingleChunkCacherDoneSignal (0.00s) === RUN TestSingleChunkCacherLookupError --- PASS: TestSingleChunkCacherLookupError (0.00s) === RUN TestSingleChunkCacherContextCancellationDuringLookup --- PASS: TestSingleChunkCacherContextCancellationDuringLookup (0.00s) === RUN TestSingleChunkCacherMultipleReadersWaitForDownload --- PASS: TestSingleChunkCacherMultipleReadersWaitForDownload (0.00s) === RUN TestReaderCacheDownloaderDedup --- PASS: TestReaderCacheDownloaderDedup (0.00s) === RUN TestSingleChunkCacherOneReaderCancelsOthersContinue --- PASS: TestSingleChunkCacherOneReaderCancelsOthersContinue (0.00s) === RUN TestReaderPatternSequentialAndHysteresis --- PASS: TestReaderPatternSequentialAndHysteresis (0.00s) === RUN TestReaderPatternFarFirstReadIsRandom --- PASS: TestReaderPatternFarFirstReadIsRandom (0.00s) === RUN TestReaderPatternToleranceAbsorbsReorder --- PASS: TestReaderPatternToleranceAbsorbsReorder (0.00s) === RUN TestReaderPatternRandomRecoveryNeedsHysteresis --- PASS: TestReaderPatternRandomRecoveryNeedsHysteresis (0.00s) === RUN TestReaderPatternFrontierNeverRegresses --- PASS: TestReaderPatternFrontierNeverRegresses (0.00s) === RUN TestReaderPatternToleranceBoundary --- PASS: TestReaderPatternToleranceBoundary (0.00s) === RUN TestReaderPatternRecoversFromRandom --- PASS: TestReaderPatternRecoversFromRandom (0.00s) === RUN TestFilerRemoteStorage_FindRemoteStorageClient --- PASS: TestFilerRemoteStorage_FindRemoteStorageClient (0.00s) === RUN TestFilerRemoteStorage_FindMountDirectory_LongestPrefixWins --- PASS: TestFilerRemoteStorage_FindMountDirectory_LongestPrefixWins (0.00s) === RUN TestS3Conf --- PASS: TestS3Conf (0.00s) === RUN TestCheckDuplicateAccessKey --- PASS: TestCheckDuplicateAccessKey (0.00s) === RUN TestPrefetchInOrderDelivery --- PASS: TestPrefetchInOrderDelivery (0.00s) === RUN TestPrefetchSingleChunk --- PASS: TestPrefetchSingleChunk (0.00s) === RUN TestPrefetchFallbackToSequential --- PASS: TestPrefetchFallbackToSequential (0.00s) === RUN TestPrefetchContextCancellation I0720 21:29:54.441002 filechunk_manifest.go:185 read http://127.0.0.1:37283/1,2 failed, err: Get "http://127.0.0.1:37283/1,2?readDeleted=true": context deadline exceeded I0720 21:29:54.441053 filechunk_manifest.go:185 read http://127.0.0.1:37283/1,3 failed, err: Get "http://127.0.0.1:37283/1,3?readDeleted=true": context deadline exceeded I0720 21:29:54.441063 filechunk_manifest.go:185 read http://127.0.0.1:37283/1,0 failed, err: Get "http://127.0.0.1:37283/1,0?readDeleted=true": context deadline exceeded I0720 21:29:54.441075 filechunk_manifest.go:185 read http://127.0.0.1:37283/1,1 failed, err: Get "http://127.0.0.1:37283/1,1?readDeleted=true": context deadline exceeded --- PASS: TestPrefetchContextCancellation (0.10s) === RUN TestPrefetchRangeRequest --- PASS: TestPrefetchRangeRequest (0.00s) === RUN TestPrefetchLargePrefetchCount --- PASS: TestPrefetchLargePrefetchCount (0.00s) === RUN TestPrefetchConcurrentDownloads --- PASS: TestPrefetchConcurrentDownloads (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer 0.366s === RUN TestGetTxOrDBInternalSystemFolder --- PASS: TestGetTxOrDBInternalSystemFolder (0.00s) === RUN TestGetTxOrDBRealBucket --- PASS: TestGetTxOrDBRealBucket (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/abstract_sql 0.109s ? github.com/seaweedfs/seaweedfs/weed/filer/arangodb [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/cassandra [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/cassandra2 [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/elastic/v7 [no test files] === RUN TestCleanupQueue_Add --- PASS: TestCleanupQueue_Add (0.00s) === RUN TestCleanupQueue_Add_OutOfOrder --- PASS: TestCleanupQueue_Add_OutOfOrder (0.00s) === RUN TestCleanupQueue_Add_DuplicateWithOlderTime --- PASS: TestCleanupQueue_Add_DuplicateWithOlderTime (0.00s) === RUN TestCleanupQueue_Remove --- PASS: TestCleanupQueue_Remove (0.00s) === RUN TestCleanupQueue_Pop --- PASS: TestCleanupQueue_Pop (0.00s) === RUN TestCleanupQueue_Peek --- PASS: TestCleanupQueue_Peek (0.00s) === RUN TestCleanupQueue_Contains --- PASS: TestCleanupQueue_Contains (0.00s) === RUN TestCleanupQueue_ShouldProcess_MaxSize --- PASS: TestCleanupQueue_ShouldProcess_MaxSize (0.00s) === RUN TestCleanupQueue_ShouldProcess_MaxAge --- PASS: TestCleanupQueue_ShouldProcess_MaxAge (0.00s) === RUN TestCleanupQueue_Clear --- PASS: TestCleanupQueue_Clear (0.00s) === RUN TestCleanupQueue_OldestAge --- PASS: TestCleanupQueue_OldestAge (0.00s) === RUN TestCleanupQueue_TimeOrder --- PASS: TestCleanupQueue_TimeOrder (0.00s) === RUN TestCleanupQueue_DuplicateWithNewerTime --- PASS: TestCleanupQueue_DuplicateWithNewerTime (0.00s) === RUN TestCleanupQueue_Concurrent --- PASS: TestCleanupQueue_Concurrent (0.00s) === RUN Test_isUnderPath === RUN Test_isUnderPath/child_under_parent === RUN Test_isUnderPath/child_is_parent === RUN Test_isUnderPath/child_not_under_parent === RUN Test_isUnderPath/empty_parent === RUN Test_isUnderPath/root_parent === RUN Test_isUnderPath/parent_with_trailing_slash === RUN Test_isUnderPath/similar_prefix_but_not_under === RUN Test_isUnderPath/deeply_nested --- PASS: Test_isUnderPath (0.00s) --- PASS: Test_isUnderPath/child_under_parent (0.00s) --- PASS: Test_isUnderPath/child_is_parent (0.00s) --- PASS: Test_isUnderPath/child_not_under_parent (0.00s) --- PASS: Test_isUnderPath/empty_parent (0.00s) --- PASS: Test_isUnderPath/root_parent (0.00s) --- PASS: Test_isUnderPath/parent_with_trailing_slash (0.00s) --- PASS: Test_isUnderPath/similar_prefix_but_not_under (0.00s) --- PASS: Test_isUnderPath/deeply_nested (0.00s) === RUN Test_isUnderBucketPath === RUN Test_isUnderBucketPath/bucket_path_itself === RUN Test_isUnderBucketPath/bucket_directory === RUN Test_isUnderBucketPath/folder_in_bucket === RUN Test_isUnderBucketPath/nested_folder === RUN Test_isUnderBucketPath/outside_buckets === RUN Test_isUnderBucketPath/empty_bucket_path --- PASS: Test_isUnderBucketPath (0.00s) --- PASS: Test_isUnderBucketPath/bucket_path_itself (0.00s) --- PASS: Test_isUnderBucketPath/bucket_directory (0.00s) --- PASS: Test_isUnderBucketPath/folder_in_bucket (0.00s) --- PASS: Test_isUnderBucketPath/nested_folder (0.00s) --- PASS: Test_isUnderBucketPath/outside_buckets (0.00s) --- PASS: Test_isUnderBucketPath/empty_bucket_path (0.00s) === RUN Test_isMultipartUploadsPath === RUN Test_isMultipartUploadsPath/staging_root === RUN Test_isMultipartUploadsPath/upload_marker_dir === RUN Test_isMultipartUploadsPath/nested_under_staging === RUN Test_isMultipartUploadsPath/normal_folder === RUN Test_isMultipartUploadsPath/registry_own_uploads_dir === RUN Test_isMultipartUploadsPath/bucket_itself === RUN Test_isMultipartUploadsPath/outside_buckets === RUN Test_isMultipartUploadsPath/empty_bucket_path --- PASS: Test_isMultipartUploadsPath (0.00s) --- PASS: Test_isMultipartUploadsPath/staging_root (0.00s) --- PASS: Test_isMultipartUploadsPath/upload_marker_dir (0.00s) --- PASS: Test_isMultipartUploadsPath/nested_under_staging (0.00s) --- PASS: Test_isMultipartUploadsPath/normal_folder (0.00s) --- PASS: Test_isMultipartUploadsPath/registry_own_uploads_dir (0.00s) --- PASS: Test_isMultipartUploadsPath/bucket_itself (0.00s) --- PASS: Test_isMultipartUploadsPath/outside_buckets (0.00s) --- PASS: Test_isMultipartUploadsPath/empty_bucket_path (0.00s) === RUN TestEmptyFolderCleaner_OnDeleteEvent_skipsMultipartUploads --- PASS: TestEmptyFolderCleaner_OnDeleteEvent_skipsMultipartUploads (0.00s) === RUN TestEmptyFolderCleaner_executeCleanup_skipsMultipartUploads I0720 21:29:54.169815 empty_folder_cleaner.go:328 EmptyFolderCleaner: deleting empty folder /buckets/mybucket/folder (triggered by file.txt) --- PASS: TestEmptyFolderCleaner_executeCleanup_skipsMultipartUploads (0.00s) === RUN Test_autoRemoveEmptyFoldersEnabled === RUN Test_autoRemoveEmptyFoldersEnabled/no_attrs_defaults_enabled === RUN Test_autoRemoveEmptyFoldersEnabled/missing_key_defaults_enabled === RUN Test_autoRemoveEmptyFoldersEnabled/allow-empty_disables_cleanup === RUN Test_autoRemoveEmptyFoldersEnabled/explicit_false_keeps_cleanup_enabled --- PASS: Test_autoRemoveEmptyFoldersEnabled (0.00s) --- PASS: Test_autoRemoveEmptyFoldersEnabled/no_attrs_defaults_enabled (0.00s) --- PASS: Test_autoRemoveEmptyFoldersEnabled/missing_key_defaults_enabled (0.00s) --- PASS: Test_autoRemoveEmptyFoldersEnabled/allow-empty_disables_cleanup (0.00s) --- PASS: Test_autoRemoveEmptyFoldersEnabled/explicit_false_keeps_cleanup_enabled (0.00s) === RUN TestEmptyFolderCleaner_ownsFolder --- PASS: TestEmptyFolderCleaner_ownsFolder (0.00s) === RUN TestEmptyFolderCleaner_ownsFolder_singleServer --- PASS: TestEmptyFolderCleaner_ownsFolder_singleServer (0.00s) === RUN TestEmptyFolderCleaner_ownsFolder_emptyRing --- PASS: TestEmptyFolderCleaner_ownsFolder_emptyRing (0.00s) === RUN TestEmptyFolderCleaner_OnCreateEvent_cancelsCleanup --- PASS: TestEmptyFolderCleaner_OnCreateEvent_cancelsCleanup (0.00s) === RUN TestEmptyFolderCleaner_OnDeleteEvent_deduplication --- PASS: TestEmptyFolderCleaner_OnDeleteEvent_deduplication (0.00s) === RUN TestEmptyFolderCleaner_OnDeleteEvent_multipleFolders --- PASS: TestEmptyFolderCleaner_OnDeleteEvent_multipleFolders (0.00s) === RUN TestEmptyFolderCleaner_OnDeleteEvent_notOwner --- PASS: TestEmptyFolderCleaner_OnDeleteEvent_notOwner (0.00s) === RUN TestEmptyFolderCleaner_OnDeleteEvent_disabled --- PASS: TestEmptyFolderCleaner_OnDeleteEvent_disabled (0.00s) === RUN TestEmptyFolderCleaner_OnDeleteEvent_directoryDeletion --- PASS: TestEmptyFolderCleaner_OnDeleteEvent_directoryDeletion (0.00s) === RUN TestEmptyFolderCleaner_cachedCounts --- PASS: TestEmptyFolderCleaner_cachedCounts (0.00s) === RUN TestEmptyFolderCleaner_Stop --- PASS: TestEmptyFolderCleaner_Stop (0.00s) === RUN TestEmptyFolderCleaner_cacheEviction --- PASS: TestEmptyFolderCleaner_cacheEviction (0.00s) === RUN TestEmptyFolderCleaner_cacheEviction_skipsEntriesInQueue --- PASS: TestEmptyFolderCleaner_cacheEviction_skipsEntriesInQueue (0.00s) === RUN TestEmptyFolderCleaner_queueFIFOOrder --- PASS: TestEmptyFolderCleaner_queueFIFOOrder (0.00s) === RUN TestEmptyFolderCleaner_processCleanupQueue_onlyProcessesAgedItems I0720 21:29:54.171514 empty_folder_cleaner.go:328 EmptyFolderCleaner: deleting empty folder /buckets/test/folder1 (triggered by i1) I0720 21:29:54.171526 empty_folder_cleaner.go:328 EmptyFolderCleaner: deleting empty folder /buckets/test/folder2 (triggered by i2) --- PASS: TestEmptyFolderCleaner_processCleanupQueue_onlyProcessesAgedItems (0.00s) === RUN TestEmptyFolderCleaner_executeCleanup_bucketPolicyDisabledSkips --- PASS: TestEmptyFolderCleaner_executeCleanup_bucketPolicyDisabledSkips (0.00s) === RUN TestEmptyFolderCleaner_executeCleanup_directoryMarker === RUN TestEmptyFolderCleaner_executeCleanup_directoryMarker/skips_explicit_directory_marker === RUN TestEmptyFolderCleaner_executeCleanup_directoryMarker/deletes_implicit_empty_folder I0720 21:29:54.171723 empty_folder_cleaner.go:328 EmptyFolderCleaner: deleting empty folder /buckets/test/folder (triggered by triggered_item) --- PASS: TestEmptyFolderCleaner_executeCleanup_directoryMarker (0.00s) --- PASS: TestEmptyFolderCleaner_executeCleanup_directoryMarker/skips_explicit_directory_marker (0.00s) --- PASS: TestEmptyFolderCleaner_executeCleanup_directoryMarker/deletes_implicit_empty_folder (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/empty_folder_cleanup 0.035s === RUN TestStore --- PASS: TestStore (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/etcd 0.055s ? github.com/seaweedfs/seaweedfs/weed/filer/foundationdb [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/hbase [no test files] === RUN TestCreateAndFind I0720 21:29:54.234130 leveldb_store.go:48 filer store dir: /tmp/TestCreateAndFind1948094198/001 I0720 21:29:54.234354 file_util.go:29 Folder /tmp/TestCreateAndFind1948094198/001 Permission: -rwxr-xr-x I0720 21:29:54.237829 filer.go:182 create filer.store.id to -1113369549 --- PASS: TestCreateAndFind (0.04s) === RUN TestEmptyRoot I0720 21:29:54.240110 leveldb_store.go:48 filer store dir: /tmp/TestEmptyRoot3742775624/001 I0720 21:29:54.240129 file_util.go:29 Folder /tmp/TestEmptyRoot3742775624/001 Permission: -rwxr-xr-x I0720 21:29:54.240814 filer.go:182 create filer.store.id to -1980107080 --- PASS: TestEmptyRoot (0.00s) === RUN TestCreateEntryRecordsObjectSize I0720 21:29:54.258682 leveldb_store.go:48 filer store dir: /tmp/TestCreateEntryRecordsObjectSize4029206097/001 I0720 21:29:54.258704 file_util.go:29 Folder /tmp/TestCreateEntryRecordsObjectSize4029206097/001 Permission: -rwxr-xr-x I0720 21:29:54.259766 filer.go:182 create filer.store.id to 459209160 --- PASS: TestCreateEntryRecordsObjectSize (0.02s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/leveldb 0.129s === RUN TestCreateAndFind I0720 21:29:54.229530 leveldb2_store.go:43 filer store leveldb2 dir: /tmp/TestCreateAndFind1604357381/001 I0720 21:29:54.229822 file_util.go:29 Folder /tmp/TestCreateAndFind1604357381/001 Permission: -rwxr-xr-x I0720 21:29:54.235599 filer.go:182 create filer.store.id to 1906608960 --- PASS: TestCreateAndFind (0.02s) === RUN TestEmptyRoot I0720 21:29:54.236992 leveldb2_store.go:43 filer store leveldb2 dir: /tmp/TestEmptyRoot3627846540/001 I0720 21:29:54.237006 file_util.go:29 Folder /tmp/TestEmptyRoot3627846540/001 Permission: -rwxr-xr-x I0720 21:29:54.239409 filer.go:182 create filer.store.id to 1471415116 --- PASS: TestEmptyRoot (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/leveldb2 0.107s === RUN TestCreateAndFind I0720 21:29:54.235781 leveldb3_store.go:50 filer store leveldb3 dir: /tmp/TestCreateAndFind2014139444/001 I0720 21:29:54.235952 file_util.go:29 Folder /tmp/TestCreateAndFind2014139444/001 Permission: -rwxr-xr-x I0720 21:29:54.241620 filer.go:182 create filer.store.id to -2084888961 --- PASS: TestCreateAndFind (0.05s) === RUN TestEmptyRoot I0720 21:29:54.246202 leveldb3_store.go:50 filer store leveldb3 dir: /tmp/TestEmptyRoot1753416306/001 I0720 21:29:54.246218 file_util.go:29 Folder /tmp/TestEmptyRoot1753416306/001 Permission: -rwxr-xr-x I0720 21:29:54.246821 filer.go:182 create filer.store.id to -1749230882 --- PASS: TestEmptyRoot (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/leveldb3 0.113s ? github.com/seaweedfs/seaweedfs/weed/filer/mongodb [no test files] === RUN TestDefaultUpsertQueryUsesOnDuplicateKey --- PASS: TestDefaultUpsertQueryUsesOnDuplicateKey (0.00s) === RUN TestDefaultCreateTableQueryRendersValidSql --- PASS: TestDefaultCreateTableQueryRendersValidSql (0.00s) === RUN TestEmptyUpsertTemplateFallsBackToPlainInsert --- PASS: TestEmptyUpsertTemplateFallsBackToPlainInsert (0.00s) === RUN TestListSqlDefaultOrderingFollowsColumn --- PASS: TestListSqlDefaultOrderingFollowsColumn (0.00s) === RUN TestListSqlBinaryOrderingOnNonBinaryColumn --- PASS: TestListSqlBinaryOrderingOnNonBinaryColumn (0.00s) === RUN TestIsBinaryCollation --- PASS: TestIsBinaryCollation (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/mysql 0.043s ? github.com/seaweedfs/seaweedfs/weed/filer/mysql2 [no test files] === RUN TestManagerGrantAndConflict --- PASS: TestManagerGrantAndConflict (0.00s) === RUN TestManagerUnlockCleansEmptyKeyAndIndex --- PASS: TestManagerUnlockCleansEmptyKeyAndIndex (0.00s) === RUN TestManagerPartialUnlockKeepsIndex --- PASS: TestManagerPartialUnlockKeepsIndex (0.00s) === RUN TestManagerGetLk --- PASS: TestManagerGetLk (0.00s) === RUN TestManagerReleasePosixOwnerKeepsFlockAndIndex --- PASS: TestManagerReleasePosixOwnerKeepsFlockAndIndex (0.00s) === RUN TestManagerReleaseSessionReapsAcrossKeys --- PASS: TestManagerReleaseSessionReapsAcrossKeys (0.00s) === RUN TestManagerReapsOnlyStaleLeasedSessions --- PASS: TestManagerReapsOnlyStaleLeasedSessions (0.00s) === RUN TestManagerConcurrentFlockMutualExclusion --- PASS: TestManagerConcurrentFlockMutualExclusion (0.03s) === RUN TestNonOverlappingLocksFromDifferentOwners --- PASS: TestNonOverlappingLocksFromDifferentOwners (0.00s) === RUN TestOverlappingReadLocksFromDifferentOwners --- PASS: TestOverlappingReadLocksFromDifferentOwners (0.00s) === RUN TestOverlappingWriteReadConflict --- PASS: TestOverlappingWriteReadConflict (0.00s) === RUN TestOverlappingWriteWriteConflict --- PASS: TestOverlappingWriteWriteConflict (0.00s) === RUN TestSameOwnerUpgradeReadToWrite --- PASS: TestSameOwnerUpgradeReadToWrite (0.00s) === RUN TestSameOwnerDowngradeWriteToRead --- PASS: TestSameOwnerDowngradeWriteToRead (0.00s) === RUN TestLockCoalescing --- PASS: TestLockCoalescing (0.00s) === RUN TestLockSplitting --- PASS: TestLockSplitting (0.00s) === RUN TestConflictReportsHolder --- PASS: TestConflictReportsHolder (0.00s) === RUN TestConflictNoneForSharedReads --- PASS: TestConflictNoneForSharedReads (0.00s) === RUN TestConflictSameOwnerNone --- PASS: TestConflictSameOwnerNone (0.00s) === RUN TestReleaseOwner --- PASS: TestReleaseOwner (0.00s) === RUN TestFlockAndFcntlDoNotConflict --- PASS: TestFlockAndFcntlDoNotConflict (0.00s) === RUN TestReleasePosixOwnerKeepsFlock --- PASS: TestReleasePosixOwnerKeepsFlock (0.00s) === RUN TestReleaseFlockOwnerKeepsPosix --- PASS: TestReleaseFlockOwnerKeepsPosix (0.00s) === RUN TestHasPosixIgnoresMissingOwnerAndFlock --- PASS: TestHasPosixIgnoresMissingOwnerAndFlock (0.00s) === RUN TestWholeFileLock --- PASS: TestWholeFileLock (0.00s) === RUN TestReleaseNoExistingLocks --- PASS: TestReleaseNoExistingLocks (0.00s) === RUN TestSameOwnerReplaceDifferentType --- PASS: TestSameOwnerReplaceDifferentType (0.00s) === RUN TestNonAdjacentRangesNotCoalesced --- PASS: TestNonAdjacentRangesNotCoalesced (0.00s) === RUN TestAdjacencyNoOverflowAtMaxUint64 --- PASS: TestAdjacencyNoOverflowAtMaxUint64 (0.00s) === RUN TestTwoSessionsSameOwnerDoNotAlias --- PASS: TestTwoSessionsSameOwnerDoNotAlias (0.00s) === RUN TestReleaseSessionReapsOnlyThatSession --- PASS: TestReleaseSessionReapsOnlyThatSession (0.00s) === RUN TestEmptyAfterReleasingAll --- PASS: TestEmptyAfterReleasingAll (0.00s) === RUN TestReassertRebuildsOnFreshOwner --- PASS: TestReassertRebuildsOnFreshOwner (0.00s) === RUN TestReassertIdempotent --- PASS: TestReassertIdempotent (0.00s) === RUN TestReassertReportsConflict --- PASS: TestReassertReportsConflict (0.00s) === RUN TestReassertRenewsLease --- PASS: TestReassertRenewsLease (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/posixlock 0.039s === RUN TestConfigureListOrderingSkipsCockroachDB --- PASS: TestConfigureListOrderingSkipsCockroachDB (0.00s) === RUN TestConfigureListOrderingForcesBinaryOnLocalePostgres W0720 21:29:54.201956 postgres_collation.go:29 postgres: filemeta.name collation "en_US.UTF-8" is not byte-ordered, so S3 list order is not byte-lexicographic and clients that merge sorted listings may report spurious diffs. Falling back to a slower COLLATE "C" sort; declare the name column COLLATE "C" (or use a C/C.UTF-8 database collation) for correct, indexed ordering. --- PASS: TestConfigureListOrderingForcesBinaryOnLocalePostgres (0.00s) === RUN TestDefaultUpsertQueryIsConflictSafe --- PASS: TestDefaultUpsertQueryIsConflictSafe (0.00s) === RUN TestDefaultCreateTableQueryRendersValidSql --- PASS: TestDefaultCreateTableQueryRendersValidSql (0.00s) === RUN TestEmptyUpsertTemplateFallsBackToPlainInsert --- PASS: TestEmptyUpsertTemplateFallsBackToPlainInsert (0.00s) === RUN TestListSqlDefaultOrderingFollowsColumn --- PASS: TestListSqlDefaultOrderingFollowsColumn (0.00s) === RUN TestListSqlBinaryOrderingOnNonBinaryColumn --- PASS: TestListSqlBinaryOrderingOnNonBinaryColumn (0.00s) === RUN TestIsByteOrderedCollation --- PASS: TestIsByteOrderedCollation (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/filer/postgres 0.092s ? github.com/seaweedfs/seaweedfs/weed/filer/postgres2 [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/redis [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/redis2 [no test files] testing: warning: no tests to run PASS ok github.com/seaweedfs/seaweedfs/weed/filer/redis3 0.070s [no tests to run] ? github.com/seaweedfs/seaweedfs/weed/filer/sqlite [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/store_test [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/tarantool [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/tikv [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer/ydb [no test files] ? github.com/seaweedfs/seaweedfs/weed/filer_client [no test files] === RUN TestCompressRotated_Toggle --- PASS: TestCompressRotated_Toggle (0.00s) === RUN TestCompressFile --- PASS: TestCompressFile (0.00s) === RUN TestCompressFile_AlreadyGz --- PASS: TestCompressFile_AlreadyGz (0.00s) === RUN TestCompressFile_NonExistent --- PASS: TestCompressFile_NonExistent (0.00s) === RUN TestCompressFile_EmptyFile --- PASS: TestCompressFile_EmptyFile (0.00s) === RUN TestCompressFile_LargeContent --- PASS: TestCompressFile_LargeContent (0.01s) === RUN TestJSONMode_Toggle --- PASS: TestJSONMode_Toggle (0.00s) === RUN TestJSONMode_Output {"ts":"2026-03-19T15:30:00Z","level":"INFO","file":"glog_json_test.go","line":39,"msg":"hello json"} --- PASS: TestJSONMode_Output (0.00s) === RUN TestJSONMode_AllLevels {"ts":"2026-07-20T21:29:54.168060893Z","level":"INFO","file":"glog_json_test.go","line":78,"msg":"info msg"} {"ts":"2026-07-20T21:29:54.168066304Z","level":"WARNING","file":"glog_json_test.go","line":79,"msg":"warn msg"} {"ts":"2026-07-20T21:29:54.168071904Z","level":"ERROR","file":"glog_json_test.go","line":80,"msg":"error msg"} --- PASS: TestJSONMode_AllLevels (0.00s) === RUN TestJSONMode_Infof {"ts":"2026-07-20T21:29:54.168124484Z","level":"INFO","file":"glog_json_test.go","line":106,"msg":"count=42 name=test"} --- PASS: TestJSONMode_Infof (0.00s) === RUN TestJSONEscapeString --- PASS: TestJSONEscapeString (0.00s) === RUN TestItoa --- PASS: TestItoa (0.00s) === RUN TestJSONMode_TextFallback I0720 21:29:54.168183 glog_json_test.go:167 text mode --- PASS: TestJSONMode_TextFallback (0.00s) === RUN TestShortHostname --- PASS: TestShortHostname (0.00s) === RUN TestInfo I0720 21:29:54.168280 glog_test.go:92 test --- PASS: TestInfo (0.00s) === RUN TestInfoDepth I0720 21:29:54.168296 glog_test.go:109 depth-test0 I0720 21:29:54.168299 glog_test.go:110 depth-test1 --- PASS: TestInfoDepth (0.00s) === RUN TestCopyStandardLogToPanic --- PASS: TestCopyStandardLogToPanic (0.00s) === RUN TestStandardLog I0720 21:29:54.168346 glog_test.go:163 test --- PASS: TestStandardLog (0.00s) === RUN TestHeader I0102 15:04:05.067890 glog_test.go:181 test --- PASS: TestHeader (0.00s) === RUN TestError E0720 21:29:54.168427 glog_test.go:202 test --- PASS: TestError (0.00s) === RUN TestWarning W0720 21:29:54.168441 glog_test.go:224 test --- PASS: TestWarning (0.00s) === RUN TestV I0720 21:29:54.168459 glog_test.go:243 test --- PASS: TestV (0.00s) === RUN TestVmoduleOn I0720 21:29:54.168492 glog_test.go:267 test --- PASS: TestVmoduleOn (0.00s) === RUN TestVmoduleOff --- PASS: TestVmoduleOff (0.00s) === RUN TestVmoduleGlob --- PASS: TestVmoduleGlob (0.00s) === RUN TestRollover I0720 21:29:54.168569 glog_test.go:337 x I0720 21:29:54.177493 glog_test.go:351 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx I0720 21:29:55.180738 glog_test.go:364 x --- PASS: TestRollover (1.01s) === RUN TestTimeBasedRollover I0720 21:29:55.182453 glog_test.go:393 x I0720 21:59:55.180767 glog_test.go:407 still within interval I0720 22:30:55.180767 glog_test.go:417 past interval --- PASS: TestTimeBasedRollover (0.00s) === RUN TestLogBacktraceAt I0720 21:29:55.182991 glog_test.go:448 we want a stack trace here goroutine 75 [running]: github.com/seaweedfs/seaweedfs/weed/glog.stacks(0x0) /startdir/src/seaweedfs-4.40/weed/glog/glog.go:832 +0x85 github.com/seaweedfs/seaweedfs/weed/glog.(*loggingT).output(0x8f0e20, 0x0, 0x1ce1d06441c0, {0x71aed5?, 0x1?}, 0x0?, 0x0) /startdir/src/seaweedfs-4.40/weed/glog/glog.go:730 +0xe5 github.com/seaweedfs/seaweedfs/weed/glog.(*loggingT).printDepth(0x8f0e20, 0x0, 0x1, {0x1ce1d0427e30, 0x1, 0x1}) /startdir/src/seaweedfs-4.40/weed/glog/glog.go:667 +0x10c github.com/seaweedfs/seaweedfs/weed/glog.(*loggingT).print(...) /startdir/src/seaweedfs-4.40/weed/glog/glog.go:654 github.com/seaweedfs/seaweedfs/weed/glog.Info(...) /startdir/src/seaweedfs-4.40/weed/glog/glog.go:1149 github.com/seaweedfs/seaweedfs/weed/glog.TestLogBacktraceAt(0x1ce1d08fe248) /startdir/src/seaweedfs-4.40/weed/glog/glog_test.go:448 +0x40b testing.tRunner(0x1ce1d08fe248, 0x6c0ef0) /usr/lib/go/src/testing/testing.go:2036 +0xea created by testing.(*T).Run in goroutine 1 /usr/lib/go/src/testing/testing.go:2101 +0x4c5 --- PASS: TestLogBacktraceAt (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/glog 1.042s === RUN TestRedactSensitiveFormValues --- PASS: TestRedactSensitiveFormValues (0.00s) === RUN TestListUsersResponseXMLOrdering --- PASS: TestListUsersResponseXMLOrdering (0.00s) === RUN TestErrorResponseXMLUsesTopLevelRequestId --- PASS: TestErrorResponseXMLUsesTopLevelRequestId (0.00s) === RUN TestValidateCallerSuppliedAccessKeyId === RUN TestValidateCallerSuppliedAccessKeyId/empty === RUN TestValidateCallerSuppliedAccessKeyId/three_chars === RUN TestValidateCallerSuppliedAccessKeyId/four_chars_ok === RUN TestValidateCallerSuppliedAccessKeyId/mixed_case_alnum === RUN TestValidateCallerSuppliedAccessKeyId/128_chars_ok === RUN TestValidateCallerSuppliedAccessKeyId/129_chars_too_long === RUN TestValidateCallerSuppliedAccessKeyId/slash_rejected === RUN TestValidateCallerSuppliedAccessKeyId/equals_rejected === RUN TestValidateCallerSuppliedAccessKeyId/dash_rejected === RUN TestValidateCallerSuppliedAccessKeyId/underscore_rejected === RUN TestValidateCallerSuppliedAccessKeyId/unicode_rejected --- PASS: TestValidateCallerSuppliedAccessKeyId (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/empty (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/three_chars (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/four_chars_ok (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/mixed_case_alnum (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/128_chars_ok (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/129_chars_too_long (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/slash_rejected (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/equals_rejected (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/dash_rejected (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/underscore_rejected (0.00s) --- PASS: TestValidateCallerSuppliedAccessKeyId/unicode_rejected (0.00s) === RUN TestValidateCallerSuppliedSecretAccessKey === RUN TestValidateCallerSuppliedSecretAccessKey/empty === RUN TestValidateCallerSuppliedSecretAccessKey/seven_chars === RUN TestValidateCallerSuppliedSecretAccessKey/eight_chars_ok === RUN TestValidateCallerSuppliedSecretAccessKey/128_chars_ok === RUN TestValidateCallerSuppliedSecretAccessKey/129_chars_too_long === RUN TestValidateCallerSuppliedSecretAccessKey/non-alnum_allowed_in_secret --- PASS: TestValidateCallerSuppliedSecretAccessKey (0.00s) --- PASS: TestValidateCallerSuppliedSecretAccessKey/empty (0.00s) --- PASS: TestValidateCallerSuppliedSecretAccessKey/seven_chars (0.00s) --- PASS: TestValidateCallerSuppliedSecretAccessKey/eight_chars_ok (0.00s) --- PASS: TestValidateCallerSuppliedSecretAccessKey/128_chars_ok (0.00s) --- PASS: TestValidateCallerSuppliedSecretAccessKey/129_chars_too_long (0.00s) --- PASS: TestValidateCallerSuppliedSecretAccessKey/non-alnum_allowed_in_secret (0.00s) === RUN TestFindAccessKeyOwner === RUN TestFindAccessKeyOwner/matches_user === RUN TestFindAccessKeyOwner/matches_service_account === RUN TestFindAccessKeyOwner/no_match === RUN TestFindAccessKeyOwner/empty_key === RUN TestFindAccessKeyOwner/nil_config --- PASS: TestFindAccessKeyOwner (0.00s) --- PASS: TestFindAccessKeyOwner/matches_user (0.00s) --- PASS: TestFindAccessKeyOwner/matches_service_account (0.00s) --- PASS: TestFindAccessKeyOwner/no_match (0.00s) --- PASS: TestFindAccessKeyOwner/empty_key (0.00s) --- PASS: TestFindAccessKeyOwner/nil_config (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iam 0.029s === RUN TestGetProviderByIssuerAndAccount === RUN TestGetProviderByIssuerAndAccount/matching_account_returns_scoped_record === RUN TestGetProviderByIssuerAndAccount/unknown_account_is_rejected === RUN TestGetProviderByIssuerAndAccount/global_provider_satisfies_any_account === RUN TestGetProviderByIssuerAndAccount/empty_caller_account_accepts_any_record --- PASS: TestGetProviderByIssuerAndAccount (0.00s) --- PASS: TestGetProviderByIssuerAndAccount/matching_account_returns_scoped_record (0.00s) --- PASS: TestGetProviderByIssuerAndAccount/unknown_account_is_rejected (0.00s) --- PASS: TestGetProviderByIssuerAndAccount/global_provider_satisfies_any_account (0.00s) --- PASS: TestGetProviderByIssuerAndAccount/empty_caller_account_accepts_any_record (0.00s) === RUN TestPolicyVariableSubstitution --- PASS: TestPolicyVariableSubstitution (0.00s) === RUN TestConditionWithNumericComparison === RUN TestConditionWithNumericComparison/duration_within_limit === RUN TestConditionWithNumericComparison/duration_at_limit === RUN TestConditionWithNumericComparison/duration_exceeding_limit --- PASS: TestConditionWithNumericComparison (0.00s) --- PASS: TestConditionWithNumericComparison/duration_within_limit (0.00s) --- PASS: TestConditionWithNumericComparison/duration_at_limit (0.00s) --- PASS: TestConditionWithNumericComparison/duration_exceeding_limit (0.00s) === RUN TestMultipleConditionOperators === RUN TestMultipleConditionOperators/all_conditions_match === RUN TestMultipleConditionOperators/aud_mismatch === RUN TestMultipleConditionOperators/sub_pattern_mismatch --- PASS: TestMultipleConditionOperators (0.00s) --- PASS: TestMultipleConditionOperators/all_conditions_match (0.00s) --- PASS: TestMultipleConditionOperators/aud_mismatch (0.00s) --- PASS: TestMultipleConditionOperators/sub_pattern_mismatch (0.00s) === RUN TestFullOIDCWorkflow === RUN TestFullOIDCWorkflow/successful_role_assumption_with_policy_validation === RUN TestFullOIDCWorkflow/role_assumption_denied_by_trust_policy === RUN TestFullOIDCWorkflow/invalid_token_rejected --- PASS: TestFullOIDCWorkflow (0.00s) --- PASS: TestFullOIDCWorkflow/successful_role_assumption_with_policy_validation (0.00s) --- PASS: TestFullOIDCWorkflow/role_assumption_denied_by_trust_policy (0.00s) --- PASS: TestFullOIDCWorkflow/invalid_token_rejected (0.00s) === RUN TestFullLDAPWorkflow === RUN TestFullLDAPWorkflow/successful_LDAP_role_assumption === RUN TestFullLDAPWorkflow/invalid_LDAP_credentials --- PASS: TestFullLDAPWorkflow (0.00s) --- PASS: TestFullLDAPWorkflow/successful_LDAP_role_assumption (0.00s) --- PASS: TestFullLDAPWorkflow/invalid_LDAP_credentials (0.00s) === RUN TestPolicyEnforcement === RUN TestPolicyEnforcement/allow_read_access === RUN TestPolicyEnforcement/allow_list_bucket === RUN TestPolicyEnforcement/deny_write_access === RUN TestPolicyEnforcement/deny_delete_access === RUN TestPolicyEnforcement/deny_filer_access --- PASS: TestPolicyEnforcement (0.00s) --- PASS: TestPolicyEnforcement/allow_read_access (0.00s) --- PASS: TestPolicyEnforcement/allow_list_bucket (0.00s) --- PASS: TestPolicyEnforcement/deny_write_access (0.00s) --- PASS: TestPolicyEnforcement/deny_delete_access (0.00s) --- PASS: TestPolicyEnforcement/deny_filer_access (0.00s) === RUN TestSessionPolicyBoundary --- PASS: TestSessionPolicyBoundary (0.00s) === RUN TestAssumeRoleWithWebIdentitySessionPolicy --- PASS: TestAssumeRoleWithWebIdentitySessionPolicy (0.00s) === RUN TestAssumeRoleWithCredentialsSessionPolicy --- PASS: TestAssumeRoleWithCredentialsSessionPolicy (0.00s) === RUN TestSessionExpiration --- PASS: TestSessionExpiration (0.01s) === RUN TestTrustPolicyValidation === RUN TestTrustPolicyValidation/OIDC_user_allowed_by_trust_policy === RUN TestTrustPolicyValidation/LDAP_user_allowed_by_different_role === RUN TestTrustPolicyValidation/Wrong_provider_for_role --- PASS: TestTrustPolicyValidation (0.00s) --- PASS: TestTrustPolicyValidation/OIDC_user_allowed_by_trust_policy (0.00s) --- PASS: TestTrustPolicyValidation/LDAP_user_allowed_by_different_role (0.00s) --- PASS: TestTrustPolicyValidation/Wrong_provider_for_role (0.00s) === RUN TestTrustPolicyWildcardPrincipal === RUN TestTrustPolicyWildcardPrincipal/Wildcard_federated_principal_allows_any_provider === RUN TestTrustPolicyWildcardPrincipal/Wildcard_in_array_allows_any_provider === RUN TestTrustPolicyWildcardPrincipal/Plain_wildcard_allows_any_provider_(regression) === RUN TestTrustPolicyWildcardPrincipal/Non-wildcard_federated_principal_requires_matching_provider === RUN TestTrustPolicyWildcardPrincipal/Interface_array_principal_works_correctly --- PASS: TestTrustPolicyWildcardPrincipal (0.00s) --- PASS: TestTrustPolicyWildcardPrincipal/Wildcard_federated_principal_allows_any_provider (0.00s) --- PASS: TestTrustPolicyWildcardPrincipal/Wildcard_in_array_allows_any_provider (0.00s) --- PASS: TestTrustPolicyWildcardPrincipal/Plain_wildcard_allows_any_provider_(regression) (0.00s) --- PASS: TestTrustPolicyWildcardPrincipal/Non-wildcard_federated_principal_requires_matching_provider (0.00s) --- PASS: TestTrustPolicyWildcardPrincipal/Interface_array_principal_works_correctly (0.00s) === RUN TestOIDCClaimsTrustPolicy --- PASS: TestOIDCClaimsTrustPolicy (0.00s) === RUN TestIsActionAllowed_RegisteredUserWithoutPoliciesIsDenied --- PASS: TestIsActionAllowed_RegisteredUserWithoutPoliciesIsDenied (0.00s) === RUN TestIsActionAllowed_RegisteredRoleWithoutPoliciesIsDenied --- PASS: TestIsActionAllowed_RegisteredRoleWithoutPoliciesIsDenied (0.00s) === RUN TestIsActionAllowed_RegisteredUserWithNonMatchingPolicyIsDenied --- PASS: TestIsActionAllowed_RegisteredUserWithNonMatchingPolicyIsDenied (0.00s) === RUN TestOIDCAuditLifecycleEventsAreEmitted --- PASS: TestOIDCAuditLifecycleEventsAreEmitted (0.00s) === RUN TestOIDCAuditDefaultsToGlogSink I0720 21:29:54.239906 oidc_provider_audit.go:66 oidc-audit: {"type":"Create","arn":"arn:aws:iam::111122223333:oidc-provider/idp.example","url":"https://idp.example","occurredAt":"2026-07-20T21:29:54.239838258Z"} --- PASS: TestOIDCAuditDefaultsToGlogSink (0.02s) === RUN TestStaticConfigSeedsProviderStore --- PASS: TestStaticConfigSeedsProviderStore (0.00s) === RUN TestStaticConfigMirrorsThumbprintsAndAdvancedFields --- PASS: TestStaticConfigMirrorsThumbprintsAndAdvancedFields (0.00s) === RUN TestStoreNotConfiguredReturnsClearError --- PASS: TestStoreNotConfiguredReturnsClearError (0.00s) === RUN TestMemoryStoreCRUD --- PASS: TestMemoryStoreCRUD (0.00s) === RUN TestMemoryStoreGetByIssuerNormalizesHost --- PASS: TestMemoryStoreGetByIssuerNormalizesHost (0.00s) === RUN TestMemoryStoreGetByIssuerMissing --- PASS: TestMemoryStoreGetByIssuerMissing (0.00s) === RUN TestDeriveOIDCProviderARN === RUN TestDeriveOIDCProviderARN/google === RUN TestDeriveOIDCProviderARN/EKS_with_path === RUN TestDeriveOIDCProviderARN/uppercase_host_normalized === RUN TestDeriveOIDCProviderARN/trailing_slash_trimmed === RUN TestDeriveOIDCProviderARN/empty_account_allowed --- PASS: TestDeriveOIDCProviderARN (0.00s) --- PASS: TestDeriveOIDCProviderARN/google (0.00s) --- PASS: TestDeriveOIDCProviderARN/EKS_with_path (0.00s) --- PASS: TestDeriveOIDCProviderARN/uppercase_host_normalized (0.00s) --- PASS: TestDeriveOIDCProviderARN/trailing_slash_trimmed (0.00s) --- PASS: TestDeriveOIDCProviderARN/empty_account_allowed (0.00s) === RUN TestDeriveOIDCProviderARNRejectsBadInput --- PASS: TestDeriveOIDCProviderARNRejectsBadInput (0.00s) === RUN TestStoreRejectsEmptyARN --- PASS: TestStoreRejectsEmptyARN (0.00s) === RUN TestStoreRejectsNil --- PASS: TestStoreRejectsNil (0.00s) === RUN TestNormalizeIssuerRejectsEmpty --- PASS: TestNormalizeIssuerRejectsEmpty (0.00s) === RUN TestNormalizeIssuerHandlesNonURL --- PASS: TestNormalizeIssuerHandlesNonURL (0.00s) === RUN TestCapDurationByRole === RUN TestCapDurationByRole/no_cap,_no_request === RUN TestCapDurationByRole/no_cap,_with_request === RUN TestCapDurationByRole/cap_only,_no_request_->_nil_so_STS_default_applies === RUN TestCapDurationByRole/request_below_cap_->_request === RUN TestCapDurationByRole/request_equal_cap_->_request === RUN TestCapDurationByRole/request_above_cap_->_cap --- PASS: TestCapDurationByRole (0.00s) --- PASS: TestCapDurationByRole/no_cap,_no_request (0.00s) --- PASS: TestCapDurationByRole/no_cap,_with_request (0.00s) --- PASS: TestCapDurationByRole/cap_only,_no_request_->_nil_so_STS_default_applies (0.00s) --- PASS: TestCapDurationByRole/request_below_cap_->_request (0.00s) --- PASS: TestCapDurationByRole/request_equal_cap_->_request (0.00s) --- PASS: TestCapDurationByRole/request_above_cap_->_cap (0.00s) === RUN TestMemoryRoleStore --- PASS: TestMemoryRoleStore (0.00s) === RUN TestRoleStoreConfiguration --- PASS: TestRoleStoreConfiguration (0.00s) === RUN TestDistributedIAMManagerWithRoleStore --- PASS: TestDistributedIAMManagerWithRoleStore (0.00s) === RUN TestMemoryRevocationStoreCRUD --- PASS: TestMemoryRevocationStoreCRUD (0.00s) === RUN TestMemoryRevocationStoreRejectsBadInput --- PASS: TestMemoryRevocationStoreRejectsBadInput (0.00s) === RUN TestMemoryRevocationStorePurge --- PASS: TestMemoryRevocationStorePurge (0.00s) === RUN TestIAMManagerRevocationDefaultIsNoop --- PASS: TestIAMManagerRevocationDefaultIsNoop (0.00s) === RUN TestIAMManagerRevocationFlow --- PASS: TestIAMManagerRevocationFlow (0.00s) === RUN TestTrustPolicyAWSUserPrincipal === RUN TestTrustPolicyAWSUserPrincipal/matching_user_principal_should_be_allowed === RUN TestTrustPolicyAWSUserPrincipal/non-matching_user_principal_should_be_denied --- PASS: TestTrustPolicyAWSUserPrincipal (0.00s) --- PASS: TestTrustPolicyAWSUserPrincipal/matching_user_principal_should_be_allowed (0.00s) --- PASS: TestTrustPolicyAWSUserPrincipal/non-matching_user_principal_should_be_denied (0.00s) === RUN TestTrustPolicyAWSWildcardPrincipal --- PASS: TestTrustPolicyAWSWildcardPrincipal (0.00s) === RUN TestTrustPolicyAWSUserArrayPrincipal === RUN TestTrustPolicyAWSUserArrayPrincipal/first_listed_user_is_allowed === RUN TestTrustPolicyAWSUserArrayPrincipal/second_listed_user_is_allowed === RUN TestTrustPolicyAWSUserArrayPrincipal/unlisted_user_is_denied --- PASS: TestTrustPolicyAWSUserArrayPrincipal (0.00s) --- PASS: TestTrustPolicyAWSUserArrayPrincipal/first_listed_user_is_allowed (0.00s) --- PASS: TestTrustPolicyAWSUserArrayPrincipal/second_listed_user_is_allowed (0.00s) --- PASS: TestTrustPolicyAWSUserArrayPrincipal/unlisted_user_is_denied (0.00s) === RUN TestTrustPolicyPlainStringPrincipal --- PASS: TestTrustPolicyPlainStringPrincipal (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iam/integration 0.128s ? github.com/seaweedfs/seaweedfs/weed/iam/ldap [no test files] === RUN TestDiscoveryHappyPath --- PASS: TestDiscoveryHappyPath (0.00s) === RUN TestDiscoveryFallback404 --- PASS: TestDiscoveryFallback404 (0.01s) === RUN TestDiscoveryDisabledByExplicitJWKSUri --- PASS: TestDiscoveryDisabledByExplicitJWKSUri (0.01s) === RUN TestDiscoveryRejectsIssuerMismatch --- PASS: TestDiscoveryRejectsIssuerMismatch (0.00s) === RUN TestDiscoveryRejectsMissingIssuer --- PASS: TestDiscoveryRejectsMissingIssuer (0.00s) === RUN TestOIDCProviderInitialization === RUN TestOIDCProviderInitialization/valid_config === RUN TestOIDCProviderInitialization/missing_issuer === RUN TestOIDCProviderInitialization/missing_client_id === RUN TestOIDCProviderInitialization/invalid_issuer_url --- PASS: TestOIDCProviderInitialization (0.00s) --- PASS: TestOIDCProviderInitialization/valid_config (0.00s) --- PASS: TestOIDCProviderInitialization/missing_issuer (0.00s) --- PASS: TestOIDCProviderInitialization/missing_client_id (0.00s) --- PASS: TestOIDCProviderInitialization/invalid_issuer_url (0.00s) === RUN TestOIDCProviderJWTValidation === RUN TestOIDCProviderJWTValidation/valid_token === RUN TestOIDCProviderJWTValidation/valid_token_with_array_audience === RUN TestOIDCProviderJWTValidation/expired_token === RUN TestOIDCProviderJWTValidation/invalid_signature --- PASS: TestOIDCProviderJWTValidation (0.41s) --- PASS: TestOIDCProviderJWTValidation/valid_token (0.00s) --- PASS: TestOIDCProviderJWTValidation/valid_token_with_array_audience (0.00s) --- PASS: TestOIDCProviderJWTValidation/expired_token (0.00s) --- PASS: TestOIDCProviderJWTValidation/invalid_signature (0.37s) === RUN TestOIDCProviderJWTValidationECDSA === RUN TestOIDCProviderJWTValidationECDSA/valid_token === RUN TestOIDCProviderJWTValidationECDSA/expired_token === RUN TestOIDCProviderJWTValidationECDSA/invalid_signature === RUN TestOIDCProviderJWTValidationECDSA/invalid_issuer === RUN TestOIDCProviderJWTValidationECDSA/invalid_audience --- PASS: TestOIDCProviderJWTValidationECDSA (0.00s) --- PASS: TestOIDCProviderJWTValidationECDSA/valid_token (0.00s) --- PASS: TestOIDCProviderJWTValidationECDSA/expired_token (0.00s) --- PASS: TestOIDCProviderJWTValidationECDSA/invalid_signature (0.00s) --- PASS: TestOIDCProviderJWTValidationECDSA/invalid_issuer (0.00s) --- PASS: TestOIDCProviderJWTValidationECDSA/invalid_audience (0.00s) === RUN TestOIDCProviderAuthentication === RUN TestOIDCProviderAuthentication/successful_authentication === RUN TestOIDCProviderAuthentication/successful_authentication_with_additional_attributes === RUN TestOIDCProviderAuthentication/authentication_with_invalid_token --- PASS: TestOIDCProviderAuthentication (0.14s) --- PASS: TestOIDCProviderAuthentication/successful_authentication (0.01s) --- PASS: TestOIDCProviderAuthentication/successful_authentication_with_additional_attributes (0.00s) --- PASS: TestOIDCProviderAuthentication/authentication_with_invalid_token (0.00s) === RUN TestOIDCProviderUserInfo === RUN TestOIDCProviderUserInfo/get_user_info_with_access_token === RUN TestOIDCProviderUserInfo/get_admin_user_info === RUN TestOIDCProviderUserInfo/get_user_info_without_token === RUN TestOIDCProviderUserInfo/get_user_info_with_invalid_token === RUN TestOIDCProviderUserInfo/get_user_info_with_custom_claims_mapping === RUN TestOIDCProviderUserInfo/get_user_info_with_empty_id --- PASS: TestOIDCProviderUserInfo (0.00s) --- PASS: TestOIDCProviderUserInfo/get_user_info_with_access_token (0.00s) --- PASS: TestOIDCProviderUserInfo/get_admin_user_info (0.00s) --- PASS: TestOIDCProviderUserInfo/get_user_info_without_token (0.00s) --- PASS: TestOIDCProviderUserInfo/get_user_info_with_invalid_token (0.00s) --- PASS: TestOIDCProviderUserInfo/get_user_info_with_custom_claims_mapping (0.00s) --- PASS: TestOIDCProviderUserInfo/get_user_info_with_empty_id (0.00s) === RUN TestExtractClaimPoliciesString --- PASS: TestExtractClaimPoliciesString (0.00s) === RUN TestExtractClaimPoliciesCommaSeparated --- PASS: TestExtractClaimPoliciesCommaSeparated (0.00s) === RUN TestExtractClaimPoliciesArray --- PASS: TestExtractClaimPoliciesArray (0.00s) === RUN TestExtractClaimPoliciesMissing --- PASS: TestExtractClaimPoliciesMissing (0.00s) === RUN TestExtractClaimPoliciesEmptyClaimName --- PASS: TestExtractClaimPoliciesEmptyClaimName (0.00s) === RUN TestExtractClaimPoliciesUnsupportedShape --- PASS: TestExtractClaimPoliciesUnsupportedShape (0.00s) === RUN TestExtractPrincipalTagsObjectShape --- PASS: TestExtractPrincipalTagsObjectShape (0.00s) === RUN TestExtractPrincipalTagsArrayValueTakesFirst --- PASS: TestExtractPrincipalTagsArrayValueTakesFirst (0.00s) === RUN TestExtractPrincipalTagsAbsent --- PASS: TestExtractPrincipalTagsAbsent (0.00s) === RUN TestExtractPrincipalTagsWrongShape --- PASS: TestExtractPrincipalTagsWrongShape (0.00s) === RUN TestFilterPrincipalTagsAllowlist === RUN TestFilterPrincipalTagsAllowlist/empty_allowlist_denies_all === RUN TestFilterPrincipalTagsAllowlist/partial_allowlist === RUN TestFilterPrincipalTagsAllowlist/allowlist_with_no_matches === RUN TestFilterPrincipalTagsAllowlist/case-insensitive_on_key --- PASS: TestFilterPrincipalTagsAllowlist (0.00s) --- PASS: TestFilterPrincipalTagsAllowlist/empty_allowlist_denies_all (0.00s) --- PASS: TestFilterPrincipalTagsAllowlist/partial_allowlist (0.00s) --- PASS: TestFilterPrincipalTagsAllowlist/allowlist_with_no_matches (0.00s) --- PASS: TestFilterPrincipalTagsAllowlist/case-insensitive_on_key (0.00s) === RUN TestThumbprintMatchAccepted W0720 21:29:54.747223 oidc_provider.go:370 OIDC provider "thumbprint-test" is configured to skip TLS verification. This is insecure and should not be used in production. --- PASS: TestThumbprintMatchAccepted (0.01s) === RUN TestThumbprintMismatchRejected W0720 21:29:54.756691 oidc_provider.go:370 OIDC provider "thumbprint-test" is configured to skip TLS verification. This is insecure and should not be used in production. 2026/07/20 21:29:54 http: TLS handshake error from 127.0.0.1:57620: remote error: tls: bad certificate 2026/07/20 21:29:54 http: TLS handshake error from 127.0.0.1:57630: read tcp 127.0.0.1:45327->127.0.0.1:57630: use of closed network connection --- PASS: TestThumbprintMismatchRejected (0.00s) === RUN TestThumbprintAllowlistAcceptsMixedCase W0720 21:29:54.761209 oidc_provider.go:370 OIDC provider "thumbprint-test" is configured to skip TLS verification. This is insecure and should not be used in production. --- PASS: TestThumbprintAllowlistAcceptsMixedCase (0.01s) === RUN TestThumbprintEmptyAllowlistSkipsCheck W0720 21:29:54.771250 oidc_provider.go:370 OIDC provider "thumbprint-test" is configured to skip TLS verification. This is insecure and should not be used in production. --- PASS: TestThumbprintEmptyAllowlistSkipsCheck (0.01s) === RUN TestThumbprintSkipVerifyHonoured W0720 21:29:54.777121 oidc_provider.go:370 OIDC provider "skip" is configured to skip TLS verification. This is insecure and should not be used in production. --- PASS: TestThumbprintSkipVerifyHonoured (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iam/oidc 0.633s === RUN TestAWSIAMMatch === RUN TestAWSIAMMatch/case_insensitive_exact_match === RUN TestAWSIAMMatch/case_insensitive_wildcard_match === RUN TestAWSIAMMatch/AWS_username_variable_expansion === RUN TestAWSIAMMatch/SAML_username_variable_expansion === RUN TestAWSIAMMatch/OIDC_subject_variable_expansion === RUN TestAWSIAMMatch/case_insensitive_with_variable === RUN TestAWSIAMMatch/universal_wildcard === RUN TestAWSIAMMatch/question_mark_wildcard === RUN TestAWSIAMMatch/no_match_different_pattern === RUN TestAWSIAMMatch/variable_not_expanded_due_to_missing_context --- PASS: TestAWSIAMMatch (0.00s) --- PASS: TestAWSIAMMatch/case_insensitive_exact_match (0.00s) --- PASS: TestAWSIAMMatch/case_insensitive_wildcard_match (0.00s) --- PASS: TestAWSIAMMatch/AWS_username_variable_expansion (0.00s) --- PASS: TestAWSIAMMatch/SAML_username_variable_expansion (0.00s) --- PASS: TestAWSIAMMatch/OIDC_subject_variable_expansion (0.00s) --- PASS: TestAWSIAMMatch/case_insensitive_with_variable (0.00s) --- PASS: TestAWSIAMMatch/universal_wildcard (0.00s) --- PASS: TestAWSIAMMatch/question_mark_wildcard (0.00s) --- PASS: TestAWSIAMMatch/no_match_different_pattern (0.00s) --- PASS: TestAWSIAMMatch/variable_not_expanded_due_to_missing_context (0.00s) === RUN TestMatchesActionsMultipartExpansion === RUN TestMatchesActionsMultipartExpansion/PutObject_directly_matches_PutObject === RUN TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_CreateMultipartUpload === RUN TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_UploadPart === RUN TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_CompleteMultipartUpload === RUN TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_AbortMultipartUpload === RUN TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_ListMultipartUploadParts === RUN TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_ListBucketMultipartUploads === RUN TestMatchesActionsMultipartExpansion/PutObject_does_not_allow_GetObject === RUN TestMatchesActionsMultipartExpansion/GetObject_does_not_allow_CreateMultipartUpload === RUN TestMatchesActionsMultipartExpansion/wildcard_s3:Put*_implicitly_allows_multipart_via_PutObject_match === RUN TestMatchesActionsMultipartExpansion/case-insensitive_multipart_action_lookup --- PASS: TestMatchesActionsMultipartExpansion (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_directly_matches_PutObject (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_CreateMultipartUpload (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_UploadPart (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_CompleteMultipartUpload (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_AbortMultipartUpload (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_ListMultipartUploadParts (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_implicitly_allows_ListBucketMultipartUploads (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/PutObject_does_not_allow_GetObject (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/GetObject_does_not_allow_CreateMultipartUpload (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/wildcard_s3:Put*_implicitly_allows_multipart_via_PutObject_match (0.00s) --- PASS: TestMatchesActionsMultipartExpansion/case-insensitive_multipart_action_lookup (0.00s) === RUN TestExpandPolicyVariables === RUN TestExpandPolicyVariables/expand_aws_username === RUN TestExpandPolicyVariables/expand_multiple_variables === RUN TestExpandPolicyVariables/no_variables_to_expand === RUN TestExpandPolicyVariables/nil_context === RUN TestExpandPolicyVariables/missing_variable_in_context --- PASS: TestExpandPolicyVariables (0.00s) --- PASS: TestExpandPolicyVariables/expand_aws_username (0.00s) --- PASS: TestExpandPolicyVariables/expand_multiple_variables (0.00s) --- PASS: TestExpandPolicyVariables/no_variables_to_expand (0.00s) --- PASS: TestExpandPolicyVariables/nil_context (0.00s) --- PASS: TestExpandPolicyVariables/missing_variable_in_context (0.00s) === RUN TestAWSWildcardMatch === RUN TestAWSWildcardMatch/case_insensitive_asterisk === RUN TestAWSWildcardMatch/case_insensitive_question_mark === RUN TestAWSWildcardMatch/mixed_wildcards === RUN TestAWSWildcardMatch/no_match --- PASS: TestAWSWildcardMatch (0.00s) --- PASS: TestAWSWildcardMatch/case_insensitive_asterisk (0.00s) --- PASS: TestAWSWildcardMatch/case_insensitive_question_mark (0.00s) --- PASS: TestAWSWildcardMatch/mixed_wildcards (0.00s) --- PASS: TestAWSWildcardMatch/no_match (0.00s) === RUN TestStringifyClaimValue === RUN TestStringifyClaimValue/string === RUN TestStringifyClaimValue/bool_true === RUN TestStringifyClaimValue/bool_false === RUN TestStringifyClaimValue/float64_integer === RUN TestStringifyClaimValue/float64_fraction === RUN TestStringifyClaimValue/float32 === RUN TestStringifyClaimValue/int === RUN TestStringifyClaimValue/int8 === RUN TestStringifyClaimValue/int16 === RUN TestStringifyClaimValue/int32 === RUN TestStringifyClaimValue/int64 === RUN TestStringifyClaimValue/uint === RUN TestStringifyClaimValue/uint8 === RUN TestStringifyClaimValue/uint16 === RUN TestStringifyClaimValue/uint32 === RUN TestStringifyClaimValue/uint64 === RUN TestStringifyClaimValue/json.Number_int === RUN TestStringifyClaimValue/json.Number_float === RUN TestStringifyClaimValue/nil === RUN TestStringifyClaimValue/slice_unsupported === RUN TestStringifyClaimValue/map_unsupported --- PASS: TestStringifyClaimValue (0.00s) --- PASS: TestStringifyClaimValue/string (0.00s) --- PASS: TestStringifyClaimValue/bool_true (0.00s) --- PASS: TestStringifyClaimValue/bool_false (0.00s) --- PASS: TestStringifyClaimValue/float64_integer (0.00s) --- PASS: TestStringifyClaimValue/float64_fraction (0.00s) --- PASS: TestStringifyClaimValue/float32 (0.00s) --- PASS: TestStringifyClaimValue/int (0.00s) --- PASS: TestStringifyClaimValue/int8 (0.00s) --- PASS: TestStringifyClaimValue/int16 (0.00s) --- PASS: TestStringifyClaimValue/int32 (0.00s) --- PASS: TestStringifyClaimValue/int64 (0.00s) --- PASS: TestStringifyClaimValue/uint (0.00s) --- PASS: TestStringifyClaimValue/uint8 (0.00s) --- PASS: TestStringifyClaimValue/uint16 (0.00s) --- PASS: TestStringifyClaimValue/uint32 (0.00s) --- PASS: TestStringifyClaimValue/uint64 (0.00s) --- PASS: TestStringifyClaimValue/json.Number_int (0.00s) --- PASS: TestStringifyClaimValue/json.Number_float (0.00s) --- PASS: TestStringifyClaimValue/nil (0.00s) --- PASS: TestStringifyClaimValue/slice_unsupported (0.00s) --- PASS: TestStringifyClaimValue/map_unsupported (0.00s) === RUN TestExpandPolicyVariablesDynamicJWTClaims === RUN TestExpandPolicyVariablesDynamicJWTClaims/dynamic_jwt_claim_substituted_in_resource_arn === RUN TestExpandPolicyVariablesDynamicJWTClaims/multiple_dynamic_jwt_claims_in_one_pattern === RUN TestExpandPolicyVariablesDynamicJWTClaims/dynamic_numeric_jwt_claim_is_stringified === RUN TestExpandPolicyVariablesDynamicJWTClaims/dynamic_saml_claim_substituted === RUN TestExpandPolicyVariablesDynamicJWTClaims/dynamic_oidc_claim_substituted === RUN TestExpandPolicyVariablesDynamicJWTClaims/missing_jwt_claim_leaves_placeholder_intact === RUN TestExpandPolicyVariablesDynamicJWTClaims/non-identity_prefix_is_not_substituted --- PASS: TestExpandPolicyVariablesDynamicJWTClaims (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/dynamic_jwt_claim_substituted_in_resource_arn (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/multiple_dynamic_jwt_claims_in_one_pattern (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/dynamic_numeric_jwt_claim_is_stringified (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/dynamic_saml_claim_substituted (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/dynamic_oidc_claim_substituted (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/missing_jwt_claim_leaves_placeholder_intact (0.00s) --- PASS: TestExpandPolicyVariablesDynamicJWTClaims/non-identity_prefix_is_not_substituted (0.00s) === RUN TestGitLabProjectUploadPolicy === RUN TestGitLabProjectUploadPolicy/user_writes_into_own_project_folder_-_allowed === RUN TestGitLabProjectUploadPolicy/user_tries_to_write_into_another_project_-_falls_through_to_default_Deny === RUN TestGitLabProjectUploadPolicy/different_user_writes_into_their_own_project_-_allowed === RUN TestGitLabProjectUploadPolicy/user_tries_to_GetObject_-_only_PutObject_allowed,_default_Deny === RUN TestGitLabProjectUploadPolicy/missing_project_path_claim_-_placeholder_prevents_match,_default_Deny --- PASS: TestGitLabProjectUploadPolicy (0.00s) --- PASS: TestGitLabProjectUploadPolicy/user_writes_into_own_project_folder_-_allowed (0.00s) --- PASS: TestGitLabProjectUploadPolicy/user_tries_to_write_into_another_project_-_falls_through_to_default_Deny (0.00s) --- PASS: TestGitLabProjectUploadPolicy/different_user_writes_into_their_own_project_-_allowed (0.00s) --- PASS: TestGitLabProjectUploadPolicy/user_tries_to_GetObject_-_only_PutObject_allowed,_default_Deny (0.00s) --- PASS: TestGitLabProjectUploadPolicy/missing_project_path_claim_-_placeholder_prevents_match,_default_Deny (0.00s) === RUN TestDistributedPolicyEngine === RUN TestDistributedPolicyEngine/policy_storage_consistency === RUN TestDistributedPolicyEngine/evaluation_consistency === RUN TestDistributedPolicyEngine/deny_precedence_consistency === RUN TestDistributedPolicyEngine/default_effect_consistency --- PASS: TestDistributedPolicyEngine (0.00s) --- PASS: TestDistributedPolicyEngine/policy_storage_consistency (0.00s) --- PASS: TestDistributedPolicyEngine/evaluation_consistency (0.00s) --- PASS: TestDistributedPolicyEngine/deny_precedence_consistency (0.00s) --- PASS: TestDistributedPolicyEngine/default_effect_consistency (0.00s) === RUN TestPolicyEngineConfigurationConsistency === RUN TestPolicyEngineConfigurationConsistency/consistent_default_effects_required === RUN TestPolicyEngineConfigurationConsistency/invalid_configuration_handling === RUN TestPolicyEngineConfigurationConsistency/invalid_configuration_handling/invalid_config_0 === RUN TestPolicyEngineConfigurationConsistency/invalid_configuration_handling/invalid_config_1 --- PASS: TestPolicyEngineConfigurationConsistency (0.00s) --- PASS: TestPolicyEngineConfigurationConsistency/consistent_default_effects_required (0.00s) --- PASS: TestPolicyEngineConfigurationConsistency/invalid_configuration_handling (0.00s) --- PASS: TestPolicyEngineConfigurationConsistency/invalid_configuration_handling/invalid_config_0 (0.00s) --- PASS: TestPolicyEngineConfigurationConsistency/invalid_configuration_handling/invalid_config_1 (0.00s) === RUN TestPolicyStoreDistributed === RUN TestPolicyStoreDistributed/memory_store_isolation === RUN TestPolicyStoreDistributed/policy_loading_error_handling --- PASS: TestPolicyStoreDistributed (0.00s) --- PASS: TestPolicyStoreDistributed/memory_store_isolation (0.00s) --- PASS: TestPolicyStoreDistributed/policy_loading_error_handling (0.00s) === RUN TestFilerPolicyStoreConfiguration === RUN TestFilerPolicyStoreConfiguration/filer_store_creation === RUN TestFilerPolicyStoreConfiguration/filer_store_custom_path === RUN TestFilerPolicyStoreConfiguration/filer_store_missing_address --- PASS: TestFilerPolicyStoreConfiguration (0.00s) --- PASS: TestFilerPolicyStoreConfiguration/filer_store_creation (0.00s) --- PASS: TestFilerPolicyStoreConfiguration/filer_store_custom_path (0.00s) --- PASS: TestFilerPolicyStoreConfiguration/filer_store_missing_address (0.00s) === RUN TestPolicyEvaluationPerformance policy_engine_distributed_test.go:384: Average policy evaluation time: 43.783µs --- PASS: TestPolicyEvaluationPerformance (0.00s) === RUN TestFilerPolicyStoreGetPolicyPrefersCanonicalFiles --- PASS: TestFilerPolicyStoreGetPolicyPrefersCanonicalFiles (0.03s) === RUN TestFilerPolicyStoreListPoliciesIncludesCanonicalAndLegacyFiles --- PASS: TestFilerPolicyStoreListPoliciesIncludesCanonicalAndLegacyFiles (0.01s) === RUN TestFilerPolicyStoreDeletePolicyRemovesCanonicalAndLegacyFiles --- PASS: TestFilerPolicyStoreDeletePolicyRemovesCanonicalAndLegacyFiles (0.00s) === RUN TestFilerPolicyStoreStorePolicyWritesCanonicalFileAndRemovesLegacyTwin --- PASS: TestFilerPolicyStoreStorePolicyWritesCanonicalFileAndRemovesLegacyTwin (0.01s) === RUN TestFilerPolicyStoreStorePolicyUpdatesExistingCanonicalFile --- PASS: TestFilerPolicyStoreStorePolicyUpdatesExistingCanonicalFile (0.02s) === RUN TestCopyPolicyDocumentClonesConditionState --- PASS: TestCopyPolicyDocumentClonesConditionState (0.00s) === RUN TestIsAlreadyExistsPolicyStoreErrorUsesStatusCode --- PASS: TestIsAlreadyExistsPolicyStoreErrorUsesStatusCode (0.00s) === RUN TestPolicyVariableMatchingInActionsAndResources === RUN TestPolicyVariableMatchingInActionsAndResources/policy_variable_in_action_matches === RUN TestPolicyVariableMatchingInActionsAndResources/policy_variable_in_resource_matches === RUN TestPolicyVariableMatchingInActionsAndResources/saml_username_variable_in_resource === RUN TestPolicyVariableMatchingInActionsAndResources/policy_variable_no_match_wrong_user === RUN TestPolicyVariableMatchingInActionsAndResources/missing_policy_variable_context --- PASS: TestPolicyVariableMatchingInActionsAndResources (0.00s) --- PASS: TestPolicyVariableMatchingInActionsAndResources/policy_variable_in_action_matches (0.00s) --- PASS: TestPolicyVariableMatchingInActionsAndResources/policy_variable_in_resource_matches (0.00s) --- PASS: TestPolicyVariableMatchingInActionsAndResources/saml_username_variable_in_resource (0.00s) --- PASS: TestPolicyVariableMatchingInActionsAndResources/policy_variable_no_match_wrong_user (0.00s) --- PASS: TestPolicyVariableMatchingInActionsAndResources/missing_policy_variable_context (0.00s) === RUN TestActionResourceConsistencyWithStringConditions --- PASS: TestActionResourceConsistencyWithStringConditions (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iam/policy 0.126s ? github.com/seaweedfs/seaweedfs/weed/iam/providers [no test files] === RUN TestIsClaimBasedPolicyRoleArn === RUN TestIsClaimBasedPolicyRoleArn/empty_matches === RUN TestIsClaimBasedPolicyRoleArn/sentinel_matches === RUN TestIsClaimBasedPolicyRoleArn/concrete_role_does_not === RUN TestIsClaimBasedPolicyRoleArn/random_ARN_does_not --- PASS: TestIsClaimBasedPolicyRoleArn (0.00s) --- PASS: TestIsClaimBasedPolicyRoleArn/empty_matches (0.00s) --- PASS: TestIsClaimBasedPolicyRoleArn/sentinel_matches (0.00s) --- PASS: TestIsClaimBasedPolicyRoleArn/concrete_role_does_not (0.00s) --- PASS: TestIsClaimBasedPolicyRoleArn/random_ARN_does_not (0.00s) === RUN TestTemporaryCredentialPrefix --- PASS: TestTemporaryCredentialPrefix (0.00s) === RUN TestTemporaryCredentialFormat --- PASS: TestTemporaryCredentialFormat (0.00s) === RUN TestLookupOIDCProviderForAccountPrefersAccountMatch --- PASS: TestLookupOIDCProviderForAccountPrefersAccountMatch (0.00s) === RUN TestLookupOIDCProviderForAccountFallsBackToGlobal --- PASS: TestLookupOIDCProviderForAccountFallsBackToGlobal (0.00s) === RUN TestLookupOIDCProviderForAccountSkipsAccountSpecificWhenAccountUnknown --- PASS: TestLookupOIDCProviderForAccountSkipsAccountSpecificWhenAccountUnknown (0.00s) === RUN TestLookupOIDCProviderForAccountFallsBackToStatic --- PASS: TestLookupOIDCProviderForAccountFallsBackToStatic (0.00s) === RUN TestComputeParentUserStability --- PASS: TestComputeParentUserStability (0.00s) === RUN TestComputeParentUserDistinguishesIssuer --- PASS: TestComputeParentUserDistinguishesIssuer (0.00s) === RUN TestComputeParentUserDistinguishesSubject --- PASS: TestComputeParentUserDistinguishesSubject (0.00s) === RUN TestComputeParentUserEmptySub --- PASS: TestComputeParentUserEmptySub (0.00s) === RUN TestComputeParentUserEmptyIss --- PASS: TestComputeParentUserEmptyIss (0.00s) === RUN TestComputeParentUserEncoding --- PASS: TestComputeParentUserEncoding (0.00s) === RUN TestSessionClaimsRoundTripParentUser --- PASS: TestSessionClaimsRoundTripParentUser (0.00s) === RUN TestSTSSessionClaimsToSessionInfo --- PASS: TestSTSSessionClaimsToSessionInfo (0.00s) === RUN TestSTSSessionClaimsToSessionInfoCredentialGeneration --- PASS: TestSTSSessionClaimsToSessionInfoCredentialGeneration (0.00s) === RUN TestSTSSessionClaimsToSessionInfoPreservesAllFields --- PASS: TestSTSSessionClaimsToSessionInfoPreservesAllFields (0.00s) === RUN TestSTSSessionClaimsToSessionInfoEmptyFields --- PASS: TestSTSSessionClaimsToSessionInfoEmptyFields (0.00s) === RUN TestSTSSessionClaimsToSessionInfoCredentialExpiration === RUN TestSTSSessionClaimsToSessionInfoCredentialExpiration/future_expiration === RUN TestSTSSessionClaimsToSessionInfoCredentialExpiration/past_expiration === RUN TestSTSSessionClaimsToSessionInfoCredentialExpiration/near_future_expiration --- PASS: TestSTSSessionClaimsToSessionInfoCredentialExpiration (0.00s) --- PASS: TestSTSSessionClaimsToSessionInfoCredentialExpiration/future_expiration (0.00s) --- PASS: TestSTSSessionClaimsToSessionInfoCredentialExpiration/past_expiration (0.00s) --- PASS: TestSTSSessionClaimsToSessionInfoCredentialExpiration/near_future_expiration (0.00s) === RUN TestSessionInfoIntegration --- PASS: TestSessionInfoIntegration (0.00s) === RUN TestSecretAccessKeyDeterminism --- PASS: TestSecretAccessKeyDeterminism (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iam/sts 0.097s ? github.com/seaweedfs/seaweedfs/weed/iam/util [no test files] === RUN TestExtractRoleNameFromArn === RUN TestExtractRoleNameFromArn/legacy_format_simple_role_name === RUN TestExtractRoleNameFromArn/legacy_format_custom_role_name === RUN TestExtractRoleNameFromArn/legacy_format_with_path === RUN TestExtractRoleNameFromArn/legacy_format_with_nested_path === RUN TestExtractRoleNameFromArn/standard_format_simple_role_name === RUN TestExtractRoleNameFromArn/standard_format_custom_role_name === RUN TestExtractRoleNameFromArn/standard_format_with_path === RUN TestExtractRoleNameFromArn/standard_format_with_nested_path === RUN TestExtractRoleNameFromArn/invalid_arn_missing_prefix === RUN TestExtractRoleNameFromArn/invalid_arn_incomplete === RUN TestExtractRoleNameFromArn/invalid_arn_no_role_marker === RUN TestExtractRoleNameFromArn/invalid_arn_wrong_service === RUN TestExtractRoleNameFromArn/empty_string === RUN TestExtractRoleNameFromArn/role_marker_no_name === RUN TestExtractRoleNameFromArn/standard_format_role_marker_no_name --- PASS: TestExtractRoleNameFromArn (0.00s) --- PASS: TestExtractRoleNameFromArn/legacy_format_simple_role_name (0.00s) --- PASS: TestExtractRoleNameFromArn/legacy_format_custom_role_name (0.00s) --- PASS: TestExtractRoleNameFromArn/legacy_format_with_path (0.00s) --- PASS: TestExtractRoleNameFromArn/legacy_format_with_nested_path (0.00s) --- PASS: TestExtractRoleNameFromArn/standard_format_simple_role_name (0.00s) --- PASS: TestExtractRoleNameFromArn/standard_format_custom_role_name (0.00s) --- PASS: TestExtractRoleNameFromArn/standard_format_with_path (0.00s) --- PASS: TestExtractRoleNameFromArn/standard_format_with_nested_path (0.00s) --- PASS: TestExtractRoleNameFromArn/invalid_arn_missing_prefix (0.00s) --- PASS: TestExtractRoleNameFromArn/invalid_arn_incomplete (0.00s) --- PASS: TestExtractRoleNameFromArn/invalid_arn_no_role_marker (0.00s) --- PASS: TestExtractRoleNameFromArn/invalid_arn_wrong_service (0.00s) --- PASS: TestExtractRoleNameFromArn/empty_string (0.00s) --- PASS: TestExtractRoleNameFromArn/role_marker_no_name (0.00s) --- PASS: TestExtractRoleNameFromArn/standard_format_role_marker_no_name (0.00s) === RUN TestExtractRoleNameFromPrincipal === RUN TestExtractRoleNameFromPrincipal/sts_assumed_role_legacy === RUN TestExtractRoleNameFromPrincipal/sts_assumed_role_legacy_no_session === RUN TestExtractRoleNameFromPrincipal/sts_assumed_role_standard === RUN TestExtractRoleNameFromPrincipal/sts_assumed_role_standard_no_session === RUN TestExtractRoleNameFromPrincipal/iam_role_legacy === RUN TestExtractRoleNameFromPrincipal/iam_role_legacy_with_path === RUN TestExtractRoleNameFromPrincipal/iam_role_standard === RUN TestExtractRoleNameFromPrincipal/iam_role_standard_with_path === RUN TestExtractRoleNameFromPrincipal/invalid_principal === RUN TestExtractRoleNameFromPrincipal/empty_string --- PASS: TestExtractRoleNameFromPrincipal (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/sts_assumed_role_legacy (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/sts_assumed_role_legacy_no_session (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/sts_assumed_role_standard (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/sts_assumed_role_standard_no_session (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/iam_role_legacy (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/iam_role_legacy_with_path (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/iam_role_standard (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/iam_role_standard_with_path (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/invalid_principal (0.00s) --- PASS: TestExtractRoleNameFromPrincipal/empty_string (0.00s) === RUN TestParseRoleARN === RUN TestParseRoleARN/legacy_format_simple_role === RUN TestParseRoleARN/legacy_format_with_path === RUN TestParseRoleARN/standard_format_simple_role === RUN TestParseRoleARN/standard_format_with_path === RUN TestParseRoleARN/invalid_arn_missing_prefix === RUN TestParseRoleARN/invalid_arn_no_role_marker === RUN TestParseRoleARN/invalid_arn_empty_role_name === RUN TestParseRoleARN/invalid_arn_empty_role_name_standard_format --- PASS: TestParseRoleARN (0.00s) --- PASS: TestParseRoleARN/legacy_format_simple_role (0.00s) --- PASS: TestParseRoleARN/legacy_format_with_path (0.00s) --- PASS: TestParseRoleARN/standard_format_simple_role (0.00s) --- PASS: TestParseRoleARN/standard_format_with_path (0.00s) --- PASS: TestParseRoleARN/invalid_arn_missing_prefix (0.00s) --- PASS: TestParseRoleARN/invalid_arn_no_role_marker (0.00s) --- PASS: TestParseRoleARN/invalid_arn_empty_role_name (0.00s) --- PASS: TestParseRoleARN/invalid_arn_empty_role_name_standard_format (0.00s) === RUN TestParsePrincipalARN === RUN TestParsePrincipalARN/sts_assumed_role_legacy === RUN TestParsePrincipalARN/sts_assumed_role_standard === RUN TestParsePrincipalARN/sts_assumed_role_no_session === RUN TestParsePrincipalARN/iam_role_legacy === RUN TestParsePrincipalARN/iam_role_standard === RUN TestParsePrincipalARN/iam_role_with_path === RUN TestParsePrincipalARN/invalid_principal === RUN TestParsePrincipalARN/invalid_sts_empty_role_name === RUN TestParsePrincipalARN/invalid_sts_empty_role_name_standard_format --- PASS: TestParsePrincipalARN (0.00s) --- PASS: TestParsePrincipalARN/sts_assumed_role_legacy (0.00s) --- PASS: TestParsePrincipalARN/sts_assumed_role_standard (0.00s) --- PASS: TestParsePrincipalARN/sts_assumed_role_no_session (0.00s) --- PASS: TestParsePrincipalARN/iam_role_legacy (0.00s) --- PASS: TestParsePrincipalARN/iam_role_standard (0.00s) --- PASS: TestParsePrincipalARN/iam_role_with_path (0.00s) --- PASS: TestParsePrincipalARN/invalid_principal (0.00s) --- PASS: TestParsePrincipalARN/invalid_sts_empty_role_name (0.00s) --- PASS: TestParsePrincipalARN/invalid_sts_empty_role_name_standard_format (0.00s) === RUN TestSecurityMaliciousUserARNs === RUN TestSecurityMaliciousUserARNs/user_ARN_with_role/_in_path === RUN TestSecurityMaliciousUserARNs/policy_ARN_with_role/_in_name === RUN TestSecurityMaliciousUserARNs/group_ARN_with_role/_in_name === RUN TestSecurityMaliciousUserARNs/legacy_user_ARN_with_role/_in_path --- PASS: TestSecurityMaliciousUserARNs (0.00s) --- PASS: TestSecurityMaliciousUserARNs/user_ARN_with_role/_in_path (0.00s) --- PASS: TestSecurityMaliciousUserARNs/policy_ARN_with_role/_in_name (0.00s) --- PASS: TestSecurityMaliciousUserARNs/group_ARN_with_role/_in_name (0.00s) --- PASS: TestSecurityMaliciousUserARNs/legacy_user_ARN_with_role/_in_path (0.00s) === RUN TestSecurityMaliciousSTSUserARNs === RUN TestSecurityMaliciousSTSUserARNs/STS_user_with_assumed-role_in_path === RUN TestSecurityMaliciousSTSUserARNs/legacy_STS_user_with_assumed-role_in_path --- PASS: TestSecurityMaliciousSTSUserARNs (0.00s) --- PASS: TestSecurityMaliciousSTSUserARNs/STS_user_with_assumed-role_in_path (0.00s) --- PASS: TestSecurityMaliciousSTSUserARNs/legacy_STS_user_with_assumed-role_in_path (0.00s) === RUN TestEdgeCaseMultipleRoleMarkers === RUN TestEdgeCaseMultipleRoleMarkers/legacy_format_role_in_path === RUN TestEdgeCaseMultipleRoleMarkers/standard_format_role_in_path === RUN TestEdgeCaseMultipleRoleMarkers/multiple_role_markers_in_path === RUN TestEdgeCaseMultipleRoleMarkers/sts_assumed_role_with_role_in_path --- PASS: TestEdgeCaseMultipleRoleMarkers (0.00s) --- PASS: TestEdgeCaseMultipleRoleMarkers/legacy_format_role_in_path (0.00s) --- PASS: TestEdgeCaseMultipleRoleMarkers/standard_format_role_in_path (0.00s) --- PASS: TestEdgeCaseMultipleRoleMarkers/multiple_role_markers_in_path (0.00s) --- PASS: TestEdgeCaseMultipleRoleMarkers/sts_assumed_role_with_role_in_path (0.00s) === RUN TestEdgeCaseConsecutiveSlashes === RUN TestEdgeCaseConsecutiveSlashes/consecutive_slashes_immediately_after_role === RUN TestEdgeCaseConsecutiveSlashes/consecutive_slashes_in_path === RUN TestEdgeCaseConsecutiveSlashes/multiple_consecutive_slashes --- PASS: TestEdgeCaseConsecutiveSlashes (0.00s) --- PASS: TestEdgeCaseConsecutiveSlashes/consecutive_slashes_immediately_after_role (0.00s) --- PASS: TestEdgeCaseConsecutiveSlashes/consecutive_slashes_in_path (0.00s) --- PASS: TestEdgeCaseConsecutiveSlashes/multiple_consecutive_slashes (0.00s) === RUN TestEdgeCaseSpecialCharactersInRoleName === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_hyphens === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_underscores === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_dots === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_at_sign === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_plus_and_equals === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_commas === RUN TestEdgeCaseSpecialCharactersInRoleName/role_name_with_mixed_special_chars === RUN TestEdgeCaseSpecialCharactersInRoleName/path_with_special_characters --- PASS: TestEdgeCaseSpecialCharactersInRoleName (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_hyphens (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_underscores (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_dots (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_at_sign (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_plus_and_equals (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_commas (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/role_name_with_mixed_special_chars (0.00s) --- PASS: TestEdgeCaseSpecialCharactersInRoleName/path_with_special_characters (0.00s) === RUN TestEdgeCaseExtremelyLongRoleName === RUN TestEdgeCaseExtremelyLongRoleName/role_name_at_max_length === RUN TestEdgeCaseExtremelyLongRoleName/role_with_long_path_components --- PASS: TestEdgeCaseExtremelyLongRoleName (0.00s) --- PASS: TestEdgeCaseExtremelyLongRoleName/role_name_at_max_length (0.00s) --- PASS: TestEdgeCaseExtremelyLongRoleName/role_with_long_path_components (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iam/utils 0.018s === RUN TestGetActionsUserPath --- PASS: TestGetActionsUserPath (0.00s) === RUN TestGetActionsWildcardPath --- PASS: TestGetActionsWildcardPath (0.00s) === RUN TestGetActionsInvalidAction --- PASS: TestGetActionsInvalidAction (0.00s) === RUN TestPutGetUserPolicyPreservesStatements --- PASS: TestPutGetUserPolicyPreservesStatements (0.00s) === RUN TestPutGetUserPolicyIssue9008 --- PASS: TestPutGetUserPolicyIssue9008 (0.00s) === RUN TestMultipleInlinePoliciesAggregateActions --- PASS: TestMultipleInlinePoliciesAggregateActions (0.00s) === RUN TestGetPolicy --- PASS: TestGetPolicy (0.00s) === RUN TestDeletePolicy --- PASS: TestDeletePolicy (0.00s) === RUN TestListPolicies --- PASS: TestListPolicies (0.00s) === RUN TestAttachUserPolicy --- PASS: TestAttachUserPolicy (0.00s) === RUN TestManagedPolicyActionsPreservedAcrossInlineMutations --- PASS: TestManagedPolicyActionsPreservedAcrossInlineMutations (0.00s) === RUN TestDetachUserPolicy --- PASS: TestDetachUserPolicy (0.00s) === RUN TestListAttachedUserPolicies --- PASS: TestListAttachedUserPolicies (0.00s) === RUN TestUserArnIsValid --- PASS: TestUserArnIsValid (0.00s) === RUN TestCreateAccessKeyWithCallerSuppliedKeys --- PASS: TestCreateAccessKeyWithCallerSuppliedKeys (0.00s) === RUN TestCreateAccessKeyRandomGeneration --- PASS: TestCreateAccessKeyRandomGeneration (0.00s) === RUN TestCreateAccessKeyRejectsWeakKeys --- PASS: TestCreateAccessKeyRejectsWeakKeys (0.00s) === RUN TestCreateAccessKeyRejectsCollision === RUN TestCreateAccessKeyRejectsCollision/identity_credential === RUN TestCreateAccessKeyRejectsCollision/service_account_credential --- PASS: TestCreateAccessKeyRejectsCollision (0.00s) --- PASS: TestCreateAccessKeyRejectsCollision/identity_credential (0.00s) --- PASS: TestCreateAccessKeyRejectsCollision/service_account_credential (0.00s) === RUN TestCreateAccessKeyBoundary --- PASS: TestCreateAccessKeyBoundary (0.00s) === RUN TestCreateAccessKeyRejectsPartialSupply --- PASS: TestCreateAccessKeyRejectsPartialSupply (0.00s) === RUN TestPutPoliciesCreatesInCredentialStore --- PASS: TestPutPoliciesCreatesInCredentialStore (0.00s) === RUN TestGetPoliciesReadsFromCredentialStore --- PASS: TestGetPoliciesReadsFromCredentialStore (0.00s) === RUN TestPutPoliciesDeletesRemovedFromCredentialStore --- PASS: TestPutPoliciesDeletesRemovedFromCredentialStore (0.00s) === RUN TestPutPoliciesUpdatesChangedInCredentialStore --- PASS: TestPutPoliciesUpdatesChangedInCredentialStore (0.00s) === RUN TestPutPoliciesRoutesUserInlineThroughCredentialStore --- PASS: TestPutPoliciesRoutesUserInlineThroughCredentialStore (0.00s) === RUN TestPutPoliciesDeletesRemovedUserInline --- PASS: TestPutPoliciesDeletesRemovedUserInline (0.00s) === RUN TestPutPoliciesRoutesGroupInlineThroughCredentialStore --- PASS: TestPutPoliciesRoutesGroupInlineThroughCredentialStore (0.00s) === RUN TestPutPoliciesDeletesRemovedGroupInline --- PASS: TestPutPoliciesDeletesRemovedGroupInline (0.00s) === RUN TestPutPoliciesIsAuthoritativeWhenStoreIsNotFilerEtc --- PASS: TestPutPoliciesIsAuthoritativeWhenStoreIsNotFilerEtc (0.00s) === RUN TestGetPoliciesReadsInlineAndGroupInlineFromCredentialStore --- PASS: TestGetPoliciesReadsInlineAndGroupInlineFromCredentialStore (0.00s) === RUN TestCreateUser --- PASS: TestCreateUser (0.00s) === RUN TestListUsers --- PASS: TestListUsers (0.00s) === RUN TestListUsersRequestIdMatchesResponseHeader --- PASS: TestListUsersRequestIdMatchesResponseHeader (0.00s) === RUN TestListAccessKeys --- PASS: TestListAccessKeys (0.00s) === RUN TestUpdateAccessKey --- PASS: TestUpdateAccessKey (0.00s) === RUN TestGetUser --- PASS: TestGetUser (0.00s) === RUN TestCreatePolicy --- PASS: TestCreatePolicy (0.00s) === RUN TestCreatePolicyVersion --- PASS: TestCreatePolicyVersion (0.00s) === RUN TestCreatePolicyVersionMissingPolicy E0720 21:29:54.280072 iamapi_handlers.go:30 Response policy does-not-exist not found --- PASS: TestCreatePolicyVersionMissingPolicy (0.01s) === RUN TestCreatePolicyVersionRequiresSetAsDefault E0720 21:29:54.289102 iamapi_handlers.go:30 Response SetAsDefault must be true: SeaweedFS stores a single managed policy version --- PASS: TestCreatePolicyVersionRequiresSetAsDefault (0.00s) === RUN TestPutUserPolicy --- PASS: TestPutUserPolicy (0.00s) === RUN TestPutUserPolicyError E0720 21:29:54.289861 iamapi_management_handlers.go:1403 PutUserPolicy: the user with name InvalidUser cannot be found E0720 21:29:54.289869 iamapi_handlers.go:30 Response the user with name InvalidUser cannot be found --- PASS: TestPutUserPolicyError (0.00s) === RUN TestGetUserPolicy --- PASS: TestGetUserPolicy (0.00s) === RUN TestUpdateUser --- PASS: TestUpdateUser (0.00s) === RUN TestDeleteUser --- PASS: TestDeleteUser (0.00s) === RUN TestHandleImplicitUsername --- PASS: TestHandleImplicitUsername (0.01s) === RUN TestCreateUserDoesNotSaveAllUsers --- PASS: TestCreateUserDoesNotSaveAllUsers (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/iamapi 0.116s === RUN TestCropping --- PASS: TestCropping (0.21s) === RUN TestXYZ --- PASS: TestXYZ (0.68s) === RUN TestResizing --- PASS: TestResizing (0.06s) PASS ok github.com/seaweedfs/seaweedfs/weed/images 0.979s === RUN TestCachedProviderDecryptHitsProviderOnce --- PASS: TestCachedProviderDecryptHitsProviderOnce (0.00s) === RUN TestCachedProviderContextSensitivity --- PASS: TestCachedProviderContextSensitivity (0.00s) === RUN TestCachedProviderTTLExpiry --- PASS: TestCachedProviderTTLExpiry (0.04s) === RUN TestCachedProviderMaxEntries --- PASS: TestCachedProviderMaxEntries (0.00s) === RUN TestCachedProviderClearsOverwrittenPlaintext --- PASS: TestCachedProviderClearsOverwrittenPlaintext (0.00s) === RUN TestCachedProviderNoWriteAfterClose --- PASS: TestCachedProviderNoWriteAfterClose (0.00s) === RUN TestAddKMSProviderWiresCache --- PASS: TestAddKMSProviderWiresCache (0.00s) === RUN TestCiphertextEnvelope_CreateAndParse --- PASS: TestCiphertextEnvelope_CreateAndParse (0.00s) === RUN TestCiphertextEnvelope_InvalidFormat --- PASS: TestCiphertextEnvelope_InvalidFormat (0.00s) === RUN TestCiphertextEnvelope_ValidationErrors === RUN TestCiphertextEnvelope_ValidationErrors/Valid === RUN TestCiphertextEnvelope_ValidationErrors/Empty_provider === RUN TestCiphertextEnvelope_ValidationErrors/Empty_keyID === RUN TestCiphertextEnvelope_ValidationErrors/Empty_ciphertext --- PASS: TestCiphertextEnvelope_ValidationErrors (0.00s) --- PASS: TestCiphertextEnvelope_ValidationErrors/Valid (0.00s) --- PASS: TestCiphertextEnvelope_ValidationErrors/Empty_provider (0.00s) --- PASS: TestCiphertextEnvelope_ValidationErrors/Empty_keyID (0.00s) --- PASS: TestCiphertextEnvelope_ValidationErrors/Empty_ciphertext (0.00s) === RUN TestCiphertextEnvelope_MultipleProviders === RUN TestCiphertextEnvelope_MultipleProviders/openbao === RUN TestCiphertextEnvelope_MultipleProviders/gcp === RUN TestCiphertextEnvelope_MultipleProviders/azure === RUN TestCiphertextEnvelope_MultipleProviders/aws --- PASS: TestCiphertextEnvelope_MultipleProviders (0.00s) --- PASS: TestCiphertextEnvelope_MultipleProviders/openbao (0.00s) --- PASS: TestCiphertextEnvelope_MultipleProviders/gcp (0.00s) --- PASS: TestCiphertextEnvelope_MultipleProviders/azure (0.00s) --- PASS: TestCiphertextEnvelope_MultipleProviders/aws (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/kms 0.061s ? github.com/seaweedfs/seaweedfs/weed/kms/aws [no test files] ? github.com/seaweedfs/seaweedfs/weed/kms/gcp [no test files] ? github.com/seaweedfs/seaweedfs/weed/kms/local [no test files] ? github.com/seaweedfs/seaweedfs/weed/kms/openbao [no test files] === RUN TestChunkedDirtyPagesFailWritesAfterUploadError --- PASS: TestChunkedDirtyPagesFailWritesAfterUploadError (0.00s) === RUN TestGrpcErrorToFuseStatusUnwrapsCanceledThroughFmtErrorf --- PASS: TestGrpcErrorToFuseStatusUnwrapsCanceledThroughFmtErrorf (0.00s) === RUN TestGrpcErrorToFuseStatusDropsCanceledThroughPercentV --- PASS: TestGrpcErrorToFuseStatusDropsCanceledThroughPercentV (0.00s) === RUN TestWriteErrorToFuseStatus === RUN TestWriteErrorToFuseStatus/assign_shed_during_growth === RUN TestWriteErrorToFuseStatus/no_free_volumes_left === RUN TestWriteErrorToFuseStatus/unrelated_upload_failure --- PASS: TestWriteErrorToFuseStatus (0.00s) --- PASS: TestWriteErrorToFuseStatus/assign_shed_during_growth (0.00s) --- PASS: TestWriteErrorToFuseStatus/no_free_volumes_left (0.00s) --- PASS: TestWriteErrorToFuseStatus/unrelated_upload_failure (0.00s) === RUN TestIsRetryableFilerError === RUN TestIsRetryableFilerError/nil === RUN TestIsRetryableFilerError/canceled === RUN TestIsRetryableFilerError/unavailable === RUN TestIsRetryableFilerError/deadline_exceeded === RUN TestIsRetryableFilerError/resource_exhausted === RUN TestIsRetryableFilerError/internal === RUN TestIsRetryableFilerError/not_found === RUN TestIsRetryableFilerError/already_exists === RUN TestIsRetryableFilerError/invalid_argument === RUN TestIsRetryableFilerError/permission_denied === RUN TestIsRetryableFilerError/unauthenticated === RUN TestIsRetryableFilerError/failed_precondition === RUN TestIsRetryableFilerError/plain_error_retries === RUN TestIsRetryableFilerError/wrapped_canceled_still_retries === RUN TestIsRetryableFilerError/wrapped_not_found_still_skipped --- PASS: TestIsRetryableFilerError (0.00s) --- PASS: TestIsRetryableFilerError/nil (0.00s) --- PASS: TestIsRetryableFilerError/canceled (0.00s) --- PASS: TestIsRetryableFilerError/unavailable (0.00s) --- PASS: TestIsRetryableFilerError/deadline_exceeded (0.00s) --- PASS: TestIsRetryableFilerError/resource_exhausted (0.00s) --- PASS: TestIsRetryableFilerError/internal (0.00s) --- PASS: TestIsRetryableFilerError/not_found (0.00s) --- PASS: TestIsRetryableFilerError/already_exists (0.00s) --- PASS: TestIsRetryableFilerError/invalid_argument (0.00s) --- PASS: TestIsRetryableFilerError/permission_denied (0.00s) --- PASS: TestIsRetryableFilerError/unauthenticated (0.00s) --- PASS: TestIsRetryableFilerError/failed_precondition (0.00s) --- PASS: TestIsRetryableFilerError/plain_error_retries (0.00s) --- PASS: TestIsRetryableFilerError/wrapped_canceled_still_retries (0.00s) --- PASS: TestIsRetryableFilerError/wrapped_not_found_still_skipped (0.00s) === RUN TestRetryMetadataFlushIfShortCircuitsOnPermanentError --- PASS: TestRetryMetadataFlushIfShortCircuitsOnPermanentError (0.00s) === RUN TestRetryMetadataFlushIfRetriesTransientErrors --- PASS: TestRetryMetadataFlushIfRetriesTransientErrors (0.00s) === RUN TestReadUncachedRemoteEntryDoesNotDeadlock I0720 21:29:54.235239 leveldb_store.go:48 filer store dir: /tmp/TestReadUncachedRemoteEntryDoesNotDeadlock1161077564/001/meta I0720 21:29:54.235390 file_util.go:29 Folder /tmp/TestReadUncachedRemoteEntryDoesNotDeadlock1161077564/001/meta Permission: -rwxr-xr-x --- PASS: TestReadUncachedRemoteEntryDoesNotDeadlock (0.02s) === RUN TestFileHandleFullPathFallsBackAfterForget --- PASS: TestFileHandleFullPathFallsBackAfterForget (0.00s) === RUN TestFileHandleFullPathUsesSavedRenamePathAfterForget --- PASS: TestFileHandleFullPathUsesSavedRenamePathAfterForget (0.00s) === RUN TestIsFilerConfUpdateEventMatchesRenameTarget --- PASS: TestIsFilerConfUpdateEventMatchesRenameTarget (0.00s) === RUN TestInodeEntry_removeOnePath === RUN TestInodeEntry_removeOnePath/actual_case === RUN TestInodeEntry_removeOnePath/empty === RUN TestInodeEntry_removeOnePath/single === RUN TestInodeEntry_removeOnePath/first === RUN TestInodeEntry_removeOnePath/middle === RUN TestInodeEntry_removeOnePath/last === RUN TestInodeEntry_removeOnePath/not_found --- PASS: TestInodeEntry_removeOnePath (0.00s) --- PASS: TestInodeEntry_removeOnePath/actual_case (0.00s) --- PASS: TestInodeEntry_removeOnePath/empty (0.00s) --- PASS: TestInodeEntry_removeOnePath/single (0.00s) --- PASS: TestInodeEntry_removeOnePath/first (0.00s) --- PASS: TestInodeEntry_removeOnePath/middle (0.00s) --- PASS: TestInodeEntry_removeOnePath/last (0.00s) --- PASS: TestInodeEntry_removeOnePath/not_found (0.00s) === RUN TestOnlyDirectoriesGetDirState W0720 21:29:54.253410 inode_to_path.go:211 MarkChildrenCached inode 4649705233135616952 not a tracked directory for /data/file.txt --- PASS: TestOnlyDirectoriesGetDirState (0.00s) === RUN TestRecordDirectoryUpdateSwitchesDirectoryToReadThrough --- PASS: TestRecordDirectoryUpdateSwitchesDirectoryToReadThrough (0.00s) === RUN TestMarkChildrenCachedClearsReadThroughMode --- PASS: TestMarkChildrenCachedClearsReadThroughMode (0.00s) === RUN TestRetryMetadataFlushEventuallySucceeds --- PASS: TestRetryMetadataFlushEventuallySucceeds (0.00s) === RUN TestRetryMetadataFlushReturnsLastError --- PASS: TestRetryMetadataFlushReturnsLastError (0.00s) === RUN TestRetryMetadataFlushStopsOnCancel --- PASS: TestRetryMetadataFlushStopsOnCancel (0.00s) === RUN TestWriterPatternSequentialAndHysteresis --- PASS: TestWriterPatternSequentialAndHysteresis (0.00s) === RUN TestWriterPatternFarFirstWriteIsRandom --- PASS: TestWriterPatternFarFirstWriteIsRandom (0.00s) === RUN TestWriterPatternToleranceAbsorbsReorder --- PASS: TestWriterPatternToleranceAbsorbsReorder (0.00s) === RUN TestWriterPatternRandomRecoveryNeedsHysteresis --- PASS: TestWriterPatternRandomRecoveryNeedsHysteresis (0.00s) === RUN TestWriterPatternFrontierNeverRegresses --- PASS: TestWriterPatternFrontierNeverRegresses (0.00s) === RUN TestWriterPatternToleranceBoundary --- PASS: TestWriterPatternToleranceBoundary (0.00s) === RUN TestWriterPatternRecoversToSequential --- PASS: TestWriterPatternRecoversToSequential (0.00s) === RUN TestResolvePeerAdvertiseAddr === RUN TestResolvePeerAdvertiseAddr/explicit_advertise_wins === RUN TestResolvePeerAdvertiseAddr/bind_host_used_verbatim === RUN TestResolvePeerAdvertiseAddr/wildcard_ipv4_bind === RUN TestResolvePeerAdvertiseAddr/empty_host_bind === RUN TestResolvePeerAdvertiseAddr/ipv6_wildcard === RUN TestResolvePeerAdvertiseAddr/unparseable_listen_errors --- PASS: TestResolvePeerAdvertiseAddr (0.00s) --- PASS: TestResolvePeerAdvertiseAddr/explicit_advertise_wins (0.00s) --- PASS: TestResolvePeerAdvertiseAddr/bind_host_used_verbatim (0.00s) --- PASS: TestResolvePeerAdvertiseAddr/wildcard_ipv4_bind (0.00s) --- PASS: TestResolvePeerAdvertiseAddr/empty_host_bind (0.00s) --- PASS: TestResolvePeerAdvertiseAddr/ipv6_wildcard (0.00s) --- PASS: TestResolvePeerAdvertiseAddr/unparseable_listen_errors (0.00s) === RUN TestIsWildcardHost --- PASS: TestIsWildcardHost (0.00s) === RUN TestPeerAnnouncer_FlushGroupsByOwner --- PASS: TestPeerAnnouncer_FlushGroupsByOwner (0.00s) === RUN TestPeerAnnouncer_SkipOwnerEqualsSelf --- PASS: TestPeerAnnouncer_SkipOwnerEqualsSelf (0.00s) === RUN TestPeerAnnouncer_RequeueOnRejection --- PASS: TestPeerAnnouncer_RequeueOnRejection (0.00s) === RUN TestPeerAnnouncer_TTLRenewal --- PASS: TestPeerAnnouncer_TTLRenewal (0.00s) === RUN TestPeerAnnouncer_ReAnnouncesOnOwnerChange --- PASS: TestPeerAnnouncer_ReAnnouncesOnOwnerChange (0.00s) === RUN TestPeerAnnouncer_DropsEvictedFids --- PASS: TestPeerAnnouncer_DropsEvictedFids (0.00s) === RUN TestPeerAnnouncer_StopWaitsForFlush --- PASS: TestPeerAnnouncer_StopWaitsForFlush (0.00s) === RUN TestPeerAnnouncer_SelfOwnedWritesToLocalDir --- PASS: TestPeerAnnouncer_SelfOwnedWritesToLocalDir (0.00s) === RUN TestPeerAnnouncer_DialerErrorRequeues --- PASS: TestPeerAnnouncer_DialerErrorRequeues (0.00s) === RUN TestPeerDirectory_AnnounceAndLookup --- PASS: TestPeerDirectory_AnnounceAndLookup (0.00s) === RUN TestPeerDirectory_OwnerRejection --- PASS: TestPeerDirectory_OwnerRejection (0.00s) === RUN TestPeerDirectory_MultipleHolders --- PASS: TestPeerDirectory_MultipleHolders (0.00s) === RUN TestPeerDirectory_RenewExtendsExpiry --- PASS: TestPeerDirectory_RenewExtendsExpiry (0.00s) === RUN TestPeerDirectory_TTLExpiry --- PASS: TestPeerDirectory_TTLExpiry (0.00s) === RUN TestPeerDirectory_Sweep --- PASS: TestPeerDirectory_Sweep (0.00s) === RUN TestPeerDirectory_LookupOrdersByRecency --- PASS: TestPeerDirectory_LookupOrdersByRecency (0.00s) === RUN TestPeerDirectory_LookupOrderAfterRenewal --- PASS: TestPeerDirectory_LookupOrderAfterRenewal (0.00s) === RUN TestPeerDirectory_StatsCountersIncrement --- PASS: TestPeerDirectory_StatsCountersIncrement (0.00s) === RUN TestPeerDirectory_LookupCapsHolderList --- PASS: TestPeerDirectory_LookupCapsHolderList (0.00s) === RUN TestFetchChunkFromPeer_Hit I0720 21:29:54.258638 peer_grpc.go:88 peer-grpc listening on 127.0.0.1:46249 --- PASS: TestFetchChunkFromPeer_Hit (0.00s) === RUN TestFetchChunkFromPeer_Base64Etag I0720 21:29:54.260744 peer_grpc.go:88 peer-grpc listening on 127.0.0.1:46869 --- PASS: TestFetchChunkFromPeer_Base64Etag (0.00s) === RUN TestFetchChunkFromPeer_EtagMismatch I0720 21:29:54.261523 peer_grpc.go:88 peer-grpc listening on 127.0.0.1:37833 --- PASS: TestFetchChunkFromPeer_EtagMismatch (0.00s) === RUN TestFetchChunkFromPeer_NotFound I0720 21:29:54.264604 peer_grpc.go:88 peer-grpc listening on 127.0.0.1:34231 --- PASS: TestFetchChunkFromPeer_NotFound (0.00s) === RUN TestSortHoldersByLocality --- PASS: TestSortHoldersByLocality (0.00s) === RUN TestSortHoldersByLocality_NoSelfLabels --- PASS: TestSortHoldersByLocality_NoSelfLabels (0.00s) === RUN TestFetchChunkFromPeer_MultiFrameChunkAssembledCorrectly I0720 21:29:54.268229 peer_grpc.go:88 peer-grpc listening on 127.0.0.1:34361 --- PASS: TestFetchChunkFromPeer_MultiFrameChunkAssembledCorrectly (0.06s) === RUN TestPeerGrpcServer_AnnounceAndLookup --- PASS: TestPeerGrpcServer_AnnounceAndLookup (0.00s) === RUN TestPeerGrpcServer_OwnerMismatch --- PASS: TestPeerGrpcServer_OwnerMismatch (0.00s) === RUN TestPeerGrpcServer_FetchChunk_StreamsHit --- PASS: TestPeerGrpcServer_FetchChunk_StreamsHit (0.01s) === RUN TestPeerGrpcServer_FetchChunk_NotFound --- PASS: TestPeerGrpcServer_FetchChunk_NotFound (0.00s) === RUN TestOwnerFor_EmptySeeds --- PASS: TestOwnerFor_EmptySeeds (0.00s) === RUN TestOwnerFor_Deterministic --- PASS: TestOwnerFor_Deterministic (0.00s) === RUN TestOwnerFor_DistributesEvenly --- PASS: TestOwnerFor_DistributesEvenly (0.00s) === RUN TestOwnerFor_MinimalShuffleOnSeedChange --- PASS: TestOwnerFor_MinimalShuffleOnSeedChange (0.00s) === RUN TestOwnerFor_TieBreakerDeterministic --- PASS: TestOwnerFor_TieBreakerDeterministic (0.00s) === RUN TestPeerRegistrar_StartPopulatesSeedsFromFiler --- PASS: TestPeerRegistrar_StartPopulatesSeedsFromFiler (0.00s) === RUN TestPeerRegistrar_HeartbeatTTLMatchesConfig --- PASS: TestPeerRegistrar_HeartbeatTTLMatchesConfig (0.00s) === RUN TestPeerRegistrar_StopIsIdempotent --- PASS: TestPeerRegistrar_StopIsIdempotent (0.00s) === RUN TestPeerRegistrar_RegisterBroadcastsToAllFilers --- PASS: TestPeerRegistrar_RegisterBroadcastsToAllFilers (0.00s) === RUN TestPeerRegistrar_ListMergesAcrossFilers --- PASS: TestPeerRegistrar_ListMergesAcrossFilers (0.00s) === RUN TestPeerRegistrar_ListMergeKeepsNewestLastSeen --- PASS: TestPeerRegistrar_ListMergeKeepsNewestLastSeen (0.00s) === RUN TestNonOverlappingLocksFromDifferentOwners --- PASS: TestNonOverlappingLocksFromDifferentOwners (0.00s) === RUN TestOverlappingReadLocksFromDifferentOwners --- PASS: TestOverlappingReadLocksFromDifferentOwners (0.00s) === RUN TestOverlappingWriteReadConflict --- PASS: TestOverlappingWriteReadConflict (0.00s) === RUN TestOverlappingWriteWriteConflict --- PASS: TestOverlappingWriteWriteConflict (0.00s) === RUN TestSameOwnerUpgradeReadToWrite --- PASS: TestSameOwnerUpgradeReadToWrite (0.00s) === RUN TestSameOwnerDowngradeWriteToRead --- PASS: TestSameOwnerDowngradeWriteToRead (0.00s) === RUN TestLockCoalescing --- PASS: TestLockCoalescing (0.00s) === RUN TestLockSplitting --- PASS: TestLockSplitting (0.00s) === RUN TestGetLkConflict --- PASS: TestGetLkConflict (0.00s) === RUN TestGetLkNoConflict --- PASS: TestGetLkNoConflict (0.00s) === RUN TestGetLkSameOwnerNoConflict --- PASS: TestGetLkSameOwnerNoConflict (0.00s) === RUN TestReleaseOwner --- PASS: TestReleaseOwner (0.00s) === RUN TestDifferentLockKindsDoNotConflict --- PASS: TestDifferentLockKindsDoNotConflict (0.00s) === RUN TestReleasePosixOwnerReleasesPosixLocksAndWakesWaiters --- PASS: TestReleasePosixOwnerReleasesPosixLocksAndWakesWaiters (0.05s) === RUN TestReleasePosixOwnerDoesNotReleaseFlockLocks --- PASS: TestReleasePosixOwnerDoesNotReleaseFlockLocks (0.00s) === RUN TestHasPosixOwnerIgnoresMissingOwnerAndFlock --- PASS: TestHasPosixOwnerIgnoresMissingOwnerAndFlock (0.00s) === RUN TestWakeEligibleWaitersKeepsInodeUntilWakeRefReleased --- PASS: TestWakeEligibleWaitersKeepsInodeUntilWakeRefReleased (0.00s) === RUN TestReleaseFlockOwnerDoesNotReleasePosixLocks --- PASS: TestReleaseFlockOwnerDoesNotReleasePosixLocks (0.00s) === RUN TestReleaseOwnerWakesWaiters --- PASS: TestReleaseOwnerWakesWaiters (0.05s) === RUN TestSetLkwBlocksAndSucceeds --- PASS: TestSetLkwBlocksAndSucceeds (0.05s) === RUN TestSetLkwCancellation --- PASS: TestSetLkwCancellation (0.05s) === RUN TestWholeFileLock --- PASS: TestWholeFileLock (0.00s) === RUN TestUnlockNoExistingLocks --- PASS: TestUnlockNoExistingLocks (0.00s) === RUN TestMultipleInodesIndependent --- PASS: TestMultipleInodesIndependent (0.00s) === RUN TestMemoryCleanup --- PASS: TestMemoryCleanup (0.00s) === RUN TestSelectiveWaking --- PASS: TestSelectiveWaking (0.15s) === RUN TestSameOwnerReplaceDifferentType --- PASS: TestSameOwnerReplaceDifferentType (0.00s) === RUN TestNonAdjacentRangesNotCoalesced --- PASS: TestNonAdjacentRangesNotCoalesced (0.00s) === RUN TestAdjacencyNoOverflowAtMaxUint64 --- PASS: TestAdjacencyNoOverflowAtMaxUint64 (0.00s) === RUN TestSetLkRetriesPastDeadInodeLocks --- PASS: TestSetLkRetriesPastDeadInodeLocks (0.00s) === RUN TestGetInodeLocksEvictsDeadEntry --- PASS: TestGetInodeLocksEvictsDeadEntry (0.00s) === RUN TestConcurrentFlockChurnPreservesMutualExclusion --- PASS: TestConcurrentFlockChurnPreservesMutualExclusion (0.02s) === RUN TestPermissionCheckPerformance === Permission Check Performance Test (WITH CACHE) === Files simulated: 10000 Checks per file: 5 Total checks: 50000 System lookups: 1 (would be 50000 without cache) Lookups eliminated: 49999 (100.0% reduction) Time elapsed: 4.322321ms Throughput: 11567859 checks/sec --- PASS: TestPermissionCheckPerformance (0.00s) === RUN TestAttrChunkRace --- PASS: TestAttrChunkRace (0.00s) === RUN TestReadFromChunksRace --- PASS: TestReadFromChunksRace (0.00s) === RUN TestLoadDirectoryEntriesDirectFiltersHiddenEntriesAndMapsIds --- PASS: TestLoadDirectoryEntriesDirectFiltersHiddenEntriesAndMapsIds (0.00s) === RUN TestLoadDirectoryEntriesDirectShowsSystemEntriesWhenEnabled --- PASS: TestLoadDirectoryEntriesDirectShowsSystemEntriesWhenEnabled (0.00s) === RUN TestWholeFileServerCopyCandidate --- PASS: TestWholeFileServerCopyCandidate (0.00s) === RUN TestCopyFileRangeUsesServerSideWholeFileCopy --- PASS: TestCopyFileRangeUsesServerSideWholeFileCopy (0.00s) === RUN TestCopyFileRangeDoesNotFallbackAfterCommittedServerCopyRefreshFailure I0720 21:29:54.725218 leveldb_store.go:48 filer store dir: /tmp/TestCopyFileRangeDoesNotFallbackAfterCommittedServerCopyRefreshF1796802305/001/meta I0720 21:29:54.725230 file_util.go:29 Folder /tmp/TestCopyFileRangeDoesNotFallbackAfterCommittedServerCopyRefreshF1796802305/001/meta Permission: -rwxr-xr-x W0720 21:29:54.725650 weedfs_file_copy_range.go:223 CopyFileRange server-side copy /src.txt => /dst.txt committed but local refresh failed: reload copied entry: transient filer read failure --- PASS: TestCopyFileRangeDoesNotFallbackAfterCommittedServerCopyRefreshFailure (0.00s) === RUN TestCopyFileRangeReturnsEIOForAmbiguousServerSideCopy W0720 21:29:54.725981 weedfs_file_copy_range.go:230 CopyFileRange server-side copy /src.txt => /dst.txt outcome ambiguous: transport timeout after request dispatch --- PASS: TestCopyFileRangeReturnsEIOForAmbiguousServerSideCopy (0.00s) === RUN TestOpenKeepCache_FirstOpen --- PASS: TestOpenKeepCache_FirstOpen (0.00s) === RUN TestOpenKeepCache_SecondOpenSameMtime --- PASS: TestOpenKeepCache_SecondOpenSameMtime (0.00s) === RUN TestOpenKeepCache_MtimeChanged --- PASS: TestOpenKeepCache_MtimeChanged (0.00s) === RUN TestOpenKeepCache_NanosecondPrecision --- PASS: TestOpenKeepCache_NanosecondPrecision (0.00s) === RUN TestOpenKeepCache_WriteInvalidation --- PASS: TestOpenKeepCache_WriteInvalidation (0.00s) === RUN TestOpenKeepCache_WriteOpenSkipped --- PASS: TestOpenKeepCache_WriteOpenSkipped (0.00s) === RUN TestOpenKeepCache_BoundedEviction --- PASS: TestOpenKeepCache_BoundedEviction (0.00s) === RUN TestCreateCreatesAndOpensFile I0720 21:29:54.727716 leveldb_store.go:48 filer store dir: /tmp/TestCreateCreatesAndOpensFile4155137936/002/meta I0720 21:29:54.727731 file_util.go:29 Folder /tmp/TestCreateCreatesAndOpensFile4155137936/002/meta Permission: -rwxr-xr-x W0720 21:29:54.750280 weedfs_access.go:27 hasAccess: user.LookupId for uid 123 failed: user: unknown userid 123 --- PASS: TestCreateCreatesAndOpensFile (0.03s) === RUN TestReleaseFlushesDirtyCreateIfFlushWasSkipped I0720 21:29:54.751885 leveldb_store.go:48 filer store dir: /tmp/TestReleaseFlushesDirtyCreateIfFlushWasSkipped4027862116/002/meta I0720 21:29:54.751899 file_util.go:29 Folder /tmp/TestReleaseFlushesDirtyCreateIfFlushWasSkipped4027862116/002/meta Permission: -rwxr-xr-x W0720 21:29:54.755997 weedfs_access.go:27 hasAccess: user.LookupId for uid 123 failed: user: unknown userid 123 --- PASS: TestReleaseFlushesDirtyCreateIfFlushWasSkipped (0.01s) === RUN TestTruncateEntryClearsDirtyPagesForOpenHandle I0720 21:29:54.757747 leveldb_store.go:48 filer store dir: /tmp/TestTruncateEntryClearsDirtyPagesForOpenHandle274600662/002/meta I0720 21:29:54.757773 file_util.go:29 Folder /tmp/TestTruncateEntryClearsDirtyPagesForOpenHandle274600662/002/meta Permission: -rwxr-xr-x --- PASS: TestTruncateEntryClearsDirtyPagesForOpenHandle (0.01s) === RUN TestAccessChecksPermissions --- PASS: TestAccessChecksPermissions (0.00s) === RUN TestHasAccessUsesSupplementaryGroups --- PASS: TestHasAccessUsesSupplementaryGroups (0.00s) === RUN TestSupplementaryGroupCaching --- PASS: TestSupplementaryGroupCaching (0.00s) === RUN TestSupplementaryGroupCacheExpiry --- PASS: TestSupplementaryGroupCacheExpiry (0.00s) === RUN TestCreateExistingFileIgnoresQuotaPreflight I0720 21:29:54.767799 leveldb_store.go:48 filer store dir: /tmp/TestCreateExistingFileIgnoresQuotaPreflight1184485116/002/meta I0720 21:29:54.767836 file_util.go:29 Folder /tmp/TestCreateExistingFileIgnoresQuotaPreflight1184485116/002/meta Permission: -rwxr-xr-x I0720 21:29:54.769112 weedfs_file_mkrm.go:49 Create O_EXCL /existing.txt: already exists (uid=123 gid=456 mode=644) --- PASS: TestCreateExistingFileIgnoresQuotaPreflight (0.00s) === RUN TestAcquireHandleHonorsDefaultPermissions === RUN TestAcquireHandleHonorsDefaultPermissions/kernel_enforces I0720 21:29:54.769391 leveldb_store.go:48 filer store dir: /tmp/TestAcquireHandleHonorsDefaultPermissionskernel_enforces2323168982/002/meta I0720 21:29:54.769401 file_util.go:29 Folder /tmp/TestAcquireHandleHonorsDefaultPermissionskernel_enforces2323168982/002/meta Permission: -rwxr-xr-x === RUN TestAcquireHandleHonorsDefaultPermissions/mount_enforces I0720 21:29:54.772483 leveldb_store.go:48 filer store dir: /tmp/TestAcquireHandleHonorsDefaultPermissionsmount_enforces407391522/002/meta I0720 21:29:54.772495 file_util.go:29 Folder /tmp/TestAcquireHandleHonorsDefaultPermissionsmount_enforces407391522/002/meta Permission: -rwxr-xr-x --- PASS: TestAcquireHandleHonorsDefaultPermissions (0.00s) --- PASS: TestAcquireHandleHonorsDefaultPermissions/kernel_enforces (0.00s) --- PASS: TestAcquireHandleHonorsDefaultPermissions/mount_enforces (0.00s) === RUN TestCreateRegularFileValidatesParentUnderDefaultPermissions I0720 21:29:54.773926 leveldb_store.go:48 filer store dir: /tmp/TestCreateRegularFileValidatesParentUnderDefaultPermissions401034045/002/meta I0720 21:29:54.773939 file_util.go:29 Folder /tmp/TestCreateRegularFileValidatesParentUnderDefaultPermissions401034045/002/meta Permission: -rwxr-xr-x --- PASS: TestCreateRegularFileValidatesParentUnderDefaultPermissions (0.01s) === RUN TestShouldMergeChunks_NoChunks --- PASS: TestShouldMergeChunks_NoChunks (0.00s) === RUN TestShouldMergeChunks_SingleChunk --- PASS: TestShouldMergeChunks_SingleChunk (0.00s) === RUN TestShouldMergeChunks_NonOverlapping --- PASS: TestShouldMergeChunks_NonOverlapping (0.00s) === RUN TestShouldMergeChunks_ExactlyDouble --- PASS: TestShouldMergeChunks_ExactlyDouble (0.00s) === RUN TestShouldMergeChunks_JustOverDouble --- PASS: TestShouldMergeChunks_JustOverDouble (0.00s) === RUN TestShouldMergeChunks_ManifestExtendFileSize --- PASS: TestShouldMergeChunks_ManifestExtendFileSize (0.00s) === RUN TestShouldMergeChunks_ManifestSizesCounted --- PASS: TestShouldMergeChunks_ManifestSizesCounted (0.00s) === RUN TestShouldMergeChunks_AccumulatedManifests --- PASS: TestShouldMergeChunks_AccumulatedManifests (0.00s) === RUN TestShouldMergeChunks_DoesNotMutateInput --- PASS: TestShouldMergeChunks_DoesNotMutateInput (0.00s) === RUN TestCompactThenMergeCondition === RUN TestCompactThenMergeCondition/fully_superseded_chunks_removed,_no_bloat === RUN TestCompactThenMergeCondition/repeated_full_overwrites_all_compacted_away === RUN TestCompactThenMergeCondition/non-overlapping_small_chunks,_no_bloat === RUN TestCompactThenMergeCondition/staggered_partial_overlaps_create_bloat === RUN TestCompactThenMergeCondition/overlapping_writes_with_75pct_overlap === RUN TestCompactThenMergeCondition/many_4K_writes_at_1K_step_over_large_range --- PASS: TestCompactThenMergeCondition (0.00s) --- PASS: TestCompactThenMergeCondition/fully_superseded_chunks_removed,_no_bloat (0.00s) --- PASS: TestCompactThenMergeCondition/repeated_full_overwrites_all_compacted_away (0.00s) --- PASS: TestCompactThenMergeCondition/non-overlapping_small_chunks,_no_bloat (0.00s) --- PASS: TestCompactThenMergeCondition/staggered_partial_overlaps_create_bloat (0.00s) --- PASS: TestCompactThenMergeCondition/overlapping_writes_with_75pct_overlap (0.00s) --- PASS: TestCompactThenMergeCondition/many_4K_writes_at_1K_step_over_large_range (0.00s) === RUN TestRandomWritesBloatDetection --- PASS: TestRandomWritesBloatDetection (0.00s) === RUN TestFlushCycleManifestAccumulation weedfs_file_sync_test.go:344: cycle 2: merge correctly triggered (4 manifests, 10 regular chunks) --- PASS: TestFlushCycleManifestAccumulation (0.00s) === RUN TestVisibleContentPreservedAfterCompact --- PASS: TestVisibleContentPreservedAfterCompact (0.00s) === RUN TestFlushFileMetadataPreservesUserMtime --- PASS: TestFlushFileMetadataPreservesUserMtime (0.00s) === RUN TestPosixLockTypeMapping --- PASS: TestPosixLockTypeMapping (0.00s) === RUN TestPosixPollAcquireGrantedImmediately --- PASS: TestPosixPollAcquireGrantedImmediately (0.00s) === RUN TestPosixPollAcquireRetriesThenGrants --- PASS: TestPosixPollAcquireRetriesThenGrants (0.03s) === RUN TestPosixPollAcquireError --- PASS: TestPosixPollAcquireError (0.00s) === RUN TestPosixPollAcquireCancel --- PASS: TestPosixPollAcquireCancel (0.00s) === RUN TestPosixLockHint --- PASS: TestPosixLockHint (0.00s) === RUN TestUncommittedBytesTracking --- PASS: TestUncommittedBytesTracking (0.00s) === RUN TestUncommittedBytesDoesNotGoNegative --- PASS: TestUncommittedBytesDoesNotGoNegative (0.00s) === RUN TestIsOverQuotaWithUncommitted === RUN TestIsOverQuotaWithUncommitted/no_quota_set === RUN TestIsOverQuotaWithUncommitted/under_quota === RUN TestIsOverQuotaWithUncommitted/over_quota_with_uncommitted === RUN TestIsOverQuotaWithUncommitted/already_over_quota_flag_set === RUN TestIsOverQuotaWithUncommitted/exactly_at_quota === RUN TestIsOverQuotaWithUncommitted/one_byte_over_quota --- PASS: TestIsOverQuotaWithUncommitted (0.00s) --- PASS: TestIsOverQuotaWithUncommitted/no_quota_set (0.00s) --- PASS: TestIsOverQuotaWithUncommitted/under_quota (0.00s) --- PASS: TestIsOverQuotaWithUncommitted/over_quota_with_uncommitted (0.00s) --- PASS: TestIsOverQuotaWithUncommitted/already_over_quota_flag_set (0.00s) --- PASS: TestIsOverQuotaWithUncommitted/exactly_at_quota (0.00s) --- PASS: TestIsOverQuotaWithUncommitted/one_byte_over_quota (0.00s) === RUN TestGetQuotaCheckInterval === RUN TestGetQuotaCheckInterval/no_quota === RUN TestGetQuotaCheckInterval/under_90%_threshold === RUN TestGetQuotaCheckInterval/at_90%_threshold === RUN TestGetQuotaCheckInterval/over_90%_with_uncommitted --- PASS: TestGetQuotaCheckInterval (0.00s) --- PASS: TestGetQuotaCheckInterval/no_quota (0.00s) --- PASS: TestGetQuotaCheckInterval/under_90%_threshold (0.00s) --- PASS: TestGetQuotaCheckInterval/at_90%_threshold (0.00s) --- PASS: TestGetQuotaCheckInterval/over_90%_with_uncommitted (0.00s) === RUN TestNoQuotaTrackingWhenDisabled --- PASS: TestNoQuotaTrackingWhenDisabled (0.00s) === RUN TestHandleRenameResponseLeavesUncachedTargetOutOfCache I0720 21:29:54.817882 leveldb_store.go:48 filer store dir: /tmp/TestHandleRenameResponseLeavesUncachedTargetOutOfCache2517435340/001/meta I0720 21:29:54.817899 file_util.go:29 Folder /tmp/TestHandleRenameResponseLeavesUncachedTargetOutOfCache2517435340/001/meta Permission: -rwxr-xr-x --- PASS: TestHandleRenameResponseLeavesUncachedTargetOutOfCache (0.00s) === RUN TestSetXAttrCopiesValueBuffer --- PASS: TestSetXAttrCopiesValueBuffer (0.00s) === RUN TestSanitizeFuseName_InvalidBytesReplaced --- PASS: TestSanitizeFuseName_InvalidBytesReplaced (0.00s) === RUN TestSanitizeFuseName_PassThroughValidUTF8 --- PASS: TestSanitizeFuseName_PassThroughValidUTF8 (0.00s) === RUN TestCheckName_SanitizesBeforeLengthCheck --- PASS: TestCheckName_SanitizesBeforeLengthCheck (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mount 0.634s === RUN TestApplyMetadataResponseAppliesEventsInOrder I0720 21:29:54.250976 leveldb_store.go:48 filer store dir: /tmp/TestApplyMetadataResponseAppliesEventsInOrder1599950171/001/meta I0720 21:29:54.251173 file_util.go:29 Folder /tmp/TestApplyMetadataResponseAppliesEventsInOrder1599950171/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyMetadataResponseAppliesEventsInOrder (0.01s) === RUN TestApplyMetadataResponseRenamesAcrossCachedDirectories I0720 21:29:54.256353 leveldb_store.go:48 filer store dir: /tmp/TestApplyMetadataResponseRenamesAcrossCachedDirectories3691105329/001/meta I0720 21:29:54.256365 file_util.go:29 Folder /tmp/TestApplyMetadataResponseRenamesAcrossCachedDirectories3691105329/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyMetadataResponseRenamesAcrossCachedDirectories (0.01s) === RUN TestApplyMetadataResponseLocalOptionsSkipInvalidations I0720 21:29:54.268208 leveldb_store.go:48 filer store dir: /tmp/TestApplyMetadataResponseLocalOptionsSkipInvalidations1419470628/001/meta I0720 21:29:54.268223 file_util.go:29 Folder /tmp/TestApplyMetadataResponseLocalOptionsSkipInvalidations1419470628/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyMetadataResponseLocalOptionsSkipInvalidations (0.00s) === RUN TestApplyMetadataResponseDeduplicatesRepeatedFilerEvent I0720 21:29:54.269256 leveldb_store.go:48 filer store dir: /tmp/TestApplyMetadataResponseDeduplicatesRepeatedFilerEvent3611599825/001/meta I0720 21:29:54.269266 file_util.go:29 Folder /tmp/TestApplyMetadataResponseDeduplicatesRepeatedFilerEvent3611599825/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyMetadataResponseDeduplicatesRepeatedFilerEvent (0.02s) === RUN TestApplyMetadataResponseSkipsHiddenSystemEntryWhenDisabled I0720 21:29:54.286934 leveldb_store.go:48 filer store dir: /tmp/TestApplyMetadataResponseSkipsHiddenSystemEntryWhenDisabled1062144019/001/meta I0720 21:29:54.286951 file_util.go:29 Folder /tmp/TestApplyMetadataResponseSkipsHiddenSystemEntryWhenDisabled1062144019/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyMetadataResponseSkipsHiddenSystemEntryWhenDisabled (0.01s) === RUN TestApplyMetadataResponsePurgesHiddenDestinationPath I0720 21:29:54.293211 leveldb_store.go:48 filer store dir: /tmp/TestApplyMetadataResponsePurgesHiddenDestinationPath1903260471/001/meta I0720 21:29:54.293230 file_util.go:29 Folder /tmp/TestApplyMetadataResponsePurgesHiddenDestinationPath1903260471/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyMetadataResponsePurgesHiddenDestinationPath (0.01s) === RUN TestEnsureVisitedReplaysBufferedEventsAfterSnapshot I0720 21:29:54.301571 leveldb_store.go:48 filer store dir: /tmp/TestEnsureVisitedReplaysBufferedEventsAfterSnapshot1548608873/001/meta I0720 21:29:54.301587 file_util.go:29 Folder /tmp/TestEnsureVisitedReplaysBufferedEventsAfterSnapshot1548608873/001/meta Permission: -rwxr-xr-x --- PASS: TestEnsureVisitedReplaysBufferedEventsAfterSnapshot (0.00s) === RUN TestDirectoryNotificationsSuppressedDuringBuild I0720 21:29:54.303885 leveldb_store.go:48 filer store dir: /tmp/TestDirectoryNotificationsSuppressedDuringBuild3939935584/001/meta I0720 21:29:54.305086 file_util.go:29 Folder /tmp/TestDirectoryNotificationsSuppressedDuringBuild3939935584/001/meta Permission: -rwxr-xr-x --- PASS: TestDirectoryNotificationsSuppressedDuringBuild (0.01s) === RUN TestEmptyDirectoryBuildReplaysAllBufferedEvents I0720 21:29:54.309920 leveldb_store.go:48 filer store dir: /tmp/TestEmptyDirectoryBuildReplaysAllBufferedEvents3070698237/001/meta I0720 21:29:54.309932 file_util.go:29 Folder /tmp/TestEmptyDirectoryBuildReplaysAllBufferedEvents3070698237/001/meta Permission: -rwxr-xr-x --- PASS: TestEmptyDirectoryBuildReplaysAllBufferedEvents (0.02s) === RUN TestBuildCompletionSurvivesCallerCancellation I0720 21:29:54.333563 leveldb_store.go:48 filer store dir: /tmp/TestBuildCompletionSurvivesCallerCancellation2103126022/001/meta I0720 21:29:54.333576 file_util.go:29 Folder /tmp/TestBuildCompletionSurvivesCallerCancellation2103126022/001/meta Permission: -rwxr-xr-x --- PASS: TestBuildCompletionSurvivesCallerCancellation (0.01s) === RUN TestBufferedRenameUpdatesOtherDirectoryBeforeBuildCompletes I0720 21:29:54.344148 leveldb_store.go:48 filer store dir: /tmp/TestBufferedRenameUpdatesOtherDirectoryBeforeBuildCompletes2343425000/001/meta I0720 21:29:54.344163 file_util.go:29 Folder /tmp/TestBufferedRenameUpdatesOtherDirectoryBeforeBuildCompletes2343425000/001/meta Permission: -rwxr-xr-x --- PASS: TestBufferedRenameUpdatesOtherDirectoryBeforeBuildCompletes (0.00s) === RUN TestEnsureVisitedPreservesLocalOnlyEntry I0720 21:29:54.346945 leveldb_store.go:48 filer store dir: /tmp/TestEnsureVisitedPreservesLocalOnlyEntry3707371766/001/meta I0720 21:29:54.346959 file_util.go:29 Folder /tmp/TestEnsureVisitedPreservesLocalOnlyEntry3707371766/001/meta Permission: -rwxr-xr-x --- PASS: TestEnsureVisitedPreservesLocalOnlyEntry (0.01s) === RUN TestEnsureVisitedDropsUnpinnedStaleEntry I0720 21:29:54.352619 leveldb_store.go:48 filer store dir: /tmp/TestEnsureVisitedDropsUnpinnedStaleEntry325995783/001/meta I0720 21:29:54.352629 file_util.go:29 Folder /tmp/TestEnsureVisitedDropsUnpinnedStaleEntry325995783/001/meta Permission: -rwxr-xr-x --- PASS: TestEnsureVisitedDropsUnpinnedStaleEntry (0.01s) === RUN TestEnsureVisitedConfirmsTransientEmptyListing I0720 21:29:54.357971 leveldb_store.go:48 filer store dir: /tmp/TestEnsureVisitedConfirmsTransientEmptyListing1001566155/001/meta I0720 21:29:54.357981 file_util.go:29 Folder /tmp/TestEnsureVisitedConfirmsTransientEmptyListing1001566155/001/meta Permission: -rwxr-xr-x W0720 21:29:54.359586 meta_cache_init.go:170 rebuild of /dir saw a transient empty listing, recovered 1 entries on confirmation --- PASS: TestEnsureVisitedConfirmsTransientEmptyListing (0.00s) === RUN TestEnsureVisitedCachesGenuinelyEmptyDirectory I0720 21:29:54.359912 leveldb_store.go:48 filer store dir: /tmp/TestEnsureVisitedCachesGenuinelyEmptyDirectory3603331956/001/meta I0720 21:29:54.359921 file_util.go:29 Folder /tmp/TestEnsureVisitedCachesGenuinelyEmptyDirectory3603331956/001/meta Permission: -rwxr-xr-x --- PASS: TestEnsureVisitedCachesGenuinelyEmptyDirectory (0.06s) === RUN TestApplyLoopInvalidateDoesNotDeadlockWithLockHoldingEnqueuer I0720 21:29:54.420867 leveldb_store.go:48 filer store dir: /tmp/TestApplyLoopInvalidateDoesNotDeadlockWithLockHoldingEnqueuer2660939673/001/meta I0720 21:29:54.420892 file_util.go:29 Folder /tmp/TestApplyLoopInvalidateDoesNotDeadlockWithLockHoldingEnqueuer2660939673/001/meta Permission: -rwxr-xr-x --- PASS: TestApplyLoopInvalidateDoesNotDeadlockWithLockHoldingEnqueuer (0.06s) === RUN TestPurgeSkippedWhileDirectoryBuilding I0720 21:29:54.477596 leveldb_store.go:48 filer store dir: /tmp/TestPurgeSkippedWhileDirectoryBuilding2289382115/001/meta I0720 21:29:54.477611 file_util.go:29 Folder /tmp/TestPurgeSkippedWhileDirectoryBuilding2289382115/001/meta Permission: -rwxr-xr-x --- PASS: TestPurgeSkippedWhileDirectoryBuilding (0.00s) === RUN TestPurgeClearsWhenNotBuilding I0720 21:29:54.480217 leveldb_store.go:48 filer store dir: /tmp/TestPurgeClearsWhenNotBuilding1256902718/001/meta I0720 21:29:54.480229 file_util.go:29 Folder /tmp/TestPurgeClearsWhenNotBuilding1256902718/001/meta Permission: -rwxr-xr-x --- PASS: TestPurgeClearsWhenNotBuilding (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/mount/meta_cache 0.300s === RUN Test_PageChunkWrittenIntervalList --- PASS: Test_PageChunkWrittenIntervalList (0.00s) === RUN Test_PageChunkWrittenIntervalList1 --- PASS: Test_PageChunkWrittenIntervalList1 (0.00s) === RUN TestIsContiguouslyWritten === RUN TestIsContiguouslyWritten/empty === RUN TestIsContiguouslyWritten/single_full === RUN TestIsContiguouslyWritten/single_partial_at_start === RUN TestIsContiguouslyWritten/two_adjacent_halves_in_order === RUN TestIsContiguouslyWritten/two_adjacent_halves_out_of_order === RUN TestIsContiguouslyWritten/overlapping_covers_everything === RUN TestIsContiguouslyWritten/three_adjacent_thirds === RUN TestIsContiguouslyWritten/single_partial_in_middle === RUN TestIsContiguouslyWritten/first_byte_missing === RUN TestIsContiguouslyWritten/gap_in_middle === RUN TestIsContiguouslyWritten/two_intervals_with_1-byte_gap === RUN TestIsContiguouslyWritten/three_intervals_with_middle_gap --- PASS: TestIsContiguouslyWritten (0.00s) --- PASS: TestIsContiguouslyWritten/empty (0.00s) --- PASS: TestIsContiguouslyWritten/single_full (0.00s) --- PASS: TestIsContiguouslyWritten/single_partial_at_start (0.00s) --- PASS: TestIsContiguouslyWritten/two_adjacent_halves_in_order (0.00s) --- PASS: TestIsContiguouslyWritten/two_adjacent_halves_out_of_order (0.00s) --- PASS: TestIsContiguouslyWritten/overlapping_covers_everything (0.00s) --- PASS: TestIsContiguouslyWritten/three_adjacent_thirds (0.00s) --- PASS: TestIsContiguouslyWritten/single_partial_in_middle (0.00s) --- PASS: TestIsContiguouslyWritten/first_byte_missing (0.00s) --- PASS: TestIsContiguouslyWritten/gap_in_middle (0.00s) --- PASS: TestIsContiguouslyWritten/two_intervals_with_1-byte_gap (0.00s) --- PASS: TestIsContiguouslyWritten/three_intervals_with_middle_gap (0.00s) === RUN TestIsComplete_AdjacentIntervals === RUN TestIsComplete_AdjacentIntervals/single_full_write === RUN TestIsComplete_AdjacentIntervals/two_adjacent_halves_in_order === RUN TestIsComplete_AdjacentIntervals/two_adjacent_halves_out_of_order === RUN TestIsComplete_AdjacentIntervals/eight_adjacent_eighths === RUN TestIsComplete_AdjacentIntervals/gap_in_the_middle === RUN TestIsComplete_AdjacentIntervals/missing_last_byte === RUN TestIsComplete_AdjacentIntervals/missing_first_byte === RUN TestIsComplete_AdjacentIntervals/overlapping_intervals_covering_everything --- PASS: TestIsComplete_AdjacentIntervals (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/single_full_write (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/two_adjacent_halves_in_order (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/two_adjacent_halves_out_of_order (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/eight_adjacent_eighths (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/gap_in_the_middle (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/missing_last_byte (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/missing_first_byte (0.00s) --- PASS: TestIsComplete_AdjacentIntervals/overlapping_intervals_covering_everything (0.00s) === RUN TestSwapFile_NewSwapFileChunk_Concurrent --- PASS: TestSwapFile_NewSwapFileChunk_Concurrent (0.00s) === RUN TestSwapFile_MkdirAll_Permissions --- PASS: TestSwapFile_MkdirAll_Permissions (0.00s) === RUN TestSwapFile_RecreateDir --- PASS: TestSwapFile_RecreateDir (0.00s) === RUN TestShutdown_ReleasesWritableReservation --- PASS: TestShutdown_ReleasesWritableReservation (0.00s) === RUN TestSealedChunk_FreeReferenceIsIdempotent --- PASS: TestSealedChunk_FreeReferenceIsIdempotent (0.00s) === RUN TestUploadPipeline --- PASS: TestUploadPipeline (25.66s) === RUN TestEvictOneWritableChunk_SkipsGappyChunks --- PASS: TestEvictOneWritableChunk_SkipsGappyChunks (0.00s) === RUN TestEvictOneWritableChunk_FallbackPicksOldestGappy --- PASS: TestEvictOneWritableChunk_FallbackPicksOldestGappy (0.02s) === RUN TestEvictOneWritableChunk_PrefersStrictOverFallback --- PASS: TestEvictOneWritableChunk_PrefersStrictOverFallback (0.00s) === RUN TestProactiveFlush_SkipsGappyChunks --- PASS: TestProactiveFlush_SkipsGappyChunks (0.00s) === RUN TestWriteBufferAccountant_Unlimited --- PASS: TestWriteBufferAccountant_Unlimited (0.00s) === RUN TestWriteBufferAccountant_Nil --- PASS: TestWriteBufferAccountant_Nil (0.00s) === RUN TestWriteBufferAccountant_ReserveAndRelease --- PASS: TestWriteBufferAccountant_ReserveAndRelease (0.00s) === RUN TestWriteBufferAccountant_BlocksWhenOverCap --- PASS: TestWriteBufferAccountant_BlocksWhenOverCap (0.06s) === RUN TestWriteBufferAccountant_SoftThrottle --- PASS: TestWriteBufferAccountant_SoftThrottle (0.01s) === RUN TestWriteBufferAccountant_HardThrottle --- PASS: TestWriteBufferAccountant_HardThrottle (0.05s) === RUN TestWriteBufferAccountant_NoThrottleBelowSoft --- PASS: TestWriteBufferAccountant_NoThrottleBelowSoft (0.00s) === RUN TestWriteBufferAccountant_ThrottleCounters --- PASS: TestWriteBufferAccountant_ThrottleCounters (0.06s) === RUN TestWriteBufferAccountant_GraduatedRecovery --- PASS: TestWriteBufferAccountant_GraduatedRecovery (0.07s) === RUN TestWriteBufferAccountant_AllowsOversizeWhenEmpty --- PASS: TestWriteBufferAccountant_AllowsOversizeWhenEmpty (0.00s) === RUN TestWriteBufferCap_SharedAcrossPipelines write_buffer_cap_integration_test.go:167: peak inflight uploads=4, peak Used()=262144 bytes (cap=262144) --- PASS: TestWriteBufferCap_SharedAcrossPipelines (0.58s) PASS ok github.com/seaweedfs/seaweedfs/weed/mount/page_writer 26.540s ? github.com/seaweedfs/seaweedfs/weed/mount/unmount [no test files] ? github.com/seaweedfs/seaweedfs/weed/mq [no test files] ? github.com/seaweedfs/seaweedfs/weed/mq/agent [no test files] === RUN TestConvertOffsetToMessagePosition === RUN TestConvertOffsetToMessagePosition/reset_to_earliest === RUN TestConvertOffsetToMessagePosition/reset_to_latest === RUN TestConvertOffsetToMessagePosition/exact_offset_zero === RUN TestConvertOffsetToMessagePosition/exact_offset_non-zero === RUN TestConvertOffsetToMessagePosition/exact_timestamp --- PASS: TestConvertOffsetToMessagePosition (0.00s) --- PASS: TestConvertOffsetToMessagePosition/reset_to_earliest (0.00s) --- PASS: TestConvertOffsetToMessagePosition/reset_to_latest (0.00s) --- PASS: TestConvertOffsetToMessagePosition/exact_offset_zero (0.00s) --- PASS: TestConvertOffsetToMessagePosition/exact_offset_non-zero (0.00s) --- PASS: TestConvertOffsetToMessagePosition/exact_timestamp (0.00s) === RUN TestConvertOffsetToMessagePosition_OffsetEncoding === RUN TestConvertOffsetToMessagePosition_OffsetEncoding/offset_10 === RUN TestConvertOffsetToMessagePosition_OffsetEncoding/offset_100 === RUN TestConvertOffsetToMessagePosition_OffsetEncoding/offset_0 === RUN TestConvertOffsetToMessagePosition_OffsetEncoding/offset_42 --- PASS: TestConvertOffsetToMessagePosition_OffsetEncoding (0.00s) --- PASS: TestConvertOffsetToMessagePosition_OffsetEncoding/offset_10 (0.00s) --- PASS: TestConvertOffsetToMessagePosition_OffsetEncoding/offset_100 (0.00s) --- PASS: TestConvertOffsetToMessagePosition_OffsetEncoding/offset_0 (0.00s) --- PASS: TestConvertOffsetToMessagePosition_OffsetEncoding/offset_42 (0.00s) === RUN TestConvertOffsetToMessagePosition_ConsistentResults --- PASS: TestConvertOffsetToMessagePosition_ConsistentResults (0.01s) === RUN TestConvertOffsetToMessagePosition_FixVerification --- PASS: TestConvertOffsetToMessagePosition_FixVerification (0.03s) === RUN TestPartitionIdentityConsistency --- PASS: TestPartitionIdentityConsistency (0.00s) === RUN TestBrokerOffsetManager_GetSubscription_Fixed --- PASS: TestBrokerOffsetManager_GetSubscription_Fixed (0.00s) === RUN TestBrokerOffsetManager_ListActiveSubscriptions_Fixed --- PASS: TestBrokerOffsetManager_ListActiveSubscriptions_Fixed (0.00s) === RUN TestMessageQueueBroker_ListActiveSubscriptions_Fixed --- PASS: TestMessageQueueBroker_ListActiveSubscriptions_Fixed (0.00s) === RUN TestSingleWriterPerPartitionCorrectness --- PASS: TestSingleWriterPerPartitionCorrectness (0.00s) === RUN TestEndToEndWorkflowAfterFixes --- PASS: TestEndToEndWorkflowAfterFixes (0.00s) === RUN TestValidateRecordValue --- PASS: TestValidateRecordValue (0.00s) === RUN TestValidateRecordValueEmptyFields --- PASS: TestValidateRecordValueEmptyFields (0.00s) === RUN TestValidateRecordValueNonKafkaTopic --- PASS: TestValidateRecordValueNonKafkaTopic (0.00s) === RUN TestValidateRecordValueNilInputs --- PASS: TestValidateRecordValueNilInputs (0.00s) === RUN TestRecordValueMarshalUnmarshalIntegration --- PASS: TestRecordValueMarshalUnmarshalIntegration (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/broker 0.090s ? github.com/seaweedfs/seaweedfs/weed/mq/client/agent_client [no test files] ? github.com/seaweedfs/seaweedfs/weed/mq/client/pub_client [no test files] ? github.com/seaweedfs/seaweedfs/weed/mq/client/sub_client [no test files] ? github.com/seaweedfs/seaweedfs/weed/mq/kafka [no test files] === RUN TestCompressionCodec_String === RUN TestCompressionCodec_String/none === RUN TestCompressionCodec_String/gzip === RUN TestCompressionCodec_String/snappy === RUN TestCompressionCodec_String/lz4 === RUN TestCompressionCodec_String/zstd === RUN TestCompressionCodec_String/unknown(99) --- PASS: TestCompressionCodec_String (0.00s) --- PASS: TestCompressionCodec_String/none (0.00s) --- PASS: TestCompressionCodec_String/gzip (0.00s) --- PASS: TestCompressionCodec_String/snappy (0.00s) --- PASS: TestCompressionCodec_String/lz4 (0.00s) --- PASS: TestCompressionCodec_String/zstd (0.00s) --- PASS: TestCompressionCodec_String/unknown(99) (0.00s) === RUN TestCompressionCodec_IsValid === RUN TestCompressionCodec_IsValid/none === RUN TestCompressionCodec_IsValid/gzip === RUN TestCompressionCodec_IsValid/snappy === RUN TestCompressionCodec_IsValid/lz4 === RUN TestCompressionCodec_IsValid/zstd === RUN TestCompressionCodec_IsValid/unknown(-1) === RUN TestCompressionCodec_IsValid/unknown(5) === RUN TestCompressionCodec_IsValid/unknown(99) --- PASS: TestCompressionCodec_IsValid (0.00s) --- PASS: TestCompressionCodec_IsValid/none (0.00s) --- PASS: TestCompressionCodec_IsValid/gzip (0.00s) --- PASS: TestCompressionCodec_IsValid/snappy (0.00s) --- PASS: TestCompressionCodec_IsValid/lz4 (0.00s) --- PASS: TestCompressionCodec_IsValid/zstd (0.00s) --- PASS: TestCompressionCodec_IsValid/unknown(-1) (0.00s) --- PASS: TestCompressionCodec_IsValid/unknown(5) (0.00s) --- PASS: TestCompressionCodec_IsValid/unknown(99) (0.00s) === RUN TestExtractCompressionCodec === RUN TestExtractCompressionCodec/None === RUN TestExtractCompressionCodec/Gzip === RUN TestExtractCompressionCodec/Snappy === RUN TestExtractCompressionCodec/Lz4 === RUN TestExtractCompressionCodec/Zstd === RUN TestExtractCompressionCodec/Gzip_with_transactional === RUN TestExtractCompressionCodec/Snappy_with_control === RUN TestExtractCompressionCodec/Lz4_with_both_flags --- PASS: TestExtractCompressionCodec (0.00s) --- PASS: TestExtractCompressionCodec/None (0.00s) --- PASS: TestExtractCompressionCodec/Gzip (0.00s) --- PASS: TestExtractCompressionCodec/Snappy (0.00s) --- PASS: TestExtractCompressionCodec/Lz4 (0.00s) --- PASS: TestExtractCompressionCodec/Zstd (0.00s) --- PASS: TestExtractCompressionCodec/Gzip_with_transactional (0.00s) --- PASS: TestExtractCompressionCodec/Snappy_with_control (0.00s) --- PASS: TestExtractCompressionCodec/Lz4_with_both_flags (0.00s) === RUN TestSetCompressionCodec === RUN TestSetCompressionCodec/Set_None === RUN TestSetCompressionCodec/Set_Gzip === RUN TestSetCompressionCodec/Set_Snappy === RUN TestSetCompressionCodec/Set_Lz4 === RUN TestSetCompressionCodec/Set_Zstd === RUN TestSetCompressionCodec/Replace_Gzip_with_Snappy === RUN TestSetCompressionCodec/Set_Gzip_preserving_transactional === RUN TestSetCompressionCodec/Set_Lz4_preserving_control === RUN TestSetCompressionCodec/Set_Zstd_preserving_both_flags --- PASS: TestSetCompressionCodec (0.00s) --- PASS: TestSetCompressionCodec/Set_None (0.00s) --- PASS: TestSetCompressionCodec/Set_Gzip (0.00s) --- PASS: TestSetCompressionCodec/Set_Snappy (0.00s) --- PASS: TestSetCompressionCodec/Set_Lz4 (0.00s) --- PASS: TestSetCompressionCodec/Set_Zstd (0.00s) --- PASS: TestSetCompressionCodec/Replace_Gzip_with_Snappy (0.00s) --- PASS: TestSetCompressionCodec/Set_Gzip_preserving_transactional (0.00s) --- PASS: TestSetCompressionCodec/Set_Lz4_preserving_control (0.00s) --- PASS: TestSetCompressionCodec/Set_Zstd_preserving_both_flags (0.00s) === RUN TestCompress_None --- PASS: TestCompress_None (0.00s) === RUN TestCompress_Gzip --- PASS: TestCompress_Gzip (0.00s) === RUN TestCompress_Snappy --- PASS: TestCompress_Snappy (0.00s) === RUN TestCompress_Lz4 --- PASS: TestCompress_Lz4 (0.01s) === RUN TestCompress_Zstd --- PASS: TestCompress_Zstd (0.00s) === RUN TestCompress_InvalidCodec --- PASS: TestCompress_InvalidCodec (0.00s) === RUN TestDecompress_None --- PASS: TestDecompress_None (0.00s) === RUN TestRoundTrip === RUN TestRoundTrip/none === RUN TestRoundTrip/none/data_0 === RUN TestRoundTrip/none/data_1 === RUN TestRoundTrip/none/data_2 === RUN TestRoundTrip/none/data_3 === RUN TestRoundTrip/none/data_4 === RUN TestRoundTrip/none/data_5 === RUN TestRoundTrip/gzip === RUN TestRoundTrip/gzip/data_0 === RUN TestRoundTrip/gzip/data_1 === RUN TestRoundTrip/gzip/data_2 === RUN TestRoundTrip/gzip/data_3 === RUN TestRoundTrip/gzip/data_4 === RUN TestRoundTrip/gzip/data_5 === RUN TestRoundTrip/snappy === RUN TestRoundTrip/snappy/data_0 === RUN TestRoundTrip/snappy/data_1 === RUN TestRoundTrip/snappy/data_2 === RUN TestRoundTrip/snappy/data_3 === RUN TestRoundTrip/snappy/data_4 === RUN TestRoundTrip/snappy/data_5 === RUN TestRoundTrip/lz4 === RUN TestRoundTrip/lz4/data_0 === RUN TestRoundTrip/lz4/data_1 === RUN TestRoundTrip/lz4/data_2 === RUN TestRoundTrip/lz4/data_3 === RUN TestRoundTrip/lz4/data_4 === RUN TestRoundTrip/lz4/data_5 === RUN TestRoundTrip/zstd === RUN TestRoundTrip/zstd/data_0 === RUN TestRoundTrip/zstd/data_1 === RUN TestRoundTrip/zstd/data_2 === RUN TestRoundTrip/zstd/data_3 === RUN TestRoundTrip/zstd/data_4 === RUN TestRoundTrip/zstd/data_5 --- PASS: TestRoundTrip (0.03s) --- PASS: TestRoundTrip/none (0.00s) --- PASS: TestRoundTrip/none/data_0 (0.00s) --- PASS: TestRoundTrip/none/data_1 (0.00s) --- PASS: TestRoundTrip/none/data_2 (0.00s) --- PASS: TestRoundTrip/none/data_3 (0.00s) --- PASS: TestRoundTrip/none/data_4 (0.00s) --- PASS: TestRoundTrip/none/data_5 (0.00s) --- PASS: TestRoundTrip/gzip (0.00s) --- PASS: TestRoundTrip/gzip/data_0 (0.00s) --- PASS: TestRoundTrip/gzip/data_1 (0.00s) --- PASS: TestRoundTrip/gzip/data_2 (0.00s) --- PASS: TestRoundTrip/gzip/data_3 (0.00s) --- PASS: TestRoundTrip/gzip/data_4 (0.00s) --- PASS: TestRoundTrip/gzip/data_5 (0.00s) --- PASS: TestRoundTrip/snappy (0.00s) --- PASS: TestRoundTrip/snappy/data_0 (0.00s) --- PASS: TestRoundTrip/snappy/data_1 (0.00s) --- PASS: TestRoundTrip/snappy/data_2 (0.00s) --- PASS: TestRoundTrip/snappy/data_3 (0.00s) --- PASS: TestRoundTrip/snappy/data_4 (0.00s) --- PASS: TestRoundTrip/snappy/data_5 (0.00s) --- PASS: TestRoundTrip/lz4 (0.00s) --- PASS: TestRoundTrip/lz4/data_0 (0.00s) --- PASS: TestRoundTrip/lz4/data_1 (0.00s) --- PASS: TestRoundTrip/lz4/data_2 (0.00s) --- PASS: TestRoundTrip/lz4/data_3 (0.00s) --- PASS: TestRoundTrip/lz4/data_4 (0.00s) --- PASS: TestRoundTrip/lz4/data_5 (0.00s) --- PASS: TestRoundTrip/zstd (0.03s) --- PASS: TestRoundTrip/zstd/data_0 (0.01s) --- PASS: TestRoundTrip/zstd/data_1 (0.00s) --- PASS: TestRoundTrip/zstd/data_2 (0.01s) --- PASS: TestRoundTrip/zstd/data_3 (0.00s) --- PASS: TestRoundTrip/zstd/data_4 (0.01s) --- PASS: TestRoundTrip/zstd/data_5 (0.00s) === RUN TestDecompress_InvalidCodec --- PASS: TestDecompress_InvalidCodec (0.00s) === RUN TestDecompress_CorruptedData === RUN TestDecompress_CorruptedData/gzip === RUN TestDecompress_CorruptedData/snappy === RUN TestDecompress_CorruptedData/lz4 === RUN TestDecompress_CorruptedData/zstd --- PASS: TestDecompress_CorruptedData (0.00s) --- PASS: TestDecompress_CorruptedData/gzip (0.00s) --- PASS: TestDecompress_CorruptedData/snappy (0.00s) --- PASS: TestDecompress_CorruptedData/lz4 (0.00s) --- PASS: TestDecompress_CorruptedData/zstd (0.00s) === RUN TestCompressRecordBatch === RUN TestCompressRecordBatch/None_codec === RUN TestCompressRecordBatch/Gzip_codec === RUN TestCompressRecordBatch/Snappy_codec --- PASS: TestCompressRecordBatch (0.01s) --- PASS: TestCompressRecordBatch/None_codec (0.00s) --- PASS: TestCompressRecordBatch/Gzip_codec (0.01s) --- PASS: TestCompressRecordBatch/Snappy_codec (0.00s) === RUN TestDecompressRecordBatch === RUN TestDecompressRecordBatch/None_codec === RUN TestDecompressRecordBatch/Round_trip_with_Gzip === RUN TestDecompressRecordBatch/Round_trip_with_Snappy --- PASS: TestDecompressRecordBatch (0.00s) --- PASS: TestDecompressRecordBatch/None_codec (0.00s) --- PASS: TestDecompressRecordBatch/Round_trip_with_Gzip (0.00s) --- PASS: TestDecompressRecordBatch/Round_trip_with_Snappy (0.00s) === RUN TestCompressionEfficiency === RUN TestCompressionEfficiency/gzip compression_test.go:304: Codec: gzip, Original: 5100 bytes, Compressed: 98 bytes, Ratio: 0.02 === RUN TestCompressionEfficiency/snappy compression_test.go:304: Codec: snappy, Original: 5100 bytes, Compressed: 291 bytes, Ratio: 0.06 === RUN TestCompressionEfficiency/lz4 compression_test.go:304: Codec: lz4, Original: 5100 bytes, Compressed: 112 bytes, Ratio: 0.02 === RUN TestCompressionEfficiency/zstd compression_test.go:304: Codec: zstd, Original: 5100 bytes, Compressed: 68 bytes, Ratio: 0.01 --- PASS: TestCompressionEfficiency (0.01s) --- PASS: TestCompressionEfficiency/gzip (0.00s) --- PASS: TestCompressionEfficiency/snappy (0.00s) --- PASS: TestCompressionEfficiency/lz4 (0.01s) --- PASS: TestCompressionEfficiency/zstd (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/compression 0.087s === RUN TestRangeAssignmentStrategy --- PASS: TestRangeAssignmentStrategy (0.00s) === RUN TestRangeAssignmentStrategy_UnevenPartitions --- PASS: TestRangeAssignmentStrategy_UnevenPartitions (0.00s) === RUN TestRangeAssignmentStrategy_MultipleTopics --- PASS: TestRangeAssignmentStrategy_MultipleTopics (0.00s) === RUN TestRoundRobinAssignmentStrategy --- PASS: TestRoundRobinAssignmentStrategy (0.00s) === RUN TestRoundRobinAssignmentStrategy_MultipleTopics --- PASS: TestRoundRobinAssignmentStrategy_MultipleTopics (0.00s) === RUN TestGetAssignmentStrategy --- PASS: TestGetAssignmentStrategy (0.00s) === RUN TestConsumerGroup_AssignPartitions --- PASS: TestConsumerGroup_AssignPartitions (0.00s) === RUN TestConsumerGroup_GetMemberAssignments --- PASS: TestConsumerGroup_GetMemberAssignments (0.00s) === RUN TestConsumerGroup_UpdateMemberSubscription --- PASS: TestConsumerGroup_UpdateMemberSubscription (0.00s) === RUN TestAssignmentStrategy_EmptyMembers --- PASS: TestAssignmentStrategy_EmptyMembers (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_Name --- PASS: TestCooperativeStickyAssignmentStrategy_Name (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_InitialAssignment --- PASS: TestCooperativeStickyAssignmentStrategy_InitialAssignment (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_StickyBehavior --- PASS: TestCooperativeStickyAssignmentStrategy_StickyBehavior (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_NewMemberJoin cooperative_sticky_test.go:187: Member1 retained 2 out of 4 original partitions --- PASS: TestCooperativeStickyAssignmentStrategy_NewMemberJoin (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_MemberLeave --- PASS: TestCooperativeStickyAssignmentStrategy_MemberLeave (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_MultipleTopics --- PASS: TestCooperativeStickyAssignmentStrategy_MultipleTopics (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_UnevenPartitions --- PASS: TestCooperativeStickyAssignmentStrategy_UnevenPartitions (0.00s) === RUN TestCooperativeStickyAssignmentStrategy_PartialSubscription --- PASS: TestCooperativeStickyAssignmentStrategy_PartialSubscription (0.00s) === RUN TestGetAssignmentStrategy_CooperativeSticky --- PASS: TestGetAssignmentStrategy_CooperativeSticky (0.00s) === RUN TestGroupCoordinator_CreateGroup --- PASS: TestGroupCoordinator_CreateGroup (0.00s) === RUN TestGroupCoordinator_ValidateSessionTimeout --- PASS: TestGroupCoordinator_ValidateSessionTimeout (0.00s) === RUN TestGroupCoordinator_MemberManagement --- PASS: TestGroupCoordinator_MemberManagement (0.00s) === RUN TestGroupCoordinator_Stats --- PASS: TestGroupCoordinator_Stats (0.00s) === RUN TestGroupCoordinator_Cleanup --- PASS: TestGroupCoordinator_Cleanup (0.00s) === RUN TestGroupCoordinator_GenerateMemberID --- PASS: TestGroupCoordinator_GenerateMemberID (0.00s) === RUN TestGroupCoordinator_EvictExpiredMembersLocked --- PASS: TestGroupCoordinator_EvictExpiredMembersLocked (0.00s) === RUN TestGroupCoordinator_Cleanup_SurvivorsRebalance --- PASS: TestGroupCoordinator_Cleanup_SurvivorsRebalance (0.00s) === RUN TestGroupCoordinator_EvictExpiredMembersLocked_ZeroSessionTimeout --- PASS: TestGroupCoordinator_EvictExpiredMembersLocked_ZeroSessionTimeout (0.00s) === RUN TestIncrementalCooperativeAssignmentStrategy_BasicAssignment incremental_rebalancing_test.go:41: Member member-1 assigned 2 partitions: [{topic-1 0} {topic-1 1}] incremental_rebalancing_test.go:41: Member member-2 assigned 2 partitions: [{topic-1 2} {topic-1 3}] --- PASS: TestIncrementalCooperativeAssignmentStrategy_BasicAssignment (0.00s) === RUN TestIncrementalCooperativeAssignmentStrategy_RebalanceWithRevocation incremental_rebalancing_test.go:106: Revocation phase - Member-1: 2 partitions, Member-2: 0 partitions incremental_rebalancing_test.go:139: Final assignment - Member-1: 2 partitions, Member-2: 2 partitions --- PASS: TestIncrementalCooperativeAssignmentStrategy_RebalanceWithRevocation (0.01s) === RUN TestIncrementalCooperativeAssignmentStrategy_NoRevocationNeeded --- PASS: TestIncrementalCooperativeAssignmentStrategy_NoRevocationNeeded (0.00s) === RUN TestIncrementalCooperativeAssignmentStrategy_MultipleTopics incremental_rebalancing_test.go:244: All assigned partitions: map[topic-1:0:true topic-1:1:true topic-1:2:true topic-2:0:true topic-2:1:true] --- PASS: TestIncrementalCooperativeAssignmentStrategy_MultipleTopics (0.00s) === RUN TestIncrementalCooperativeAssignmentStrategy_ForceComplete --- PASS: TestIncrementalCooperativeAssignmentStrategy_ForceComplete (0.00s) === RUN TestIncrementalCooperativeAssignmentStrategy_RevocationTimeout --- PASS: TestIncrementalCooperativeAssignmentStrategy_RevocationTimeout (0.01s) === RUN TestIncrementalCooperativeAssignmentStrategy_StateTransitions --- PASS: TestIncrementalCooperativeAssignmentStrategy_StateTransitions (0.00s) === RUN TestRebalanceTimeoutManager_CheckRebalanceTimeouts --- PASS: TestRebalanceTimeoutManager_CheckRebalanceTimeouts (0.00s) === RUN TestRebalanceTimeoutManager_SessionTimeoutFallback --- PASS: TestRebalanceTimeoutManager_SessionTimeoutFallback (0.00s) === RUN TestRebalanceTimeoutManager_LeaderEviction --- PASS: TestRebalanceTimeoutManager_LeaderEviction (0.00s) === RUN TestRebalanceTimeoutManager_IsRebalanceStuck --- PASS: TestRebalanceTimeoutManager_IsRebalanceStuck (0.00s) === RUN TestRebalanceTimeoutManager_ForceCompleteRebalance --- PASS: TestRebalanceTimeoutManager_ForceCompleteRebalance (0.00s) === RUN TestRebalanceTimeoutManager_GetRebalanceStatus --- PASS: TestRebalanceTimeoutManager_GetRebalanceStatus (0.00s) === RUN TestRebalanceTimeoutManager_DefaultRebalanceTimeout --- PASS: TestRebalanceTimeoutManager_DefaultRebalanceTimeout (0.00s) === RUN TestGroupCoordinator_StaticMembership --- PASS: TestGroupCoordinator_StaticMembership (0.00s) === RUN TestGroupCoordinator_StaticMemberReconnection --- PASS: TestGroupCoordinator_StaticMemberReconnection (0.00s) === RUN TestGroupCoordinator_StaticMembershipEdgeCases --- PASS: TestGroupCoordinator_StaticMembershipEdgeCases (0.00s) === RUN TestGroupCoordinator_StaticMembershipConcurrency --- PASS: TestGroupCoordinator_StaticMembershipConcurrency (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/consumer 0.042s === RUN TestMemoryStorageCommitAndFetch --- PASS: TestMemoryStorageCommitAndFetch (0.00s) === RUN TestMemoryStorageFetchNonExistent --- PASS: TestMemoryStorageFetchNonExistent (0.00s) === RUN TestMemoryStorageFetchAllOffsets --- PASS: TestMemoryStorageFetchAllOffsets (0.00s) === RUN TestMemoryStorageDeleteGroup --- PASS: TestMemoryStorageDeleteGroup (0.00s) === RUN TestMemoryStorageListGroups --- PASS: TestMemoryStorageListGroups (0.00s) === RUN TestMemoryStorageConcurrency --- PASS: TestMemoryStorageConcurrency (0.00s) === RUN TestMemoryStorageInvalidInputs --- PASS: TestMemoryStorageInvalidInputs (0.00s) === RUN TestMemoryStorageClosedOperations --- PASS: TestMemoryStorageClosedOperations (0.00s) === RUN TestMemoryStorageOverwrite --- PASS: TestMemoryStorageOverwrite (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/consumer_offset 0.046s === RUN TestCoordinatorRegistry_DeterministicNodeID --- PASS: TestCoordinatorRegistry_DeterministicNodeID (0.00s) === RUN TestCoordinatorRegistry_BasicOperations --- PASS: TestCoordinatorRegistry_BasicOperations (0.00s) === RUN TestCoordinatorRegistry_AssignCoordinator --- PASS: TestCoordinatorRegistry_AssignCoordinator (0.00s) === RUN TestCoordinatorRegistry_HealthyGateways --- PASS: TestCoordinatorRegistry_HealthyGateways (0.00s) === RUN TestCoordinatorRegistry_ConsistentHashing --- PASS: TestCoordinatorRegistry_ConsistentHashing (0.00s) === RUN TestCoordinatorRegistry_CleanupStaleEntries --- PASS: TestCoordinatorRegistry_CleanupStaleEntries (0.00s) === RUN TestCoordinatorRegistry_GetStats --- PASS: TestCoordinatorRegistry_GetStats (0.00s) === RUN TestCoordinatorRegistry_HeartbeatGateway --- PASS: TestCoordinatorRegistry_HeartbeatGateway (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/gateway 0.081s === RUN TestNeedsRestart === RUN TestNeedsRestart/Stream_is_nil_-_needs_restart === RUN TestNeedsRestart/Offset_in_cache_-_no_restart_needed === RUN TestNeedsRestart/Offset_before_current_-_needs_restart === RUN TestNeedsRestart/Large_gap_ahead_-_needs_restart === RUN TestNeedsRestart/Small_gap_ahead_-_no_restart_needed === RUN TestNeedsRestart/Exact_match_-_no_restart_needed === RUN TestNeedsRestart/Context_is_nil_-_needs_restart --- PASS: TestNeedsRestart (0.00s) --- PASS: TestNeedsRestart/Stream_is_nil_-_needs_restart (0.00s) --- PASS: TestNeedsRestart/Offset_in_cache_-_no_restart_needed (0.00s) --- PASS: TestNeedsRestart/Offset_before_current_-_needs_restart (0.00s) --- PASS: TestNeedsRestart/Large_gap_ahead_-_needs_restart (0.00s) --- PASS: TestNeedsRestart/Small_gap_ahead_-_no_restart_needed (0.00s) --- PASS: TestNeedsRestart/Exact_match_-_no_restart_needed (0.00s) --- PASS: TestNeedsRestart/Context_is_nil_-_needs_restart (0.00s) === RUN TestNeedsRestart_CacheLogic === RUN TestNeedsRestart_CacheLogic/First_offset_in_cache === RUN TestNeedsRestart_CacheLogic/Middle_offset_in_cache === RUN TestNeedsRestart_CacheLogic/Last_offset_in_cache === RUN TestNeedsRestart_CacheLogic/Before_cache_start === RUN TestNeedsRestart_CacheLogic/Current_position === RUN TestNeedsRestart_CacheLogic/One_ahead === RUN TestNeedsRestart_CacheLogic/Large_gap_>_1000 --- PASS: TestNeedsRestart_CacheLogic (0.00s) --- PASS: TestNeedsRestart_CacheLogic/First_offset_in_cache (0.00s) --- PASS: TestNeedsRestart_CacheLogic/Middle_offset_in_cache (0.00s) --- PASS: TestNeedsRestart_CacheLogic/Last_offset_in_cache (0.00s) --- PASS: TestNeedsRestart_CacheLogic/Before_cache_start (0.00s) --- PASS: TestNeedsRestart_CacheLogic/Current_position (0.00s) --- PASS: TestNeedsRestart_CacheLogic/One_ahead (0.00s) --- PASS: TestNeedsRestart_CacheLogic/Large_gap_>_1000 (0.00s) === RUN TestNeedsRestart_EmptyCache === RUN TestNeedsRestart_EmptyCache/Before_current === RUN TestNeedsRestart_EmptyCache/At_current === RUN TestNeedsRestart_EmptyCache/Small_gap_ahead === RUN TestNeedsRestart_EmptyCache/Large_gap_ahead --- PASS: TestNeedsRestart_EmptyCache (0.00s) --- PASS: TestNeedsRestart_EmptyCache/Before_current (0.00s) --- PASS: TestNeedsRestart_EmptyCache/At_current (0.00s) --- PASS: TestNeedsRestart_EmptyCache/Small_gap_ahead (0.00s) --- PASS: TestNeedsRestart_EmptyCache/Large_gap_ahead (0.00s) === RUN TestNeedsRestart_ThreadSafety --- PASS: TestNeedsRestart_ThreadSafety (0.00s) === RUN TestRestartSubscriber_StateManagement --- PASS: TestRestartSubscriber_StateManagement (0.00s) === RUN TestMapBrokerErrorToKafka === RUN TestMapBrokerErrorToKafka/No_error === RUN TestMapBrokerErrorToKafka/Unknown_server_error === RUN TestMapBrokerErrorToKafka/Topic_not_found === RUN TestMapBrokerErrorToKafka/Partition_not_found === RUN TestMapBrokerErrorToKafka/Not_leader_or_follower === RUN TestMapBrokerErrorToKafka/Request_timed_out === RUN TestMapBrokerErrorToKafka/Broker_not_available === RUN TestMapBrokerErrorToKafka/Message_too_large === RUN TestMapBrokerErrorToKafka/Network_exception === RUN TestMapBrokerErrorToKafka/Offset_load_in_progress === RUN TestMapBrokerErrorToKafka/Invalid_record === RUN TestMapBrokerErrorToKafka/Topic_already_exists === RUN TestMapBrokerErrorToKafka/Invalid_partitions === RUN TestMapBrokerErrorToKafka/Invalid_config === RUN TestMapBrokerErrorToKafka/Publisher_not_found === RUN TestMapBrokerErrorToKafka/Connection_failed === RUN TestMapBrokerErrorToKafka/Follower_connection_failed === RUN TestMapBrokerErrorToKafka/Unknown_error_code --- PASS: TestMapBrokerErrorToKafka (0.00s) --- PASS: TestMapBrokerErrorToKafka/No_error (0.00s) --- PASS: TestMapBrokerErrorToKafka/Unknown_server_error (0.00s) --- PASS: TestMapBrokerErrorToKafka/Topic_not_found (0.00s) --- PASS: TestMapBrokerErrorToKafka/Partition_not_found (0.00s) --- PASS: TestMapBrokerErrorToKafka/Not_leader_or_follower (0.00s) --- PASS: TestMapBrokerErrorToKafka/Request_timed_out (0.00s) --- PASS: TestMapBrokerErrorToKafka/Broker_not_available (0.00s) --- PASS: TestMapBrokerErrorToKafka/Message_too_large (0.00s) --- PASS: TestMapBrokerErrorToKafka/Network_exception (0.00s) --- PASS: TestMapBrokerErrorToKafka/Offset_load_in_progress (0.00s) --- PASS: TestMapBrokerErrorToKafka/Invalid_record (0.00s) --- PASS: TestMapBrokerErrorToKafka/Topic_already_exists (0.00s) --- PASS: TestMapBrokerErrorToKafka/Invalid_partitions (0.00s) --- PASS: TestMapBrokerErrorToKafka/Invalid_config (0.00s) --- PASS: TestMapBrokerErrorToKafka/Publisher_not_found (0.00s) --- PASS: TestMapBrokerErrorToKafka/Connection_failed (0.00s) --- PASS: TestMapBrokerErrorToKafka/Follower_connection_failed (0.00s) --- PASS: TestMapBrokerErrorToKafka/Unknown_error_code (0.00s) === RUN TestHandleBrokerResponse === RUN TestHandleBrokerResponse/No_error === RUN TestHandleBrokerResponse/Structured_error_-_Not_leader === RUN TestHandleBrokerResponse/Structured_error_-_Topic_not_found === RUN TestHandleBrokerResponse/Fallback_string_parsing_-_Not_leader === RUN TestHandleBrokerResponse/Fallback_string_parsing_-_Topic_not_found === RUN TestHandleBrokerResponse/Fallback_string_parsing_-_Unknown_error --- PASS: TestHandleBrokerResponse (0.00s) --- PASS: TestHandleBrokerResponse/No_error (0.00s) --- PASS: TestHandleBrokerResponse/Structured_error_-_Not_leader (0.00s) --- PASS: TestHandleBrokerResponse/Structured_error_-_Topic_not_found (0.00s) --- PASS: TestHandleBrokerResponse/Fallback_string_parsing_-_Not_leader (0.00s) --- PASS: TestHandleBrokerResponse/Fallback_string_parsing_-_Topic_not_found (0.00s) --- PASS: TestHandleBrokerResponse/Fallback_string_parsing_-_Unknown_error (0.00s) === RUN TestParseStringErrorToKafkaCode === RUN TestParseStringErrorToKafkaCode/Empty_error === RUN TestParseStringErrorToKafkaCode/Not_leader_error === RUN TestParseStringErrorToKafkaCode/Not_leader_error_variant === RUN TestParseStringErrorToKafkaCode/Topic_not_found === RUN TestParseStringErrorToKafkaCode/Topic_does_not_exist === RUN TestParseStringErrorToKafkaCode/Partition_not_found === RUN TestParseStringErrorToKafkaCode/Timeout_error === RUN TestParseStringErrorToKafkaCode/Timeout_error_variant === RUN TestParseStringErrorToKafkaCode/Network_error === RUN TestParseStringErrorToKafkaCode/Connection_error === RUN TestParseStringErrorToKafkaCode/Message_too_large === RUN TestParseStringErrorToKafkaCode/Size_error === RUN TestParseStringErrorToKafkaCode/Unknown_error --- PASS: TestParseStringErrorToKafkaCode (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Empty_error (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Not_leader_error (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Not_leader_error_variant (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Topic_not_found (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Topic_does_not_exist (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Partition_not_found (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Timeout_error (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Timeout_error_variant (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Network_error (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Connection_error (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Message_too_large (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Size_error (0.00s) --- PASS: TestParseStringErrorToKafkaCode/Unknown_error (0.00s) === RUN TestAdaptiveFetchTimeout fetch_performance_test.go:11: Testing adaptive fetch timeout strategy... === RUN TestAdaptiveFetchTimeout/OldStrategy_50ms_Timeout fetch_performance_test.go:32: Record 1 timed out (readTime=150ms > timeout=50ms) fetch_performance_test.go:38: Old strategy: received 0/4 records in 20.349µs fetch_performance_test.go:43: ✓ Bug reproduced: old strategy times out too quickly === RUN TestAdaptiveFetchTimeout/NewStrategy_1s_Timeout fetch_performance_test.go:56: Record 1 received (readTime=150ms) fetch_performance_test.go:56: Record 2 received (readTime=150ms) fetch_performance_test.go:56: Record 3 received (readTime=150ms) fetch_performance_test.go:56: Record 4 received (readTime=150ms) fetch_performance_test.go:64: New strategy: received 4/4 records in 17.734µs fetch_performance_test.go:69: ✓ Fix verified: new strategy receives all records === RUN TestAdaptiveFetchTimeout/SchemaRegistry_CatchUp_Scenario fetch_performance_test.go:99: Schema Registry catch-up simulation: fetch_performance_test.go:100: Old strategy: 4 round trips, ~800ms total time fetch_performance_test.go:101: New strategy: 1 round trip, ~600ms total time fetch_performance_test.go:102: Schema Registry timeout: 500ms fetch_performance_test.go:108: ✓ Old strategy exceeds timeout: 800ms > 500ms fetch_performance_test.go:112: ✓ New strategy completes within timeout: 600ms <= 700ms --- PASS: TestAdaptiveFetchTimeout (0.00s) --- PASS: TestAdaptiveFetchTimeout/OldStrategy_50ms_Timeout (0.00s) --- PASS: TestAdaptiveFetchTimeout/NewStrategy_1s_Timeout (0.00s) --- PASS: TestAdaptiveFetchTimeout/SchemaRegistry_CatchUp_Scenario (0.00s) === RUN TestFetchTimeoutProgression fetch_performance_test.go:121: Testing fetch timeout progression... fetch_performance_test.go:134: Timeout progression: fetch_performance_test.go:137: Record 1: timeout = 1s fetch_performance_test.go:137: Record 2: timeout = 1s fetch_performance_test.go:137: Record 3: timeout = 1s fetch_performance_test.go:137: Record 4: timeout = 1s fetch_performance_test.go:137: Record 5: timeout = 1s fetch_performance_test.go:137: Record 6: timeout = 100ms fetch_performance_test.go:137: Record 7: timeout = 100ms fetch_performance_test.go:137: Record 8: timeout = 100ms fetch_performance_test.go:137: Record 9: timeout = 100ms fetch_performance_test.go:137: Record 10: timeout = 100ms fetch_performance_test.go:154: ✓ Timeout progression is correct --- PASS: TestFetchTimeoutProgression (0.00s) === RUN TestSeaweedSMQRecord_Interface --- PASS: TestSeaweedSMQRecord_Interface (0.00s) === RUN TestSeaweedMQHandler_GetStoredRecords_EmptyTopic record_retrieval_test.go:96: Test obsolete: ledgers removed, SMQ broker handles offset management --- SKIP: TestSeaweedMQHandler_GetStoredRecords_EmptyTopic (0.00s) === RUN TestSeaweedMQHandler_GetStoredRecords_EmptyPartition record_retrieval_test.go:102: Test obsolete: ledgers removed, SMQ broker handles offset management --- SKIP: TestSeaweedMQHandler_GetStoredRecords_EmptyPartition (0.00s) === RUN TestSeaweedMQHandler_GetStoredRecords_OffsetBeyondHighWaterMark record_retrieval_test.go:108: Test obsolete: ledgers removed, SMQ broker handles offset management --- SKIP: TestSeaweedMQHandler_GetStoredRecords_OffsetBeyondHighWaterMark (0.00s) === RUN TestSeaweedMQHandler_GetStoredRecords_MaxRecordsLimit record_retrieval_test.go:114: Test obsolete: ledgers removed, SMQ broker handles offset management --- SKIP: TestSeaweedMQHandler_GetStoredRecords_MaxRecordsLimit (0.00s) === RUN TestSeaweedMQHandler_MapSeaweedToKafkaOffsets seaweedmq_handler_test.go:15: Test obsolete: ledger system removed, SMQ uses native offsets --- SKIP: TestSeaweedMQHandler_MapSeaweedToKafkaOffsets (0.00s) === RUN TestSeaweedMQHandler_MapSeaweedToKafkaOffsets_EmptyRecords seaweedmq_handler_test.go:21: Test obsolete: ledger system removed, SMQ uses native offsets --- SKIP: TestSeaweedMQHandler_MapSeaweedToKafkaOffsets_EmptyRecords (0.00s) === RUN TestSeaweedMQHandler_ConvertSeaweedToKafkaRecordBatch seaweedmq_handler_test.go:68: Successfully converted 2 records to 119 byte batch --- PASS: TestSeaweedMQHandler_ConvertSeaweedToKafkaRecordBatch (0.00s) === RUN TestSeaweedMQHandler_ConvertSeaweedToKafkaRecordBatch_EmptyRecords --- PASS: TestSeaweedMQHandler_ConvertSeaweedToKafkaRecordBatch_EmptyRecords (0.00s) === RUN TestSeaweedMQHandler_ConvertSingleSeaweedRecord === RUN TestSeaweedMQHandler_ConvertSingleSeaweedRecord/Record_with_key_and_value seaweedmq_handler_test.go:161: Successfully converted single record: 24 bytes === RUN TestSeaweedMQHandler_ConvertSingleSeaweedRecord/Record_with_null_key seaweedmq_handler_test.go:161: Successfully converted single record: 23 bytes === RUN TestSeaweedMQHandler_ConvertSingleSeaweedRecord/Record_with_empty_value seaweedmq_handler_test.go:161: Successfully converted single record: 26 bytes --- PASS: TestSeaweedMQHandler_ConvertSingleSeaweedRecord (0.00s) --- PASS: TestSeaweedMQHandler_ConvertSingleSeaweedRecord/Record_with_key_and_value (0.00s) --- PASS: TestSeaweedMQHandler_ConvertSingleSeaweedRecord/Record_with_null_key (0.00s) --- PASS: TestSeaweedMQHandler_ConvertSingleSeaweedRecord/Record_with_empty_value (0.00s) === RUN TestSeaweedMQHandler_Creation seaweedmq_handler_test.go:171: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_Creation (0.00s) === RUN TestSeaweedMQHandler_TopicLifecycle seaweedmq_handler_test.go:190: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_TopicLifecycle (0.00s) === RUN TestSeaweedMQHandler_ProduceRecord seaweedmq_handler_test.go:252: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_ProduceRecord (0.00s) === RUN TestSeaweedMQHandler_MultiplePartitions seaweedmq_handler_test.go:297: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_MultiplePartitions (0.00s) === RUN TestSeaweedMQHandler_FetchRecords seaweedmq_handler_test.go:341: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_FetchRecords (0.00s) === RUN TestSeaweedMQHandler_FetchRecords_ErrorHandling seaweedmq_handler_test.go:435: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_FetchRecords_ErrorHandling (0.00s) === RUN TestSeaweedMQHandler_ErrorHandling seaweedmq_handler_test.go:485: Integration test requires real SeaweedMQ Broker - run manually with broker available --- SKIP: TestSeaweedMQHandler_ErrorHandling (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/integration 0.052s === RUN TestBatchConstruction batch_crc_compat_test.go:24: Batch size: 86 bytes batch_crc_compat_test.go:25: Batch hex: 0000: 00000000000000000000004affffffff 0016: 02d72e7dc20000000000000000019f81 0032: 6f79730000019f816f7973ffffffffff 0048: ffffffffffffffffff00000001300000 0064: 0010746573742d6b657914746573742d 0080: 76616c756500 batch_crc_compat_test.go:33: Stored CRC: 0xd72e7dc2 batch_crc_compat_test.go:38: Calculated CRC: 0xd72e7dc2 (over 65 bytes) batch_crc_compat_test.go:57: CRC verification PASSED batch_crc_compat_test.go:61: === Batch Structure === batch_crc_compat_test.go:238: Base Offset: 0000000000000000 (value: 0) batch_crc_compat_test.go:238: Batch Length: 0000004a (value: 74) batch_crc_compat_test.go:238: Leader Epoch: ffffffff (value: -1) batch_crc_compat_test.go:238: Magic: 02 (value: 2) batch_crc_compat_test.go:238: CRC: d72e7dc2 (value: 3610148290) batch_crc_compat_test.go:238: Attributes: 0000 (value: 0) batch_crc_compat_test.go:238: Last Offset Delta: 00000000 (value: 0) batch_crc_compat_test.go:238: Base Timestamp: 0000019f816f7973 (value: 1784582994291) batch_crc_compat_test.go:238: Max Timestamp: 0000019f816f7973 (value: 1784582994291) batch_crc_compat_test.go:238: Record Count: 00000001 (value: 1) batch_crc_compat_test.go:79: Batch length correct: 74 --- PASS: TestBatchConstruction (0.00s) === RUN TestMultipleRecordsBatch batch_crc_compat_test.go:93: Batch 1 size: 78, CRC: 0xfe5d690c batch_crc_compat_test.go:94: Batch 2 size: 78, CRC: 0xd315cdbc batch_crc_compat_test.go:104: Batch 1 CRC valid batch_crc_compat_test.go:104: Batch 2 CRC valid --- PASS: TestMultipleRecordsBatch (0.00s) === RUN TestVarintEncoding batch_crc_compat_test.go:131: encodeVarint(0) = 00 batch_crc_compat_test.go:131: encodeVarint(1) = 02 batch_crc_compat_test.go:131: encodeVarint(-1) = 01 batch_crc_compat_test.go:131: encodeVarint(5) = 0a batch_crc_compat_test.go:131: encodeVarint(-5) = 09 batch_crc_compat_test.go:131: encodeVarint(127) = fe01 batch_crc_compat_test.go:131: encodeVarint(128) = 8002 batch_crc_compat_test.go:131: encodeVarint(-127) = fd01 batch_crc_compat_test.go:131: encodeVarint(-128) = ff01 --- PASS: TestVarintEncoding (0.00s) === RUN TestClientSideCRCValidation batch_crc_compat_test.go:259: Constructed batch: 86 bytes batch_crc_compat_test.go:268: Client read CRC from header: 0xc9e5a485 batch_crc_compat_test.go:272: Client calculated CRC: 0xc9e5a485 batch_crc_compat_test.go:280: CLIENT WOULD ACCEPT: CRC valid --- PASS: TestClientSideCRCValidation (0.00s) === RUN TestConcurrentBatchConstruction batch_crc_compat_test.go:316: All 10 concurrent batches have valid CRCs --- PASS: TestConcurrentBatchConstruction (0.00s) === RUN TestProductionBatchConstruction batch_crc_compat_test.go:338: Production batch size: 86 bytes batch_crc_compat_test.go:348: Production batch CRC: stored=0xece17711 calculated=0xece17711 batch_crc_compat_test.go:354: PRODUCTION CODE CRC VALID --- PASS: TestProductionBatchConstruction (0.00s) === RUN TestHeartbeatResponseFormat_V0 --- PASS: TestHeartbeatResponseFormat_V0 (0.00s) === RUN TestHeartbeatResponseFormat_V1ToV3 === RUN TestHeartbeatResponseFormat_V1ToV3/v1 === RUN TestHeartbeatResponseFormat_V1ToV3/v2 === RUN TestHeartbeatResponseFormat_V1ToV3/v3 --- PASS: TestHeartbeatResponseFormat_V1ToV3 (0.00s) --- PASS: TestHeartbeatResponseFormat_V1ToV3/v1 (0.00s) --- PASS: TestHeartbeatResponseFormat_V1ToV3/v2 (0.00s) --- PASS: TestHeartbeatResponseFormat_V1ToV3/v3 (0.00s) === RUN TestHeartbeatResponseFormat_V4Plus === RUN TestHeartbeatResponseFormat_V4Plus/v4 --- PASS: TestHeartbeatResponseFormat_V4Plus (0.00s) --- PASS: TestHeartbeatResponseFormat_V4Plus/v4 (0.00s) === RUN TestHeartbeatResponseFormat_ErrorCode === RUN TestHeartbeatResponseFormat_ErrorCode/None === RUN TestHeartbeatResponseFormat_ErrorCode/UnknownMemberID === RUN TestHeartbeatResponseFormat_ErrorCode/IllegalGeneration === RUN TestHeartbeatResponseFormat_ErrorCode/RebalanceInProgress --- PASS: TestHeartbeatResponseFormat_ErrorCode (0.00s) --- PASS: TestHeartbeatResponseFormat_ErrorCode/None (0.00s) --- PASS: TestHeartbeatResponseFormat_ErrorCode/UnknownMemberID (0.00s) --- PASS: TestHeartbeatResponseFormat_ErrorCode/IllegalGeneration (0.00s) --- PASS: TestHeartbeatResponseFormat_ErrorCode/RebalanceInProgress (0.00s) === RUN TestHeartbeatResponseFormat_BugReproduce heartbeat_response_format_test.go:157: This test documents the original bug - skip to avoid false failures --- SKIP: TestHeartbeatResponseFormat_BugReproduce (0.00s) === RUN TestMetadataRequestBlocking metadata_blocking_test.go:22: This test documents the original bug. The fix is in the broker's ListTopics with filer timeout. Run TestMetadataRequestWithFastMock to verify fast path works. --- SKIP: TestMetadataRequestBlocking (0.00s) === RUN TestMetadataRequestWithFastMock metadata_blocking_test.go:68: Testing Metadata handler with fast-responding backend... metadata_blocking_test.go:88: ✓ Metadata completed in 5.601µs (283 bytes) --- PASS: TestMetadataRequestWithFastMock (0.00s) === RUN TestMetadataRequestWithTimeoutFix metadata_blocking_test.go:98: Testing Metadata handler with timeout-aware backend... metadata_blocking_test.go:115: Metadata completed in 2.008643852s metadata_blocking_test.go:121: ✓ Metadata returned response (33 bytes) without blocking --- PASS: TestMetadataRequestWithTimeoutFix (2.01s) === RUN TestOffsetCommitFetchPattern offset_fetch_pattern_test.go:18: Integration test - requires mock broker setup --- SKIP: TestOffsetCommitFetchPattern (0.00s) === RUN TestOffsetFetchAfterCommit offset_fetch_pattern_test.go:98: Integration test - requires mock broker setup --- SKIP: TestOffsetFetchAfterCommit (0.00s) === RUN TestOffsetPersistencePattern offset_fetch_pattern_test.go:126: Integration test - requires mock broker setup --- SKIP: TestOffsetPersistencePattern (0.00s) === RUN TestOffsetCommitConsistency offset_fetch_pattern_test.go:167: Integration test - requires mock broker setup --- SKIP: TestOffsetCommitConsistency (0.00s) === RUN TestLongPollWithOffsetCommit offset_fetch_pattern_test.go:200: Integration test - requires mock broker setup --- SKIP: TestLongPollWithOffsetCommit (0.00s) === RUN TestRecordBatchParser_ParseRecordBatch --- PASS: TestRecordBatchParser_ParseRecordBatch (0.00s) === RUN TestRecordBatchParser_ParseRecordBatch_TooSmall --- PASS: TestRecordBatchParser_ParseRecordBatch_TooSmall (0.00s) === RUN TestRecordBatchParser_ParseRecordBatch_InvalidMagic --- PASS: TestRecordBatchParser_ParseRecordBatch_InvalidMagic (0.00s) === RUN TestRecordBatchParser_Compression === RUN TestRecordBatchParser_Compression/none === RUN TestRecordBatchParser_Compression/gzip === RUN TestRecordBatchParser_Compression/snappy === RUN TestRecordBatchParser_Compression/lz4 === RUN TestRecordBatchParser_Compression/zstd --- PASS: TestRecordBatchParser_Compression (0.02s) --- PASS: TestRecordBatchParser_Compression/none (0.00s) --- PASS: TestRecordBatchParser_Compression/gzip (0.00s) --- PASS: TestRecordBatchParser_Compression/snappy (0.00s) --- PASS: TestRecordBatchParser_Compression/lz4 (0.01s) --- PASS: TestRecordBatchParser_Compression/zstd (0.01s) === RUN TestRecordBatchParser_CRCValidation === RUN TestRecordBatchParser_CRCValidation/Valid_CRC === RUN TestRecordBatchParser_CRCValidation/Invalid_CRC === RUN TestRecordBatchParser_CRCValidation/Skip_CRC_validation --- PASS: TestRecordBatchParser_CRCValidation (0.00s) --- PASS: TestRecordBatchParser_CRCValidation/Valid_CRC (0.00s) --- PASS: TestRecordBatchParser_CRCValidation/Invalid_CRC (0.00s) --- PASS: TestRecordBatchParser_CRCValidation/Skip_CRC_validation (0.00s) === RUN TestRecordBatchParser_ExtractRecords --- PASS: TestRecordBatchParser_ExtractRecords (0.00s) === RUN TestCompressRecordBatch === RUN TestCompressRecordBatch/No_compression === RUN TestCompressRecordBatch/Gzip_compression --- PASS: TestCompressRecordBatch (0.00s) --- PASS: TestCompressRecordBatch/No_compression (0.00s) --- PASS: TestCompressRecordBatch/Gzip_compression (0.00s) === RUN TestCreateRecordBatch === RUN TestCreateRecordBatch/Uncompressed_batch === RUN TestCreateRecordBatch/Compressed_batch --- PASS: TestCreateRecordBatch (0.00s) --- PASS: TestCreateRecordBatch/Uncompressed_batch (0.00s) --- PASS: TestCreateRecordBatch/Compressed_batch (0.00s) === RUN TestRecordBatchParser_InvalidRecordCount --- PASS: TestRecordBatchParser_InvalidRecordCount (0.00s) === RUN TestExtractAllRecords_RealKafkaFormat record_extraction_test.go:125: Created batch of 85 bytes, record value: {"type":"string"} record_extraction_test.go:150: Successfully extracted record with value: {"type":"string"} --- PASS: TestExtractAllRecords_RealKafkaFormat (0.00s) === RUN TestExtractAllRecords_CompressedBatch record_extraction_test.go:157: Compressed batch test - implement after uncompressed works --- SKIP: TestExtractAllRecords_CompressedBatch (0.00s) === RUN TestResponseFormatsNoCorrelationID === RUN TestResponseFormatsNoCorrelationID/ApiVersions_v0 response_format_test.go:153: Testing ApiVersions_v0: ApiVersions v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 18 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/ApiVersions_v4 response_format_test.go:153: Testing ApiVersions_v4: ApiVersions v4 (flexible) should not include correlation ID in body response_format_test.go:160: ✓ API Key 18 Version 4: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Metadata_v0 response_format_test.go:153: Testing Metadata_v0: Metadata v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 3 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Metadata_v7 response_format_test.go:153: Testing Metadata_v7: Metadata v7 should not include correlation ID in body response_format_test.go:160: ✓ API Key 3 Version 7: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/FindCoordinator_v0 response_format_test.go:153: Testing FindCoordinator_v0: FindCoordinator v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 10 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/FindCoordinator_v2 response_format_test.go:153: Testing FindCoordinator_v2: FindCoordinator v2 should not include correlation ID in body response_format_test.go:160: ✓ API Key 10 Version 2: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/DescribeConfigs_v0 response_format_test.go:153: Testing DescribeConfigs_v0: DescribeConfigs v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 32 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/DescribeConfigs_v4 response_format_test.go:153: Testing DescribeConfigs_v4: DescribeConfigs v4 (flexible) should not include correlation ID in body response_format_test.go:160: ✓ API Key 32 Version 4: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/DescribeCluster_v0 response_format_test.go:153: Testing DescribeCluster_v0: DescribeCluster v0 (flexible) should not include correlation ID in body response_format_test.go:160: ✓ API Key 60 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/InitProducerId_v0 response_format_test.go:153: Testing InitProducerId_v0: InitProducerId v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 22 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/InitProducerId_v4 response_format_test.go:153: Testing InitProducerId_v4: InitProducerId v4 (flexible) should not include correlation ID in body response_format_test.go:160: ✓ API Key 22 Version 4: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/JoinGroup_v0 response_format_test.go:153: Testing JoinGroup_v0: JoinGroup v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 11 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/SyncGroup_v0 response_format_test.go:153: Testing SyncGroup_v0: SyncGroup v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 14 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Heartbeat_v0 response_format_test.go:153: Testing Heartbeat_v0: Heartbeat v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 12 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/LeaveGroup_v0 response_format_test.go:153: Testing LeaveGroup_v0: LeaveGroup v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 13 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/OffsetFetch_v0 response_format_test.go:153: Testing OffsetFetch_v0: OffsetFetch v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 9 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/OffsetCommit_v0 response_format_test.go:153: Testing OffsetCommit_v0: OffsetCommit v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 8 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Produce_v0 response_format_test.go:153: Testing Produce_v0: Produce v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 0 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Produce_v7 response_format_test.go:153: Testing Produce_v7: Produce v7 should not include correlation ID in body response_format_test.go:160: ✓ API Key 0 Version 7: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Fetch_v0 response_format_test.go:153: Testing Fetch_v0: Fetch v0 should not include correlation ID in body response_format_test.go:160: ✓ API Key 1 Version 0: Correlation ID should be handled by writeResponseWithHeader === RUN TestResponseFormatsNoCorrelationID/Fetch_v7 response_format_test.go:153: Testing Fetch_v7: Fetch v7 should not include correlation ID in body response_format_test.go:160: ✓ API Key 1 Version 7: Correlation ID should be handled by writeResponseWithHeader --- PASS: TestResponseFormatsNoCorrelationID (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/ApiVersions_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/ApiVersions_v4 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Metadata_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Metadata_v7 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/FindCoordinator_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/FindCoordinator_v2 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/DescribeConfigs_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/DescribeConfigs_v4 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/DescribeCluster_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/InitProducerId_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/InitProducerId_v4 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/JoinGroup_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/SyncGroup_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Heartbeat_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/LeaveGroup_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/OffsetFetch_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/OffsetCommit_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Produce_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Produce_v7 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Fetch_v0 (0.00s) --- PASS: TestResponseFormatsNoCorrelationID/Fetch_v7 (0.00s) === RUN TestFlexibleResponseHeaderFormat === RUN TestFlexibleResponseHeaderFormat/ApiVersions_v0 response_format_test.go:252: ✓ ApiVersions_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/ApiVersions_v3 response_format_test.go:252: ✓ ApiVersions_v3: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/ApiVersions_v4 response_format_test.go:252: ✓ ApiVersions_v4: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Metadata_v0 response_format_test.go:252: ✓ Metadata_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Metadata_v7 response_format_test.go:252: ✓ Metadata_v7: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Metadata_v9 response_format_test.go:252: ✓ Metadata_v9: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/Produce_v0 response_format_test.go:252: ✓ Produce_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Produce_v7 response_format_test.go:252: ✓ Produce_v7: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Produce_v9 response_format_test.go:252: ✓ Produce_v9: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/Fetch_v0 response_format_test.go:252: ✓ Fetch_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Fetch_v7 response_format_test.go:252: ✓ Fetch_v7: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Fetch_v12 response_format_test.go:252: ✓ Fetch_v12: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/FindCoordinator_v0 response_format_test.go:252: ✓ FindCoordinator_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/FindCoordinator_v2 response_format_test.go:252: ✓ FindCoordinator_v2: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/FindCoordinator_v3 response_format_test.go:252: ✓ FindCoordinator_v3: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/JoinGroup_v0 response_format_test.go:252: ✓ JoinGroup_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/JoinGroup_v5 response_format_test.go:252: ✓ JoinGroup_v5: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/JoinGroup_v6 response_format_test.go:252: ✓ JoinGroup_v6: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/SyncGroup_v0 response_format_test.go:252: ✓ SyncGroup_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/SyncGroup_v3 response_format_test.go:252: ✓ SyncGroup_v3: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/SyncGroup_v4 response_format_test.go:252: ✓ SyncGroup_v4: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/Heartbeat_v0 response_format_test.go:252: ✓ Heartbeat_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Heartbeat_v3 response_format_test.go:252: ✓ Heartbeat_v3: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/Heartbeat_v4 response_format_test.go:252: ✓ Heartbeat_v4: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/LeaveGroup_v0 response_format_test.go:252: ✓ LeaveGroup_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/LeaveGroup_v3 response_format_test.go:252: ✓ LeaveGroup_v3: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/LeaveGroup_v4 response_format_test.go:252: ✓ LeaveGroup_v4: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/OffsetFetch_v0 response_format_test.go:252: ✓ OffsetFetch_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/OffsetFetch_v5 response_format_test.go:252: ✓ OffsetFetch_v5: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/OffsetFetch_v6 response_format_test.go:252: ✓ OffsetFetch_v6: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/OffsetCommit_v0 response_format_test.go:252: ✓ OffsetCommit_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/OffsetCommit_v7 response_format_test.go:252: ✓ OffsetCommit_v7: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/OffsetCommit_v8 response_format_test.go:252: ✓ OffsetCommit_v8: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/DescribeConfigs_v0 response_format_test.go:252: ✓ DescribeConfigs_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/DescribeConfigs_v3 response_format_test.go:252: ✓ DescribeConfigs_v3: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/DescribeConfigs_v4 response_format_test.go:252: ✓ DescribeConfigs_v4: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/InitProducerId_v0 response_format_test.go:252: ✓ InitProducerId_v0: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/InitProducerId_v1 response_format_test.go:252: ✓ InitProducerId_v1: correctly identified as flexible=false === RUN TestFlexibleResponseHeaderFormat/InitProducerId_v2 response_format_test.go:252: ✓ InitProducerId_v2: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/DescribeCluster_v0 response_format_test.go:252: ✓ DescribeCluster_v0: correctly identified as flexible=true === RUN TestFlexibleResponseHeaderFormat/DescribeCluster_v1 response_format_test.go:252: ✓ DescribeCluster_v1: correctly identified as flexible=true --- PASS: TestFlexibleResponseHeaderFormat (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/ApiVersions_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/ApiVersions_v3 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/ApiVersions_v4 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Metadata_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Metadata_v7 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Metadata_v9 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Produce_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Produce_v7 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Produce_v9 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Fetch_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Fetch_v7 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Fetch_v12 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/FindCoordinator_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/FindCoordinator_v2 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/FindCoordinator_v3 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/JoinGroup_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/JoinGroup_v5 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/JoinGroup_v6 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/SyncGroup_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/SyncGroup_v3 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/SyncGroup_v4 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Heartbeat_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Heartbeat_v3 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/Heartbeat_v4 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/LeaveGroup_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/LeaveGroup_v3 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/LeaveGroup_v4 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/OffsetFetch_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/OffsetFetch_v5 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/OffsetFetch_v6 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/OffsetCommit_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/OffsetCommit_v7 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/OffsetCommit_v8 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/DescribeConfigs_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/DescribeConfigs_v3 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/DescribeConfigs_v4 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/InitProducerId_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/InitProducerId_v1 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/InitProducerId_v2 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/DescribeCluster_v0 (0.00s) --- PASS: TestFlexibleResponseHeaderFormat/DescribeCluster_v1 (0.00s) === RUN TestCorrelationIDNotInResponseBody === RUN TestCorrelationIDNotInResponseBody/DetectCorrelationIDInBody response_format_test.go:283: ✓ Successfully detected correlation ID in body (bad response) response_format_test.go:294: ✓ Correctly identified response without correlation ID --- PASS: TestCorrelationIDNotInResponseBody (0.00s) --- PASS: TestCorrelationIDNotInResponseBody/DetectCorrelationIDInBody (0.00s) === RUN TestWireProtocolFormat response_format_test.go:301: Kafka Wire Protocol Format (KIP-482): response_format_test.go:302: Non-flexible responses: response_format_test.go:303: [Size: 4 bytes][Correlation ID: 4 bytes][Response Body] response_format_test.go:304: response_format_test.go:305: Flexible responses (header version 1+): response_format_test.go:306: [Size: 4 bytes][Correlation ID: 4 bytes][Tagged Fields: 1+ bytes][Response Body] response_format_test.go:307: response_format_test.go:308: Size field: includes correlation ID + tagged fields + body response_format_test.go:309: Tagged Fields: varint-encoded, 0x00 for empty response_format_test.go:310: response_format_test.go:311: CRITICAL: Response body should NEVER include correlation ID! response_format_test.go:312: It is written ONLY by writeResponseWithHeader --- PASS: TestWireProtocolFormat (0.00s) === RUN TestJoinGroupResponseStructure response_validation_example_test.go:14: This is a demonstration test - shows what we SHOULD check --- SKIP: TestJoinGroupResponseStructure (0.00s) === RUN TestProduceResponseStructure response_validation_example_test.go:48: This is a demonstration test - shows what we SHOULD check --- SKIP: TestProduceResponseStructure (0.00s) === RUN TestCompareWithReferenceImplementation response_validation_example_test.go:64: This would require a reference Kafka broker or client library --- SKIP: TestCompareWithReferenceImplementation (0.00s) === RUN TestCurrentTestingApproach response_validation_example_test.go:82: Current testing strategy (as of Oct 2025): response_validation_example_test.go:83: response_validation_example_test.go:84: LEVEL 1: Static Code Analysis response_validation_example_test.go:85: Tool: check_responses.sh response_validation_example_test.go:86: Checks: Correlation ID patterns response_validation_example_test.go:87: Coverage: Good for known issues response_validation_example_test.go:88: response_validation_example_test.go:89: LEVEL 2: Protocol Format Tests response_validation_example_test.go:90: Tool: TestFlexibleResponseHeaderFormat response_validation_example_test.go:91: Checks: Flexible vs non-flexible classification response_validation_example_test.go:92: Coverage: Header format only response_validation_example_test.go:93: response_validation_example_test.go:94: LEVEL 3: Integration Testing response_validation_example_test.go:95: Tool: Schema Registry, kafka-go, Sarama, Java client response_validation_example_test.go:96: Checks: Real client compatibility response_validation_example_test.go:97: Coverage: Complete but requires manual debugging response_validation_example_test.go:98: response_validation_example_test.go:99: MISSING: Field-level response body validation response_validation_example_test.go:100: This is why JoinGroup issue wasn't caught by unit tests --- PASS: TestCurrentTestingApproach (0.00s) === RUN TestMetadataResponseHasBrokers response_validation_example_test.go:105: Example of what a real field-level test would look like --- SKIP: TestMetadataResponseHasBrokers (0.00s) === RUN TestSyncGroup_RaceCondition_BugDocumentation syncgroup_assignment_test.go:52: Original bug: Non-leader in Stable state would trigger server-side assignment syncgroup_assignment_test.go:53: This caused duplicate partition assignments and message re-reads (66.7% duplicates) syncgroup_assignment_test.go:54: Fix: Check if member is non-leader with empty assignments, regardless of group state --- PASS: TestSyncGroup_RaceCondition_BugDocumentation (0.00s) === RUN TestSyncGroup_FixVerification === RUN TestSyncGroup_FixVerification/Leader_with_assignments syncgroup_assignment_test.go:125: ✓ Leader provides client-side assignments, processes them: isLeader=true hasAssignments=true shouldWait=false === RUN TestSyncGroup_FixVerification/Non-leader_without_assignments_(PreparingRebalance) syncgroup_assignment_test.go:125: ✓ Non-leader waits for leader to provide assignments: isLeader=false hasAssignments=false shouldWait=true === RUN TestSyncGroup_FixVerification/Non-leader_without_assignments_(Stable)_-_THE_BUG_CASE syncgroup_assignment_test.go:125: ✓ Non-leader retrieves assignment from leader (already processed): isLeader=false hasAssignments=false shouldWait=true === RUN TestSyncGroup_FixVerification/Leader_without_assignments syncgroup_assignment_test.go:125: ✓ Edge case: server-side assignment (should not happen with Sarama): isLeader=true hasAssignments=false shouldWait=false --- PASS: TestSyncGroup_FixVerification (0.00s) --- PASS: TestSyncGroup_FixVerification/Leader_with_assignments (0.00s) --- PASS: TestSyncGroup_FixVerification/Non-leader_without_assignments_(PreparingRebalance) (0.00s) --- PASS: TestSyncGroup_FixVerification/Non-leader_without_assignments_(Stable)_-_THE_BUG_CASE (0.00s) --- PASS: TestSyncGroup_FixVerification/Leader_without_assignments (0.00s) === RUN TestParseSyncGroupRequestV5SkipsProtocolFields --- PASS: TestParseSyncGroupRequestV5SkipsProtocolFields (0.00s) === RUN TestSyncGroupSchemaRegistryV5LeaderAssignmentDoesNotRebalance --- PASS: TestSyncGroupSchemaRegistryV5LeaderAssignmentDoesNotRebalance (0.00s) === RUN TestEnsureTopicExistsTreatsAlreadyExistsAsSuccess --- PASS: TestEnsureTopicExistsTreatsAlreadyExistsAsSuccess (0.00s) === RUN TestEnsureTopicExistsReportsGenuineFailure --- PASS: TestEnsureTopicExistsReportsGenuineFailure (0.00s) === RUN TestEnsureTopicExistsWhenPresent --- PASS: TestEnsureTopicExistsWhenPresent (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/protocol 2.072s === RUN TestNewAvroDecoder === RUN TestNewAvroDecoder/valid_record_schema === RUN TestNewAvroDecoder/valid_enum_schema === RUN TestNewAvroDecoder/invalid_schema === RUN TestNewAvroDecoder/empty_schema --- PASS: TestNewAvroDecoder (0.00s) --- PASS: TestNewAvroDecoder/valid_record_schema (0.00s) --- PASS: TestNewAvroDecoder/valid_enum_schema (0.00s) --- PASS: TestNewAvroDecoder/invalid_schema (0.00s) --- PASS: TestNewAvroDecoder/empty_schema (0.00s) === RUN TestAvroDecoder_Decode --- PASS: TestAvroDecoder_Decode (0.00s) === RUN TestAvroDecoder_DecodeToRecordValue --- PASS: TestAvroDecoder_DecodeToRecordValue (0.00s) === RUN TestMapToRecordValue --- PASS: TestMapToRecordValue (0.00s) === RUN TestGoValueToSchemaValue === RUN TestGoValueToSchemaValue/nil_value === RUN TestGoValueToSchemaValue/bool_value === RUN TestGoValueToSchemaValue/int32_value === RUN TestGoValueToSchemaValue/int64_value === RUN TestGoValueToSchemaValue/string_value === RUN TestGoValueToSchemaValue/bytes_value === RUN TestGoValueToSchemaValue/time_value --- PASS: TestGoValueToSchemaValue (0.00s) --- PASS: TestGoValueToSchemaValue/nil_value (0.00s) --- PASS: TestGoValueToSchemaValue/bool_value (0.00s) --- PASS: TestGoValueToSchemaValue/int32_value (0.00s) --- PASS: TestGoValueToSchemaValue/int64_value (0.00s) --- PASS: TestGoValueToSchemaValue/string_value (0.00s) --- PASS: TestGoValueToSchemaValue/bytes_value (0.00s) --- PASS: TestGoValueToSchemaValue/time_value (0.00s) === RUN TestInferRecordTypeFromMap --- PASS: TestInferRecordTypeFromMap (0.00s) === RUN TestInferTypeFromValue === RUN TestInferTypeFromValue/nil === RUN TestInferTypeFromValue/bool === RUN TestInferTypeFromValue/int32 === RUN TestInferTypeFromValue/int64 === RUN TestInferTypeFromValue/int === RUN TestInferTypeFromValue/float32 === RUN TestInferTypeFromValue/float64 === RUN TestInferTypeFromValue/string === RUN TestInferTypeFromValue/bytes === RUN TestInferTypeFromValue/time --- PASS: TestInferTypeFromValue (0.00s) --- PASS: TestInferTypeFromValue/nil (0.00s) --- PASS: TestInferTypeFromValue/bool (0.00s) --- PASS: TestInferTypeFromValue/int32 (0.00s) --- PASS: TestInferTypeFromValue/int64 (0.00s) --- PASS: TestInferTypeFromValue/int (0.00s) --- PASS: TestInferTypeFromValue/float32 (0.00s) --- PASS: TestInferTypeFromValue/float64 (0.00s) --- PASS: TestInferTypeFromValue/string (0.00s) --- PASS: TestInferTypeFromValue/bytes (0.00s) --- PASS: TestInferTypeFromValue/time (0.00s) === RUN TestAvroDecoder_Integration --- PASS: TestAvroDecoder_Integration (0.00s) === RUN TestBrokerClient_FetchIntegration === RUN TestBrokerClient_FetchIntegration/Fetch_Schema_Integration I0720 21:29:54.340345 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.343446 connect_to_sub_coordinator.go:100 subscriber kafka.fetch-test-topic/kafka-gateway waiting for more assignments broker_client_fetch_test.go:57: Fetch integration test completed - connection failed as expected with mock broker: failed to get subscriber for topic fetch-test-topic: failed to connect to brokers: connection timeout === RUN TestBrokerClient_FetchIntegration/Envelope_Reconstruction broker_client_fetch_test.go:88: Expected error in envelope reconstruction due to schema mismatch: failed to encode RecordValue: failed to encode to Avro binary: cannot encode binary record "FetchTest" field "id": schema does not specify default value and no value provided === RUN TestBrokerClient_FetchIntegration/Subscriber_Management I0720 21:29:54.442431 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.442466 connect_to_sub_coordinator.go:100 subscriber kafka.subscriber-test-topic/kafka-gateway waiting for more assignments broker_client_fetch_test.go:106: Subscriber creation failed as expected with mock brokers: failed to connect to brokers: connection timeout broker_client_fetch_test.go:118: Active subscribers: 0 --- PASS: TestBrokerClient_FetchIntegration (0.21s) --- PASS: TestBrokerClient_FetchIntegration/Fetch_Schema_Integration (0.10s) --- PASS: TestBrokerClient_FetchIntegration/Envelope_Reconstruction (0.00s) --- PASS: TestBrokerClient_FetchIntegration/Subscriber_Management (0.10s) === RUN TestBrokerClient_RoundTripIntegration === RUN TestBrokerClient_RoundTripIntegration/Complete_Schema_Workflow broker_client_fetch_test.go:182: Round-trip test completed - schema validation and processing successful === RUN TestBrokerClient_RoundTripIntegration/Error_Handling_in_Fetch I0720 21:29:54.545317 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.545347 connect_to_sub_coordinator.go:100 subscriber kafka.non-existent-topic/kafka-gateway waiting for more assignments broker_client_fetch_test.go:200: Reconstruction result: failed to encode RecordValue: failed to get schema for encoding: schema registry error 404: {"error_code": 40403, "message": "Schema not found"} --- PASS: TestBrokerClient_RoundTripIntegration (0.10s) --- PASS: TestBrokerClient_RoundTripIntegration/Complete_Schema_Workflow (0.00s) --- PASS: TestBrokerClient_RoundTripIntegration/Error_Handling_in_Fetch (0.10s) === RUN TestBrokerClient_SubscriberConfiguration === RUN TestBrokerClient_SubscriberConfiguration/Subscriber_Cache_Management I0720 21:29:54.648108 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.648118 connect_to_sub_coordinator.go:100 subscriber kafka.cache-test-topic/kafka-gateway waiting for more assignments I0720 21:29:54.750950 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.750962 connect_to_sub_coordinator.go:100 subscriber kafka.cache-test-topic/kafka-gateway waiting for more assignments broker_client_fetch_test.go:230: Subscriber creation results: err1=failed to connect to brokers: connection timeout, err2=failed to connect to brokers: connection timeout broker_client_fetch_test.go:235: Broker client remains functional after subscriber creation attempts === RUN TestBrokerClient_SubscriberConfiguration/Multiple_Topic_Subscribers I0720 21:29:54.854099 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.854107 connect_to_sub_coordinator.go:100 subscriber kafka.topic-a/kafka-gateway waiting for more assignments broker_client_fetch_test.go:244: Subscriber creation for topic-a: failed to connect to brokers: connection timeout I0720 21:29:54.957382 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:54.957392 connect_to_sub_coordinator.go:100 subscriber kafka.topic-b/kafka-gateway waiting for more assignments broker_client_fetch_test.go:244: Subscriber creation for topic-b: failed to connect to brokers: connection timeout I0720 21:29:55.060897 connect_to_sub_coordinator.go:33 broker coordinator on localhost:17777: getOrCreateConnection localhost:17777: fail to dial localhost:17777: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:29:55.060940 connect_to_sub_coordinator.go:100 subscriber kafka.topic-c/kafka-gateway waiting for more assignments broker_client_fetch_test.go:244: Subscriber creation for topic-c: failed to connect to brokers: connection timeout --- PASS: TestBrokerClient_SubscriberConfiguration (0.51s) --- PASS: TestBrokerClient_SubscriberConfiguration/Subscriber_Cache_Management (0.21s) --- PASS: TestBrokerClient_SubscriberConfiguration/Multiple_Topic_Subscribers (0.31s) === RUN TestBrokerClient_SchematizedMessage === RUN TestBrokerClient_SchematizedMessage/Avro_Schematized_Message broker_client_test.go:78: Known issue: Integer value decoded as 0 instead of 42 broker_client_test.go:87: Successfully validated schematized message with schema ID 1 === RUN TestBrokerClient_SchematizedMessage/RecordType_Creation === RUN TestBrokerClient_SchematizedMessage/Publisher_Stats --- PASS: TestBrokerClient_SchematizedMessage (0.00s) --- PASS: TestBrokerClient_SchematizedMessage/Avro_Schematized_Message (0.00s) --- PASS: TestBrokerClient_SchematizedMessage/RecordType_Creation (0.00s) --- PASS: TestBrokerClient_SchematizedMessage/Publisher_Stats (0.00s) === RUN TestBrokerClient_ErrorHandling === RUN TestBrokerClient_ErrorHandling/Invalid_Schematized_Message === RUN TestBrokerClient_ErrorHandling/Non-Schematized_Message === RUN TestBrokerClient_ErrorHandling/Unknown_Schema_ID === RUN TestBrokerClient_ErrorHandling/Invalid_RecordType_Creation --- PASS: TestBrokerClient_ErrorHandling (0.00s) --- PASS: TestBrokerClient_ErrorHandling/Invalid_Schematized_Message (0.00s) --- PASS: TestBrokerClient_ErrorHandling/Non-Schematized_Message (0.00s) --- PASS: TestBrokerClient_ErrorHandling/Unknown_Schema_ID (0.00s) --- PASS: TestBrokerClient_ErrorHandling/Invalid_RecordType_Creation (0.00s) === RUN TestBrokerClient_Integration === RUN TestBrokerClient_Integration/Multiple_Schema_Formats broker_client_test.go:251: Successfully validated JSON Schema message with schema ID 11 === RUN TestBrokerClient_Integration/Cache_Behavior --- PASS: TestBrokerClient_Integration (0.00s) --- PASS: TestBrokerClient_Integration/Multiple_Schema_Formats (0.00s) --- PASS: TestBrokerClient_Integration/Cache_Behavior (0.00s) === RUN TestBasicSchemaDecodeEncode === RUN TestBasicSchemaDecodeEncode/Simple_Avro_String_Record === RUN TestBasicSchemaDecodeEncode/JSON_Schema_with_String_Field === RUN TestBasicSchemaDecodeEncode/Cache_Performance --- PASS: TestBasicSchemaDecodeEncode (0.00s) --- PASS: TestBasicSchemaDecodeEncode/Simple_Avro_String_Record (0.00s) --- PASS: TestBasicSchemaDecodeEncode/JSON_Schema_with_String_Field (0.00s) --- PASS: TestBasicSchemaDecodeEncode/Cache_Performance (0.00s) === RUN TestSchemaValidation === RUN TestSchemaValidation/Valid_Schema_Message === RUN TestSchemaValidation/Non-Schematized_Message === RUN TestSchemaValidation/Invalid_Envelope --- PASS: TestSchemaValidation (0.01s) --- PASS: TestSchemaValidation/Valid_Schema_Message (0.00s) --- PASS: TestSchemaValidation/Non-Schematized_Message (0.00s) --- PASS: TestSchemaValidation/Invalid_Envelope (0.00s) === RUN TestSchemaDecodeEncode_Avro === RUN TestSchemaDecodeEncode_Avro/Simple_User_Record === RUN TestSchemaDecodeEncode_Avro/Complex_Record_with_Arrays === RUN TestSchemaDecodeEncode_Avro/Union_Types --- PASS: TestSchemaDecodeEncode_Avro (0.00s) --- PASS: TestSchemaDecodeEncode_Avro/Simple_User_Record (0.00s) --- PASS: TestSchemaDecodeEncode_Avro/Complex_Record_with_Arrays (0.00s) --- PASS: TestSchemaDecodeEncode_Avro/Union_Types (0.00s) === RUN TestSchemaDecodeEncode_JSONSchema === RUN TestSchemaDecodeEncode_JSONSchema/Product_Schema === RUN TestSchemaDecodeEncode_JSONSchema/Nested_Object_Schema --- PASS: TestSchemaDecodeEncode_JSONSchema (0.00s) --- PASS: TestSchemaDecodeEncode_JSONSchema/Product_Schema (0.00s) --- PASS: TestSchemaDecodeEncode_JSONSchema/Nested_Object_Schema (0.00s) === RUN TestSchemaDecodeEncode_Protobuf --- PASS: TestSchemaDecodeEncode_Protobuf (0.00s) === RUN TestSchemaDecodeEncode_ErrorHandling === RUN TestSchemaDecodeEncode_ErrorHandling/Invalid_Confluent_Envelope === RUN TestSchemaDecodeEncode_ErrorHandling/Schema_Not_Found === RUN TestSchemaDecodeEncode_ErrorHandling/Invalid_Avro_Data === RUN TestSchemaDecodeEncode_ErrorHandling/Invalid_JSON_Data --- PASS: TestSchemaDecodeEncode_ErrorHandling (0.00s) --- PASS: TestSchemaDecodeEncode_ErrorHandling/Invalid_Confluent_Envelope (0.00s) --- PASS: TestSchemaDecodeEncode_ErrorHandling/Schema_Not_Found (0.00s) --- PASS: TestSchemaDecodeEncode_ErrorHandling/Invalid_Avro_Data (0.00s) --- PASS: TestSchemaDecodeEncode_ErrorHandling/Invalid_JSON_Data (0.00s) === RUN TestSchemaDecodeEncode_CachePerformance --- PASS: TestSchemaDecodeEncode_CachePerformance (0.01s) === RUN TestParseConfluentEnvelope === RUN TestParseConfluentEnvelope/valid_Avro_message === RUN TestParseConfluentEnvelope/valid_message_with_larger_schema_ID === RUN TestParseConfluentEnvelope/too_short_message === RUN TestParseConfluentEnvelope/no_magic_byte === RUN TestParseConfluentEnvelope/empty_message === RUN TestParseConfluentEnvelope/minimal_valid_message --- PASS: TestParseConfluentEnvelope (0.00s) --- PASS: TestParseConfluentEnvelope/valid_Avro_message (0.00s) --- PASS: TestParseConfluentEnvelope/valid_message_with_larger_schema_ID (0.00s) --- PASS: TestParseConfluentEnvelope/too_short_message (0.00s) --- PASS: TestParseConfluentEnvelope/no_magic_byte (0.00s) --- PASS: TestParseConfluentEnvelope/empty_message (0.00s) --- PASS: TestParseConfluentEnvelope/minimal_valid_message (0.00s) === RUN TestIsSchematized === RUN TestIsSchematized/schematized_message === RUN TestIsSchematized/non-schematized_message === RUN TestIsSchematized/empty_message --- PASS: TestIsSchematized (0.00s) --- PASS: TestIsSchematized/schematized_message (0.00s) --- PASS: TestIsSchematized/non-schematized_message (0.00s) --- PASS: TestIsSchematized/empty_message (0.00s) === RUN TestExtractSchemaID === RUN TestExtractSchemaID/valid_schema_ID === RUN TestExtractSchemaID/large_schema_ID === RUN TestExtractSchemaID/no_magic_byte === RUN TestExtractSchemaID/too_short --- PASS: TestExtractSchemaID (0.00s) --- PASS: TestExtractSchemaID/valid_schema_ID (0.00s) --- PASS: TestExtractSchemaID/large_schema_ID (0.00s) --- PASS: TestExtractSchemaID/no_magic_byte (0.00s) --- PASS: TestExtractSchemaID/too_short (0.00s) === RUN TestCreateConfluentEnvelope === RUN TestCreateConfluentEnvelope/simple_Avro_message === RUN TestCreateConfluentEnvelope/large_schema_ID === RUN TestCreateConfluentEnvelope/empty_payload --- PASS: TestCreateConfluentEnvelope (0.00s) --- PASS: TestCreateConfluentEnvelope/simple_Avro_message (0.00s) --- PASS: TestCreateConfluentEnvelope/large_schema_ID (0.00s) --- PASS: TestCreateConfluentEnvelope/empty_payload (0.00s) === RUN TestEnvelopeValidate === RUN TestEnvelopeValidate/valid_Avro_envelope === RUN TestEnvelopeValidate/zero_schema_ID === RUN TestEnvelopeValidate/empty_payload === RUN TestEnvelopeValidate/unknown_format --- PASS: TestEnvelopeValidate (0.00s) --- PASS: TestEnvelopeValidate/valid_Avro_envelope (0.00s) --- PASS: TestEnvelopeValidate/zero_schema_ID (0.00s) --- PASS: TestEnvelopeValidate/empty_payload (0.00s) --- PASS: TestEnvelopeValidate/unknown_format (0.00s) === RUN TestEnvelopeMetadata --- PASS: TestEnvelopeMetadata (0.00s) === RUN TestEncodeDecodeVarint === RUN TestEncodeDecodeVarint/zero === RUN TestEncodeDecodeVarint/small === RUN TestEncodeDecodeVarint/medium === RUN TestEncodeDecodeVarint/large === RUN TestEncodeDecodeVarint/very_large === RUN TestEncodeDecodeVarint/max_uint32 --- PASS: TestEncodeDecodeVarint (0.00s) --- PASS: TestEncodeDecodeVarint/zero (0.00s) --- PASS: TestEncodeDecodeVarint/small (0.00s) --- PASS: TestEncodeDecodeVarint/medium (0.00s) --- PASS: TestEncodeDecodeVarint/large (0.00s) --- PASS: TestEncodeDecodeVarint/very_large (0.00s) --- PASS: TestEncodeDecodeVarint/max_uint32 (0.00s) === RUN TestCreateConfluentEnvelopeWithProtobufIndexes === RUN TestCreateConfluentEnvelopeWithProtobufIndexes/avro_no_indexes === RUN TestCreateConfluentEnvelopeWithProtobufIndexes/protobuf_no_indexes === RUN TestCreateConfluentEnvelopeWithProtobufIndexes/protobuf_single_index === RUN TestCreateConfluentEnvelopeWithProtobufIndexes/protobuf_multiple_indexes --- PASS: TestCreateConfluentEnvelopeWithProtobufIndexes (0.00s) --- PASS: TestCreateConfluentEnvelopeWithProtobufIndexes/avro_no_indexes (0.00s) --- PASS: TestCreateConfluentEnvelopeWithProtobufIndexes/protobuf_no_indexes (0.00s) --- PASS: TestCreateConfluentEnvelopeWithProtobufIndexes/protobuf_single_index (0.00s) --- PASS: TestCreateConfluentEnvelopeWithProtobufIndexes/protobuf_multiple_indexes (0.00s) === RUN TestProtobufEnvelopeRoundTrip --- PASS: TestProtobufEnvelopeRoundTrip (0.00s) === RUN TestVarintEdgeCases === RUN TestVarintEdgeCases/empty_data === RUN TestVarintEdgeCases/incomplete_varint === RUN TestVarintEdgeCases/max_varint_length --- PASS: TestVarintEdgeCases (0.00s) --- PASS: TestVarintEdgeCases/empty_data (0.00s) --- PASS: TestVarintEdgeCases/incomplete_varint (0.00s) --- PASS: TestVarintEdgeCases/max_varint_length (0.00s) === RUN TestProtobufEnvelopeValidation === RUN TestProtobufEnvelopeValidation/valid_envelope === RUN TestProtobufEnvelopeValidation/zero_schema_id === RUN TestProtobufEnvelopeValidation/empty_payload --- PASS: TestProtobufEnvelopeValidation (0.00s) --- PASS: TestProtobufEnvelopeValidation/valid_envelope (0.00s) --- PASS: TestProtobufEnvelopeValidation/zero_schema_id (0.00s) --- PASS: TestProtobufEnvelopeValidation/empty_payload (0.00s) === RUN TestSchemaEvolutionChecker_AvroBackwardCompatibility === RUN TestSchemaEvolutionChecker_AvroBackwardCompatibility/Compatible_-_Add_optional_field === RUN TestSchemaEvolutionChecker_AvroBackwardCompatibility/Incompatible_-_Remove_field === RUN TestSchemaEvolutionChecker_AvroBackwardCompatibility/Incompatible_-_Add_required_field === RUN TestSchemaEvolutionChecker_AvroBackwardCompatibility/Compatible_-_Type_promotion --- PASS: TestSchemaEvolutionChecker_AvroBackwardCompatibility (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroBackwardCompatibility/Compatible_-_Add_optional_field (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroBackwardCompatibility/Incompatible_-_Remove_field (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroBackwardCompatibility/Incompatible_-_Add_required_field (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroBackwardCompatibility/Compatible_-_Type_promotion (0.00s) === RUN TestSchemaEvolutionChecker_AvroForwardCompatibility === RUN TestSchemaEvolutionChecker_AvroForwardCompatibility/Compatible_-_Remove_optional_field === RUN TestSchemaEvolutionChecker_AvroForwardCompatibility/Incompatible_-_Add_field_without_default_in_old_schema --- PASS: TestSchemaEvolutionChecker_AvroForwardCompatibility (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroForwardCompatibility/Compatible_-_Remove_optional_field (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroForwardCompatibility/Incompatible_-_Add_field_without_default_in_old_schema (0.00s) === RUN TestSchemaEvolutionChecker_AvroFullCompatibility === RUN TestSchemaEvolutionChecker_AvroFullCompatibility/Compatible_-_Add_optional_field_with_default === RUN TestSchemaEvolutionChecker_AvroFullCompatibility/Incompatible_-_Remove_field --- PASS: TestSchemaEvolutionChecker_AvroFullCompatibility (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroFullCompatibility/Compatible_-_Add_optional_field_with_default (0.00s) --- PASS: TestSchemaEvolutionChecker_AvroFullCompatibility/Incompatible_-_Remove_field (0.00s) === RUN TestSchemaEvolutionChecker_JSONSchemaCompatibility === RUN TestSchemaEvolutionChecker_JSONSchemaCompatibility/Compatible_-_Add_optional_property === RUN TestSchemaEvolutionChecker_JSONSchemaCompatibility/Incompatible_-_Add_required_property === RUN TestSchemaEvolutionChecker_JSONSchemaCompatibility/Incompatible_-_Remove_property --- PASS: TestSchemaEvolutionChecker_JSONSchemaCompatibility (0.00s) --- PASS: TestSchemaEvolutionChecker_JSONSchemaCompatibility/Compatible_-_Add_optional_property (0.00s) --- PASS: TestSchemaEvolutionChecker_JSONSchemaCompatibility/Incompatible_-_Add_required_property (0.00s) --- PASS: TestSchemaEvolutionChecker_JSONSchemaCompatibility/Incompatible_-_Remove_property (0.00s) === RUN TestSchemaEvolutionChecker_ProtobufCompatibility === RUN TestSchemaEvolutionChecker_ProtobufCompatibility/Simplified_Protobuf_check --- PASS: TestSchemaEvolutionChecker_ProtobufCompatibility (0.00s) --- PASS: TestSchemaEvolutionChecker_ProtobufCompatibility/Simplified_Protobuf_check (0.00s) === RUN TestSchemaEvolutionChecker_NoCompatibility --- PASS: TestSchemaEvolutionChecker_NoCompatibility (0.00s) === RUN TestSchemaEvolutionChecker_TypePromotion === RUN TestSchemaEvolutionChecker_TypePromotion/int_to_long === RUN TestSchemaEvolutionChecker_TypePromotion/int_to_float === RUN TestSchemaEvolutionChecker_TypePromotion/int_to_double === RUN TestSchemaEvolutionChecker_TypePromotion/long_to_float === RUN TestSchemaEvolutionChecker_TypePromotion/long_to_double === RUN TestSchemaEvolutionChecker_TypePromotion/float_to_double === RUN TestSchemaEvolutionChecker_TypePromotion/string_to_bytes === RUN TestSchemaEvolutionChecker_TypePromotion/bytes_to_string === RUN TestSchemaEvolutionChecker_TypePromotion/long_to_int === RUN TestSchemaEvolutionChecker_TypePromotion/double_to_float === RUN TestSchemaEvolutionChecker_TypePromotion/string_to_int --- PASS: TestSchemaEvolutionChecker_TypePromotion (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/int_to_long (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/int_to_float (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/int_to_double (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/long_to_float (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/long_to_double (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/float_to_double (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/string_to_bytes (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/bytes_to_string (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/long_to_int (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/double_to_float (0.00s) --- PASS: TestSchemaEvolutionChecker_TypePromotion/string_to_int (0.00s) === RUN TestSchemaEvolutionChecker_SuggestEvolution === RUN TestSchemaEvolutionChecker_SuggestEvolution/Compatible_schema === RUN TestSchemaEvolutionChecker_SuggestEvolution/Incompatible_schema_with_suggestions --- PASS: TestSchemaEvolutionChecker_SuggestEvolution (0.00s) --- PASS: TestSchemaEvolutionChecker_SuggestEvolution/Compatible_schema (0.00s) --- PASS: TestSchemaEvolutionChecker_SuggestEvolution/Incompatible_schema_with_suggestions (0.00s) === RUN TestSchemaEvolutionChecker_CanEvolve --- PASS: TestSchemaEvolutionChecker_CanEvolve (0.00s) === RUN TestSchemaEvolutionChecker_ExtractFields === RUN TestSchemaEvolutionChecker_ExtractFields/Extract_Avro_fields === RUN TestSchemaEvolutionChecker_ExtractFields/Extract_JSON_Schema_required_fields === RUN TestSchemaEvolutionChecker_ExtractFields/Extract_JSON_Schema_properties --- PASS: TestSchemaEvolutionChecker_ExtractFields (0.00s) --- PASS: TestSchemaEvolutionChecker_ExtractFields/Extract_Avro_fields (0.00s) --- PASS: TestSchemaEvolutionChecker_ExtractFields/Extract_JSON_Schema_required_fields (0.00s) --- PASS: TestSchemaEvolutionChecker_ExtractFields/Extract_JSON_Schema_properties (0.00s) === RUN TestFullIntegration_AvroWorkflow === RUN TestFullIntegration_AvroWorkflow/Producer_Workflow integration_test.go:88: Successfully processed producer message with 5 fields === RUN TestFullIntegration_AvroWorkflow/Consumer_Workflow integration_test.go:140: Successfully reconstructed consumer message: 38 bytes === RUN TestFullIntegration_AvroWorkflow/Round_Trip_Integrity integration_test.go:203: Round-trip integrity test passed --- PASS: TestFullIntegration_AvroWorkflow (0.00s) --- PASS: TestFullIntegration_AvroWorkflow/Producer_Workflow (0.00s) --- PASS: TestFullIntegration_AvroWorkflow/Consumer_Workflow (0.00s) --- PASS: TestFullIntegration_AvroWorkflow/Round_Trip_Integrity (0.00s) === RUN TestFullIntegration_MultiFormatSupport === RUN TestFullIntegration_MultiFormatSupport/Avro_Format integration_test.go:281: Successfully processed Avro_Format format === RUN TestFullIntegration_MultiFormatSupport/JSON_Schema_Format integration_test.go:281: Successfully processed JSON_Schema_Format format --- PASS: TestFullIntegration_MultiFormatSupport (0.00s) --- PASS: TestFullIntegration_MultiFormatSupport/Avro_Format (0.00s) --- PASS: TestFullIntegration_MultiFormatSupport/JSON_Schema_Format (0.00s) === RUN TestIntegration_CachePerformance integration_test.go:343: Cache performance: First decode: 487.199µs, Average cached: 44.044µs integration_test.go:345: Cache stats: 1 decoders, 1 schemas, 0 subjects --- PASS: TestIntegration_CachePerformance (0.01s) === RUN TestIntegration_ErrorHandling === RUN TestIntegration_ErrorHandling/Non_Schematized_Message integration_test.go:405: Expected error occurred: message is not schematized === RUN TestIntegration_ErrorHandling/Invalid_Schema_ID integration_test.go:405: Expected error occurred: failed to get schema 999: schema registry error 404: === RUN TestIntegration_ErrorHandling/Empty_Payload integration_test.go:405: Expected error occurred: invalid envelope: empty payload === RUN TestIntegration_ErrorHandling/Corrupted_Avro_Data integration_test.go:405: Expected error occurred: failed to decode Avro message: strict validation failed: failed to decode Avro data: cannot decode binary record "User" field "name": cannot decode binary string: cannot decode binary bytes: short buffer --- PASS: TestIntegration_ErrorHandling (0.00s) --- PASS: TestIntegration_ErrorHandling/Non_Schematized_Message (0.00s) --- PASS: TestIntegration_ErrorHandling/Invalid_Schema_ID (0.00s) --- PASS: TestIntegration_ErrorHandling/Empty_Payload (0.00s) --- PASS: TestIntegration_ErrorHandling/Corrupted_Avro_Data (0.00s) === RUN TestIntegration_SchemaEvolution === RUN TestIntegration_SchemaEvolution/Schema_V1_Message === RUN TestIntegration_SchemaEvolution/Schema_V2_Message --- PASS: TestIntegration_SchemaEvolution (0.00s) --- PASS: TestIntegration_SchemaEvolution/Schema_V1_Message (0.00s) --- PASS: TestIntegration_SchemaEvolution/Schema_V2_Message (0.00s) === RUN TestNewJSONSchemaDecoder === RUN TestNewJSONSchemaDecoder/valid_object_schema === RUN TestNewJSONSchemaDecoder/valid_array_schema === RUN TestNewJSONSchemaDecoder/valid_string_schema_with_format === RUN TestNewJSONSchemaDecoder/invalid_JSON === RUN TestNewJSONSchemaDecoder/empty_schema --- PASS: TestNewJSONSchemaDecoder (0.00s) --- PASS: TestNewJSONSchemaDecoder/valid_object_schema (0.00s) --- PASS: TestNewJSONSchemaDecoder/valid_array_schema (0.00s) --- PASS: TestNewJSONSchemaDecoder/valid_string_schema_with_format (0.00s) --- PASS: TestNewJSONSchemaDecoder/invalid_JSON (0.00s) --- PASS: TestNewJSONSchemaDecoder/empty_schema (0.00s) === RUN TestJSONSchemaDecoder_Decode === RUN TestJSONSchemaDecoder_Decode/valid_complete_data === RUN TestJSONSchemaDecoder_Decode/valid_minimal_data === RUN TestJSONSchemaDecoder_Decode/missing_required_field === RUN TestJSONSchemaDecoder_Decode/invalid_type === RUN TestJSONSchemaDecoder_Decode/invalid_email_format === RUN TestJSONSchemaDecoder_Decode/negative_age --- PASS: TestJSONSchemaDecoder_Decode (0.00s) --- PASS: TestJSONSchemaDecoder_Decode/valid_complete_data (0.00s) --- PASS: TestJSONSchemaDecoder_Decode/valid_minimal_data (0.00s) --- PASS: TestJSONSchemaDecoder_Decode/missing_required_field (0.00s) --- PASS: TestJSONSchemaDecoder_Decode/invalid_type (0.00s) --- PASS: TestJSONSchemaDecoder_Decode/invalid_email_format (0.00s) --- PASS: TestJSONSchemaDecoder_Decode/negative_age (0.00s) === RUN TestJSONSchemaDecoder_DecodeToRecordValue --- PASS: TestJSONSchemaDecoder_DecodeToRecordValue (0.00s) === RUN TestJSONSchemaDecoder_InferRecordType --- PASS: TestJSONSchemaDecoder_InferRecordType (0.00s) === RUN TestJSONSchemaDecoder_EncodeFromRecordValue --- PASS: TestJSONSchemaDecoder_EncodeFromRecordValue (0.00s) === RUN TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas === RUN TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/array_schema === RUN TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/string_schema === RUN TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/number_schema === RUN TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/boolean_schema --- PASS: TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas (0.00s) --- PASS: TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/array_schema (0.00s) --- PASS: TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/string_schema (0.00s) --- PASS: TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/number_schema (0.00s) --- PASS: TestJSONSchemaDecoder_ArrayAndPrimitiveSchemas/boolean_schema (0.00s) === RUN TestJSONSchemaDecoder_GetSchemaInfo --- PASS: TestJSONSchemaDecoder_GetSchemaInfo (0.00s) === RUN TestAvroLoadTestDecoding loadtest_decode_test.go:127: Avro encoded size: 68 bytes loadtest_decode_test.go:133: Confluent wire format size: 73 bytes loadtest_decode_test.go:170: ✅ Avro decoding successful: 7 fields --- PASS: TestAvroLoadTestDecoding (0.00s) === RUN TestJSONSchemaLoadTestDecoding loadtest_decode_test.go:183: JSON encoded size: 174 bytes loadtest_decode_test.go:184: JSON content: {"id":"msg-test-123","timestamp":1784582995208233766,"producer_id":0,"counter":42,"user_id":"user-789","event_type":"click","properties":{"browser":"chrome","version":"1.0"}} loadtest_decode_test.go:190: Confluent wire format size: 179 bytes loadtest_decode_test.go:227: ✅ JSON Schema decoding successful: 7 fields --- PASS: TestJSONSchemaLoadTestDecoding (0.00s) === RUN TestProtobufLoadTestDecoding loadtest_decode_test.go:241: JSON (for Protobuf) encoded size: 174 bytes loadtest_decode_test.go:242: JSON content: {"id":"msg-test-123","timestamp":1784582995208416945,"producer_id":0,"counter":42,"user_id":"user-789","event_type":"click","properties":{"browser":"chrome","version":"1.0"}} loadtest_decode_test.go:248: Confluent wire format size: 179 bytes loadtest_decode_test.go:275: Unexpectedly succeeded in decoding JSON as Protobuf loadtest_decode_test.go:277: RecordValue has 7 fields --- PASS: TestProtobufLoadTestDecoding (0.00s) === RUN TestManager_SchemaEvolution === RUN TestManager_SchemaEvolution/Compatible_Avro_evolution === RUN TestManager_SchemaEvolution/Incompatible_Avro_evolution === RUN TestManager_SchemaEvolution/Schema_evolution_suggestions === RUN TestManager_SchemaEvolution/JSON_Schema_evolution === RUN TestManager_SchemaEvolution/Full_compatibility_check === RUN TestManager_SchemaEvolution/Type_promotion_compatibility --- PASS: TestManager_SchemaEvolution (0.00s) --- PASS: TestManager_SchemaEvolution/Compatible_Avro_evolution (0.00s) --- PASS: TestManager_SchemaEvolution/Incompatible_Avro_evolution (0.00s) --- PASS: TestManager_SchemaEvolution/Schema_evolution_suggestions (0.00s) --- PASS: TestManager_SchemaEvolution/JSON_Schema_evolution (0.00s) --- PASS: TestManager_SchemaEvolution/Full_compatibility_check (0.00s) --- PASS: TestManager_SchemaEvolution/Type_promotion_compatibility (0.00s) === RUN TestManager_CompatibilityLevels === RUN TestManager_CompatibilityLevels/Get_default_compatibility_level === RUN TestManager_CompatibilityLevels/Set_compatibility_level --- PASS: TestManager_CompatibilityLevels (0.00s) --- PASS: TestManager_CompatibilityLevels/Get_default_compatibility_level (0.00s) --- PASS: TestManager_CompatibilityLevels/Set_compatibility_level (0.00s) === RUN TestManager_CanEvolveSchema === RUN TestManager_CanEvolveSchema/Compatible_evolution === RUN TestManager_CanEvolveSchema/Incompatible_evolution --- PASS: TestManager_CanEvolveSchema (0.00s) --- PASS: TestManager_CanEvolveSchema/Compatible_evolution (0.00s) --- PASS: TestManager_CanEvolveSchema/Incompatible_evolution (0.00s) === RUN TestManager_SchemaEvolutionWorkflow === RUN TestManager_SchemaEvolutionWorkflow/Complete_evolution_workflow --- PASS: TestManager_SchemaEvolutionWorkflow (0.00s) --- PASS: TestManager_SchemaEvolutionWorkflow/Complete_evolution_workflow (0.00s) === RUN TestManager_DecodeMessage --- PASS: TestManager_DecodeMessage (0.00s) === RUN TestManager_IsSchematized === RUN TestManager_IsSchematized/schematized_message === RUN TestManager_IsSchematized/non-schematized_message === RUN TestManager_IsSchematized/empty_message --- PASS: TestManager_IsSchematized (0.00s) --- PASS: TestManager_IsSchematized/schematized_message (0.00s) --- PASS: TestManager_IsSchematized/non-schematized_message (0.00s) --- PASS: TestManager_IsSchematized/empty_message (0.00s) === RUN TestManager_GetSchemaInfo --- PASS: TestManager_GetSchemaInfo (0.00s) === RUN TestManager_CacheManagement --- PASS: TestManager_CacheManagement (0.00s) === RUN TestManager_EncodeMessage --- PASS: TestManager_EncodeMessage (0.00s) === RUN TestProtobufDecoder_BasicDecoding === RUN TestProtobufDecoder_BasicDecoding/NewProtobufDecoder_with_binary_descriptor protobuf_decoder_test.go:40: Protobuf decoder creation succeeded - Phase E3 is working! === RUN TestProtobufDecoder_BasicDecoding/NewProtobufDecoder_with_empty_message_name protobuf_decoder_test.go:60: Empty message name resolution succeeded - Phase E3 is working! --- PASS: TestProtobufDecoder_BasicDecoding (0.00s) --- PASS: TestProtobufDecoder_BasicDecoding/NewProtobufDecoder_with_binary_descriptor (0.00s) --- PASS: TestProtobufDecoder_BasicDecoding/NewProtobufDecoder_with_empty_message_name (0.00s) === RUN TestProtobufDecoder_Integration === RUN TestProtobufDecoder_Integration/Parse_complex_descriptor protobuf_decoder_test.go:86: Empty message name resolution succeeded! protobuf_decoder_test.go:100: Complex message resolution succeeded! --- PASS: TestProtobufDecoder_Integration (0.00s) --- PASS: TestProtobufDecoder_Integration/Parse_complex_descriptor (0.00s) === RUN TestProtobufDecoder_Caching === RUN TestProtobufDecoder_Caching/Decoder_creation_uses_cache --- PASS: TestProtobufDecoder_Caching (0.00s) --- PASS: TestProtobufDecoder_Caching/Decoder_creation_uses_cache (0.00s) === RUN TestProtobufDecoder_ErrorHandling === RUN TestProtobufDecoder_ErrorHandling/Invalid_binary_data === RUN TestProtobufDecoder_ErrorHandling/Empty_binary_data === RUN TestProtobufDecoder_ErrorHandling/FileDescriptorSet_with_no_messages --- PASS: TestProtobufDecoder_ErrorHandling (0.00s) --- PASS: TestProtobufDecoder_ErrorHandling/Invalid_binary_data (0.00s) --- PASS: TestProtobufDecoder_ErrorHandling/Empty_binary_data (0.00s) --- PASS: TestProtobufDecoder_ErrorHandling/FileDescriptorSet_with_no_messages (0.00s) === RUN TestProtobufDescriptorParser_BasicParsing === RUN TestProtobufDescriptorParser_BasicParsing/Parse_Simple_Message_Descriptor protobuf_descriptor_test.go:42: Simple message descriptor resolution succeeded - Phase E3 is working! === RUN TestProtobufDescriptorParser_BasicParsing/Parse_Complex_Message_Descriptor === RUN TestProtobufDescriptorParser_BasicParsing/Cache_Functionality protobuf_descriptor_test.go:101: Cache functionality working with successful descriptor resolution! --- PASS: TestProtobufDescriptorParser_BasicParsing (0.00s) --- PASS: TestProtobufDescriptorParser_BasicParsing/Parse_Simple_Message_Descriptor (0.00s) --- PASS: TestProtobufDescriptorParser_BasicParsing/Parse_Complex_Message_Descriptor (0.00s) --- PASS: TestProtobufDescriptorParser_BasicParsing/Cache_Functionality (0.00s) === RUN TestProtobufDescriptorParser_Validation === RUN TestProtobufDescriptorParser_Validation/Invalid_Binary_Data === RUN TestProtobufDescriptorParser_Validation/Empty_FileDescriptorSet === RUN TestProtobufDescriptorParser_Validation/FileDescriptor_Without_Name === RUN TestProtobufDescriptorParser_Validation/FileDescriptor_Without_Package --- PASS: TestProtobufDescriptorParser_Validation (0.00s) --- PASS: TestProtobufDescriptorParser_Validation/Invalid_Binary_Data (0.00s) --- PASS: TestProtobufDescriptorParser_Validation/Empty_FileDescriptorSet (0.00s) --- PASS: TestProtobufDescriptorParser_Validation/FileDescriptor_Without_Name (0.00s) --- PASS: TestProtobufDescriptorParser_Validation/FileDescriptor_Without_Package (0.00s) === RUN TestProtobufDescriptorParser_MessageSearch === RUN TestProtobufDescriptorParser_MessageSearch/Message_Not_Found === RUN TestProtobufDescriptorParser_MessageSearch/Nested_Message_Search protobuf_descriptor_test.go:235: Nested message resolution succeeded - Phase E3 is working! --- PASS: TestProtobufDescriptorParser_MessageSearch (0.00s) --- PASS: TestProtobufDescriptorParser_MessageSearch/Message_Not_Found (0.00s) --- PASS: TestProtobufDescriptorParser_MessageSearch/Nested_Message_Search (0.00s) === RUN TestProtobufDescriptorParser_Dependencies === RUN TestProtobufDescriptorParser_Dependencies/Extract_Dependencies --- PASS: TestProtobufDescriptorParser_Dependencies (0.00s) --- PASS: TestProtobufDescriptorParser_Dependencies/Extract_Dependencies (0.00s) === RUN TestProtobufSchema_Methods === RUN TestProtobufSchema_Methods/GetMessageFields_Implemented === RUN TestProtobufSchema_Methods/GetFieldByName_Implemented === RUN TestProtobufSchema_Methods/GetFieldByNumber_Implemented === RUN TestProtobufSchema_Methods/ValidateMessage_Requires_MessageDescriptor --- PASS: TestProtobufSchema_Methods (0.00s) --- PASS: TestProtobufSchema_Methods/GetMessageFields_Implemented (0.00s) --- PASS: TestProtobufSchema_Methods/GetFieldByName_Implemented (0.00s) --- PASS: TestProtobufSchema_Methods/GetFieldByNumber_Implemented (0.00s) --- PASS: TestProtobufSchema_Methods/ValidateMessage_Requires_MessageDescriptor (0.00s) === RUN TestProtobufDescriptorParser_CacheManagement --- PASS: TestProtobufDescriptorParser_CacheManagement (0.00s) === RUN TestSchemaReconstruction_Avro reconstruction_test.go:77: Original Avro binary length: 11 reconstruction_test.go:78: Original Confluent message length: 16 reconstruction_test.go:85: Parsed envelope - SchemaID: 1, Format: AVRO, Payload length: 11 reconstruction_test.go:124: Successfully completed round-trip: Original -> Decode -> Encode -> Decode reconstruction_test.go:125: Original message size: 16 bytes reconstruction_test.go:126: Reconstructed message size: 16 bytes --- PASS: TestSchemaReconstruction_Avro (0.00s) === RUN TestSchemaReconstruction_MultipleFormats === RUN TestSchemaReconstruction_MultipleFormats/Avro === RUN TestSchemaReconstruction_MultipleFormats/Protobuf === RUN TestSchemaReconstruction_MultipleFormats/JSON_Schema --- PASS: TestSchemaReconstruction_MultipleFormats (0.00s) --- PASS: TestSchemaReconstruction_MultipleFormats/Avro (0.00s) --- PASS: TestSchemaReconstruction_MultipleFormats/Protobuf (0.00s) --- PASS: TestSchemaReconstruction_MultipleFormats/JSON_Schema (0.00s) === RUN TestConfluentEnvelope_RoundTrip === RUN TestConfluentEnvelope_RoundTrip/Avro_message reconstruction_test.go:259: Successfully round-tripped Avro message envelope: 17 bytes === RUN TestConfluentEnvelope_RoundTrip/Protobuf_message_with_indexes reconstruction_test.go:259: Successfully round-tripped Protobuf message with indexes envelope: 21 bytes === RUN TestConfluentEnvelope_RoundTrip/JSON_Schema_message reconstruction_test.go:259: Successfully round-tripped JSON Schema message envelope: 17 bytes --- PASS: TestConfluentEnvelope_RoundTrip (0.00s) --- PASS: TestConfluentEnvelope_RoundTrip/Avro_message (0.00s) --- PASS: TestConfluentEnvelope_RoundTrip/Protobuf_message_with_indexes (0.00s) --- PASS: TestConfluentEnvelope_RoundTrip/JSON_Schema_message (0.00s) === RUN TestSchemaMetadata_Preservation reconstruction_test.go:306: Successfully preserved and reconstructed schema metadata --- PASS: TestSchemaMetadata_Preservation (0.00s) === RUN TestNewRegistryClient --- PASS: TestNewRegistryClient (0.00s) === RUN TestRegistryClient_GetSchemaByID === RUN TestRegistryClient_GetSchemaByID/successful_fetch === RUN TestRegistryClient_GetSchemaByID/schema_not_found === RUN TestRegistryClient_GetSchemaByID/cache_hit --- PASS: TestRegistryClient_GetSchemaByID (0.00s) --- PASS: TestRegistryClient_GetSchemaByID/successful_fetch (0.00s) --- PASS: TestRegistryClient_GetSchemaByID/schema_not_found (0.00s) --- PASS: TestRegistryClient_GetSchemaByID/cache_hit (0.00s) === RUN TestRegistryClient_GetLatestSchema --- PASS: TestRegistryClient_GetLatestSchema (0.00s) === RUN TestRegistryClient_RegisterSchema --- PASS: TestRegistryClient_RegisterSchema (0.00s) === RUN TestRegistryClient_CheckCompatibility --- PASS: TestRegistryClient_CheckCompatibility (0.00s) === RUN TestRegistryClient_ListSubjects --- PASS: TestRegistryClient_ListSubjects (0.00s) === RUN TestRegistryClient_DetectSchemaFormat === RUN TestRegistryClient_DetectSchemaFormat/Avro_record_schema === RUN TestRegistryClient_DetectSchemaFormat/Avro_enum_schema === RUN TestRegistryClient_DetectSchemaFormat/JSON_Schema === RUN TestRegistryClient_DetectSchemaFormat/Protobuf_(non-JSON) === RUN TestRegistryClient_DetectSchemaFormat/Simple_Avro_primitive --- PASS: TestRegistryClient_DetectSchemaFormat (0.00s) --- PASS: TestRegistryClient_DetectSchemaFormat/Avro_record_schema (0.00s) --- PASS: TestRegistryClient_DetectSchemaFormat/Avro_enum_schema (0.00s) --- PASS: TestRegistryClient_DetectSchemaFormat/JSON_Schema (0.00s) --- PASS: TestRegistryClient_DetectSchemaFormat/Protobuf_(non-JSON) (0.00s) --- PASS: TestRegistryClient_DetectSchemaFormat/Simple_Avro_primitive (0.00s) === RUN TestRegistryClient_CacheManagement --- PASS: TestRegistryClient_CacheManagement (0.00s) === RUN TestRegistryClient_HealthCheck === RUN TestRegistryClient_HealthCheck/healthy_registry === RUN TestRegistryClient_HealthCheck/unhealthy_registry --- PASS: TestRegistryClient_HealthCheck (0.00s) --- PASS: TestRegistryClient_HealthCheck/healthy_registry (0.00s) --- PASS: TestRegistryClient_HealthCheck/unhealthy_registry (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/kafka/schema 0.954s === RUN TestWriteRowsNoPanic --- PASS: TestWriteRowsNoPanic (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/logstore 0.071s === RUN TestConsumerGroupPosition_JSON === RUN TestConsumerGroupPosition_JSON/offset-based_position consumer_group_storage_test.go:53: JSON: {"type":"offset","value":12345,"offset_type":"EXACT_OFFSET","committed_at":1784582994322,"metadata":"test metadata"} === RUN TestConsumerGroupPosition_JSON/timestamp-based_position consumer_group_storage_test.go:53: JSON: {"type":"timestamp","value":1784582994322199941,"offset_type":"EXACT_TS_NS","committed_at":1784582994322,"metadata":"checkpoint at 2024-10-05"} === RUN TestConsumerGroupPosition_JSON/minimal_position consumer_group_storage_test.go:53: JSON: {"type":"offset","value":42,"offset_type":"","committed_at":0,"metadata":""} --- PASS: TestConsumerGroupPosition_JSON (0.00s) --- PASS: TestConsumerGroupPosition_JSON/offset-based_position (0.00s) --- PASS: TestConsumerGroupPosition_JSON/timestamp-based_position (0.00s) --- PASS: TestConsumerGroupPosition_JSON/minimal_position (0.00s) === RUN TestConsumerGroupPosition_JSONExamples consumer_group_storage_test.go:93: Example 0: Type=offset, Value=12345 consumer_group_storage_test.go:93: Example 1: Type=timestamp, Value=1696521600000000000 consumer_group_storage_test.go:93: Example 2: Type=offset, Value=42 --- PASS: TestConsumerGroupPosition_JSONExamples (0.00s) === RUN TestConsumerGroupPosition_TypeValidation --- PASS: TestConsumerGroupPosition_TypeValidation (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/offset 0.048s === RUN Test_allocateOneBroker === RUN Test_allocateOneBroker/test_only_one_broker I0720 21:29:54.329174 allocate.go:54 allocate topic partitions 1: [partition:{ring_size:2520 range_stop:2520 unix_time_ns:1784582994329058126} leader_broker:"localhost:17777"] --- PASS: Test_allocateOneBroker (0.00s) --- PASS: Test_allocateOneBroker/test_only_one_broker (0.00s) === RUN TestEnsureAssignmentsToActiveBrokersX === RUN TestEnsureAssignmentsToActiveBrokersX/test_empty_leader test empty leader before [partition:{} follower_broker:"localhost:2"] test empty leader after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:2"] === RUN TestEnsureAssignmentsToActiveBrokersX/test_empty_follower test empty follower before [partition:{} leader_broker:"localhost:1"] test empty follower after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:4"] === RUN TestEnsureAssignmentsToActiveBrokersX/test_dead_follower test dead follower before [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:200"] test dead follower after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:4"] === RUN TestEnsureAssignmentsToActiveBrokersX/test_dead_leader_and_follower test dead leader and follower before [partition:{} leader_broker:"localhost:100" follower_broker:"localhost:200"] test dead leader and follower after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:6"] === RUN TestEnsureAssignmentsToActiveBrokersX/test_low_active_brokers test low active brokers before [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:2"] test low active brokers after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:2"] === RUN TestEnsureAssignmentsToActiveBrokersX/test_low_active_brokers_with_one_follower test low active brokers with one follower before [partition:{} leader_broker:"localhost:1"] test low active brokers with one follower after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:2"] === RUN TestEnsureAssignmentsToActiveBrokersX/test_single_active_broker test single active broker before [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:2"] test single active broker after [partition:{} leader_broker:"localhost:1" follower_broker:"localhost:1"] --- PASS: TestEnsureAssignmentsToActiveBrokersX (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_empty_leader (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_empty_follower (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_dead_follower (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_dead_leader_and_follower (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_low_active_brokers (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_low_active_brokers_with_one_follower (0.00s) --- PASS: TestEnsureAssignmentsToActiveBrokersX/test_single_active_broker (0.00s) === RUN TestBalanceTopicPartitionOnBrokers === RUN TestBalanceTopicPartitionOnBrokers/test --- PASS: TestBalanceTopicPartitionOnBrokers (0.00s) --- PASS: TestBalanceTopicPartitionOnBrokers/test (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/pub_balancer 0.051s === RUN TestSplitFlatSchemaToKeyValue --- PASS: TestSplitFlatSchemaToKeyValue (0.00s) === RUN TestSplitFlatSchemaToKeyValueMissingColumns --- PASS: TestSplitFlatSchemaToKeyValueMissingColumns (0.00s) === RUN TestCombineFlatSchemaFromKeyValue --- PASS: TestCombineFlatSchemaFromKeyValue (0.00s) === RUN TestExtractKeyColumnsFromCombinedSchema --- PASS: TestExtractKeyColumnsFromCombinedSchema (0.00s) === RUN TestValidateKeyColumns --- PASS: TestValidateKeyColumns (0.00s) === RUN TestEnumScalarType === RUN TestEnumScalarType/Boolean === RUN TestEnumScalarType/Integer === RUN TestEnumScalarType/Long === RUN TestEnumScalarType/Float === RUN TestEnumScalarType/Double === RUN TestEnumScalarType/Bytes === RUN TestEnumScalarType/String --- PASS: TestEnumScalarType (0.00s) --- PASS: TestEnumScalarType/Boolean (0.00s) --- PASS: TestEnumScalarType/Integer (0.00s) --- PASS: TestEnumScalarType/Long (0.00s) --- PASS: TestEnumScalarType/Float (0.00s) --- PASS: TestEnumScalarType/Double (0.00s) --- PASS: TestEnumScalarType/Bytes (0.00s) --- PASS: TestEnumScalarType/String (0.00s) === RUN TestField --- PASS: TestField (0.00s) === RUN TestRecordType fields: < name: "field_key" field_index: 1 type: < scalar_type: INT32 > > fields: < name: "field_record" field_index: 2 type: < record_type: < fields: < name: "field_1" field_index: 1 type: < scalar_type: INT32 > > fields: < name: "field_2" field_index: 2 type: < scalar_type: STRING > > > > > {"fields":[{"name":"field_key","field_index":1,"type":{"Kind":{"ScalarType":1}}},{"name":"field_record","field_index":2,"type":{"Kind":{"RecordType":{"fields":[{"name":"field_1","field_index":1,"type":{"Kind":{"ScalarType":1}}},{"name":"field_2","field_index":2,"type":{"Kind":{"ScalarType":7}}}]}}}}]} --- PASS: TestRecordType (0.00s) === RUN TestStructToSchema === RUN TestStructToSchema/scalar_type === RUN TestStructToSchema/simple_struct_type === RUN TestStructToSchema/simple_list === RUN TestStructToSchema/simple_[]byte === RUN TestStructToSchema/nested_simpe_structs === RUN TestStructToSchema/nested_struct_type --- PASS: TestStructToSchema (0.00s) --- PASS: TestStructToSchema/scalar_type (0.00s) --- PASS: TestStructToSchema/simple_struct_type (0.00s) --- PASS: TestStructToSchema/simple_list (0.00s) --- PASS: TestStructToSchema/simple_[]byte (0.00s) --- PASS: TestStructToSchema/nested_simpe_structs (0.00s) --- PASS: TestStructToSchema/nested_struct_type (0.00s) === RUN TestToParquetLevels === RUN TestToParquetLevels/nested_type --- PASS: TestToParquetLevels (0.00s) --- PASS: TestToParquetLevels/nested_type (0.00s) === RUN TestToParquetValue_BasicTypes === RUN TestToParquetValue_BasicTypes/BoolValue_true === RUN TestToParquetValue_BasicTypes/Int32Value === RUN TestToParquetValue_BasicTypes/Int64Value === RUN TestToParquetValue_BasicTypes/FloatValue === RUN TestToParquetValue_BasicTypes/DoubleValue === RUN TestToParquetValue_BasicTypes/BytesValue === RUN TestToParquetValue_BasicTypes/BytesValue_empty === RUN TestToParquetValue_BasicTypes/StringValue --- PASS: TestToParquetValue_BasicTypes (0.00s) --- PASS: TestToParquetValue_BasicTypes/BoolValue_true (0.00s) --- PASS: TestToParquetValue_BasicTypes/Int32Value (0.00s) --- PASS: TestToParquetValue_BasicTypes/Int64Value (0.00s) --- PASS: TestToParquetValue_BasicTypes/FloatValue (0.00s) --- PASS: TestToParquetValue_BasicTypes/DoubleValue (0.00s) --- PASS: TestToParquetValue_BasicTypes/BytesValue (0.00s) --- PASS: TestToParquetValue_BasicTypes/BytesValue_empty (0.00s) --- PASS: TestToParquetValue_BasicTypes/StringValue (0.00s) === RUN TestToParquetValue_TimestampValue === RUN TestToParquetValue_TimestampValue/Valid_TimestampValue_UTC === RUN TestToParquetValue_TimestampValue/Valid_TimestampValue_local === RUN TestToParquetValue_TimestampValue/TimestampValue_zero === RUN TestToParquetValue_TimestampValue/TimestampValue_negative_(before_epoch) === RUN TestToParquetValue_TimestampValue/TimestampValue_nil_pointer --- PASS: TestToParquetValue_TimestampValue (0.00s) --- PASS: TestToParquetValue_TimestampValue/Valid_TimestampValue_UTC (0.00s) --- PASS: TestToParquetValue_TimestampValue/Valid_TimestampValue_local (0.00s) --- PASS: TestToParquetValue_TimestampValue/TimestampValue_zero (0.00s) --- PASS: TestToParquetValue_TimestampValue/TimestampValue_negative_(before_epoch) (0.00s) --- PASS: TestToParquetValue_TimestampValue/TimestampValue_nil_pointer (0.00s) === RUN TestToParquetValue_DateValue === RUN TestToParquetValue_DateValue/Valid_DateValue_(2024-01-01) === RUN TestToParquetValue_DateValue/DateValue_epoch_(1970-01-01) === RUN TestToParquetValue_DateValue/DateValue_before_epoch === RUN TestToParquetValue_DateValue/DateValue_nil_pointer --- PASS: TestToParquetValue_DateValue (0.00s) --- PASS: TestToParquetValue_DateValue/Valid_DateValue_(2024-01-01) (0.00s) --- PASS: TestToParquetValue_DateValue/DateValue_epoch_(1970-01-01) (0.00s) --- PASS: TestToParquetValue_DateValue/DateValue_before_epoch (0.00s) --- PASS: TestToParquetValue_DateValue/DateValue_nil_pointer (0.00s) === RUN TestToParquetValue_DecimalValue === RUN TestToParquetValue_DecimalValue/Small_Decimal_(precision_<=_9)_-_positive === RUN TestToParquetValue_DecimalValue/Small_Decimal_(precision_<=_9)_-_negative === RUN TestToParquetValue_DecimalValue/Medium_Decimal_(9_<_precision_<=_18) === RUN TestToParquetValue_DecimalValue/Large_Decimal_(precision_>_18) === RUN TestToParquetValue_DecimalValue/Decimal_with_zero_precision === RUN TestToParquetValue_DecimalValue/Decimal_nil_pointer === RUN TestToParquetValue_DecimalValue/Decimal_with_nil_Value_bytes === RUN TestToParquetValue_DecimalValue/Decimal_with_empty_Value_bytes === RUN TestToParquetValue_DecimalValue/Decimal_out_of_int32_range_(stored_as_binary) === RUN TestToParquetValue_DecimalValue/Decimal_out_of_int64_range_(stored_as_binary) === RUN TestToParquetValue_DecimalValue/Decimal_extremely_large_value_(should_be_rejected) --- PASS: TestToParquetValue_DecimalValue (0.00s) --- PASS: TestToParquetValue_DecimalValue/Small_Decimal_(precision_<=_9)_-_positive (0.00s) --- PASS: TestToParquetValue_DecimalValue/Small_Decimal_(precision_<=_9)_-_negative (0.00s) --- PASS: TestToParquetValue_DecimalValue/Medium_Decimal_(9_<_precision_<=_18) (0.00s) --- PASS: TestToParquetValue_DecimalValue/Large_Decimal_(precision_>_18) (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_with_zero_precision (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_nil_pointer (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_with_nil_Value_bytes (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_with_empty_Value_bytes (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_out_of_int32_range_(stored_as_binary) (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_out_of_int64_range_(stored_as_binary) (0.00s) --- PASS: TestToParquetValue_DecimalValue/Decimal_extremely_large_value_(should_be_rejected) (0.00s) === RUN TestToParquetValue_TimeValue === RUN TestToParquetValue_TimeValue/Valid_TimeValue_(12:34:56.789) === RUN TestToParquetValue_TimeValue/TimeValue_midnight === RUN TestToParquetValue_TimeValue/TimeValue_end_of_day_(23:59:59.999999) === RUN TestToParquetValue_TimeValue/TimeValue_nil_pointer --- PASS: TestToParquetValue_TimeValue (0.00s) --- PASS: TestToParquetValue_TimeValue/Valid_TimeValue_(12:34:56.789) (0.00s) --- PASS: TestToParquetValue_TimeValue/TimeValue_midnight (0.00s) --- PASS: TestToParquetValue_TimeValue/TimeValue_end_of_day_(23:59:59.999999) (0.00s) --- PASS: TestToParquetValue_TimeValue/TimeValue_nil_pointer (0.00s) === RUN TestToParquetValue_EdgeCases === RUN TestToParquetValue_EdgeCases/Nil_value === RUN TestToParquetValue_EdgeCases/Completely_nil_value === RUN TestToParquetValue_EdgeCases/BytesValue_with_nil_slice --- PASS: TestToParquetValue_EdgeCases (0.00s) --- PASS: TestToParquetValue_EdgeCases/Nil_value (0.00s) --- PASS: TestToParquetValue_EdgeCases/Completely_nil_value (0.00s) --- PASS: TestToParquetValue_EdgeCases/BytesValue_with_nil_slice (0.00s) === RUN TestWriteReadParquet RecordType: fields:{name:"Address" type:{record_type:{fields:{name:"City" type:{scalar_type:STRING}} fields:{name:"Street" type:{scalar_type:STRING}}}}} fields:{name:"Company" type:{scalar_type:STRING}} fields:{name:"CreatedAt" type:{scalar_type:INT64}} fields:{name:"ID" type:{scalar_type:INT64}} fields:{name:"Person" type:{record_type:{fields:{name:"emails" type:{list_type:{element_type:{scalar_type:STRING}}}} fields:{name:"zName" type:{scalar_type:STRING}}}}} ParquetSchema: message example { optional group Address { optional binary City; optional binary Street; } optional binary Company; optional int64 CreatedAt (INT(64,true)); optional int64 ID (INT(64,true)); optional group Person { repeated binary emails; optional binary zName; } } Go Type: struct { Address *struct { City *[]uint8 "parquet:\"City\""; Street *[]uint8 "parquet:\"Street\"" } "parquet:\"Address\""; Company *[]uint8 "parquet:\"Company\""; CreatedAt *int64 "parquet:\"CreatedAt\""; ID *int64 "parquet:\"ID\""; Person *struct { Emails []*[]uint8 "parquet:\"emails\""; ZName *[]uint8 "parquet:\"zName\"" } "parquet:\"Person\"" } Write RecordValue: fields:{key:"Company" value:{string_value:"company_0"}} fields:{key:"CreatedAt" value:{int64_value:2}} fields:{key:"ID" value:{int64_value:1}} fields:{key:"Person" value:{record_value:{fields:{key:"emails" value:{list_value:{values:{string_value:"john_0@a.com"} values:{string_value:"john_0@b.com"} values:{string_value:"john_0@c.com"} values:{string_value:"john_0@d.com"} values:{string_value:"john_0@e.com"}}}} fields:{key:"zName" value:{string_value:"john_0"}}}}} Build Row: [C:0 D:2 R:0 V: C:1 D:2 R:0 V: C:2 D:1 R:0 V:company_0 C:3 D:1 R:0 V:2 C:4 D:1 R:0 V:1 C:5 D:2 R:0 V:john_0@a.com C:5 D:2 R:1 V:john_0@b.com C:5 D:2 R:1 V:john_0@c.com C:5 D:2 R:1 V:john_0@d.com C:5 D:2 R:1 V:john_0@e.com C:6 D:2 R:0 V:john_0] Write RecordValue: fields:{key:"Company" value:{string_value:"company_1"}} fields:{key:"CreatedAt" value:{int64_value:4}} fields:{key:"ID" value:{int64_value:2}} fields:{key:"Person" value:{record_value:{fields:{key:"emails" value:{list_value:{values:{string_value:"john_1@a.com"} values:{string_value:"john_1@b.com"} values:{string_value:"john_1@c.com"} values:{string_value:"john_1@d.com"} values:{string_value:"john_1@e.com"}}}} fields:{key:"zName" value:{string_value:"john_1"}}}}} Build Row: [C:0 D:2 R:0 V: C:1 D:2 R:0 V: C:2 D:1 R:0 V:company_1 C:3 D:1 R:0 V:4 C:4 D:1 R:0 V:2 C:5 D:2 R:0 V:john_1@a.com C:5 D:2 R:1 V:john_1@b.com C:5 D:2 R:1 V:john_1@c.com C:5 D:2 R:1 V:john_1@d.com C:5 D:2 R:1 V:john_1@e.com C:6 D:2 R:0 V:john_1] Write RecordValue: fields:{key:"Company" value:{string_value:"company_2"}} fields:{key:"CreatedAt" value:{int64_value:6}} fields:{key:"ID" value:{int64_value:3}} fields:{key:"Person" value:{record_value:{fields:{key:"emails" value:{list_value:{values:{string_value:"john_2@a.com"} values:{string_value:"john_2@b.com"} values:{string_value:"john_2@c.com"} values:{string_value:"john_2@d.com"} values:{string_value:"john_2@e.com"}}}} fields:{key:"zName" value:{string_value:"john_2"}}}}} Build Row: [C:0 D:2 R:0 V: C:1 D:2 R:0 V: C:2 D:1 R:0 V:company_2 C:3 D:1 R:0 V:6 C:4 D:1 R:0 V:3 C:5 D:2 R:0 V:john_2@a.com C:5 D:2 R:1 V:john_2@b.com C:5 D:2 R:1 V:john_2@c.com C:5 D:2 R:1 V:john_2@d.com C:5 D:2 R:1 V:john_2@e.com C:6 D:2 R:0 V:john_2] Read RecordValue: fields:{key:"Address" value:{record_value:{fields:{key:"City" value:{string_value:""}} fields:{key:"Street" value:{string_value:""}}}}} fields:{key:"Company" value:{string_value:"company_0"}} fields:{key:"CreatedAt" value:{int64_value:2}} fields:{key:"ID" value:{int64_value:1}} fields:{key:"Person" value:{record_value:{fields:{key:"emails" value:{list_value:{values:{string_value:"john_0@a.com"} values:{string_value:"john_0@b.com"} values:{string_value:"john_0@c.com"} values:{string_value:"john_0@d.com"} values:{string_value:"john_0@e.com"}}}} fields:{key:"zName" value:{string_value:"john_0"}}}}} Read RecordValue: fields:{key:"Address" value:{record_value:{fields:{key:"City" value:{string_value:""}} fields:{key:"Street" value:{string_value:""}}}}} fields:{key:"Company" value:{string_value:"company_1"}} fields:{key:"CreatedAt" value:{int64_value:4}} fields:{key:"ID" value:{int64_value:2}} fields:{key:"Person" value:{record_value:{fields:{key:"emails" value:{list_value:{values:{string_value:"john_1@a.com"} values:{string_value:"john_1@b.com"} values:{string_value:"john_1@c.com"} values:{string_value:"john_1@d.com"} values:{string_value:"john_1@e.com"}}}} fields:{key:"zName" value:{string_value:"john_1"}}}}} Read RecordValue: fields:{key:"Address" value:{record_value:{fields:{key:"City" value:{string_value:""}} fields:{key:"Street" value:{string_value:""}}}}} fields:{key:"Company" value:{string_value:"company_2"}} fields:{key:"CreatedAt" value:{int64_value:6}} fields:{key:"ID" value:{int64_value:3}} fields:{key:"Person" value:{record_value:{fields:{key:"emails" value:{list_value:{values:{string_value:"john_2@a.com"} values:{string_value:"john_2@b.com"} values:{string_value:"john_2@c.com"} values:{string_value:"john_2@d.com"} values:{string_value:"john_2@e.com"}}}} fields:{key:"zName" value:{string_value:"john_2"}}}}} total: 3 --- PASS: TestWriteReadParquet (0.02s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/schema 0.051s ? github.com/seaweedfs/seaweedfs/weed/mq/segment [no test files] === RUN TestAddConsumerInstance &{isAssign:true partition:{RangeStart:1 RangeStop:2 RingSize:3 UnixTimeNs:0} consumer:first ts:{wall:14019638246358988299 ext:529675052 loc:0x19c2a40}} &{isAssign:true partition:{RangeStart:2 RangeStop:3 RingSize:3 UnixTimeNs:0} consumer:first ts:{wall:14019638246358995312 ext:529682065 loc:0x19c2a40}} --- PASS: TestAddConsumerInstance (1.00s) === RUN TestMultipleConsumerInstances &{isAssign:true partition:{RangeStart:0 RangeStop:1 RingSize:3 UnixTimeNs:0} consumer:first ts:{wall:14019638247436056827 ext:1533001746 loc:0x19c2a40}} &{isAssign:true partition:{RangeStart:1 RangeStop:2 RingSize:3 UnixTimeNs:0} consumer:second ts:{wall:14019638247436086433 ext:1533031362 loc:0x19c2a40}} &{isAssign:true partition:{RangeStart:2 RangeStop:3 RingSize:3 UnixTimeNs:0} consumer:third ts:{wall:14019638247436088016 ext:1533032945 loc:0x19c2a40}} --- PASS: TestMultipleConsumerInstances (1.00s) === RUN TestConfirmAdjustment &{isAssign:true partition:{RangeStart:0 RangeStop:1 RingSize:3 UnixTimeNs:0} consumer:first ts:{wall:14019638248513136332 ext:2536339437 loc:0x19c2a40}} &{isAssign:true partition:{RangeStart:1 RangeStop:2 RingSize:3 UnixTimeNs:0} consumer:second ts:{wall:14019638248513140960 ext:2536344065 loc:0x19c2a40}} &{isAssign:true partition:{RangeStart:2 RangeStop:3 RingSize:3 UnixTimeNs:0} consumer:third ts:{wall:14019638248513142483 ext:2536345588 loc:0x19c2a40}} &{isAssign:true partition:{RangeStart:2 RangeStop:3 RingSize:3 UnixTimeNs:0} consumer:first ts:{wall:14019638250660987375 ext:4536706832 loc:0x19c2a40}} --- PASS: TestConfirmAdjustment (5.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/sub_coordinator 7.052s === RUN TestOffsetBasedSubscribe_AllDataInMemory === RUN TestOffsetBasedSubscribe_AllDataInMemory/ReadFromOffset0 === RUN TestOffsetBasedSubscribe_AllDataInMemory/ReadFromOffset2 --- PASS: TestOffsetBasedSubscribe_AllDataInMemory (0.00s) --- PASS: TestOffsetBasedSubscribe_AllDataInMemory/ReadFromOffset0 (0.00s) --- PASS: TestOffsetBasedSubscribe_AllDataInMemory/ReadFromOffset2 (0.00s) === RUN TestOffsetBasedSubscribe_DataOnDisk === RUN TestOffsetBasedSubscribe_DataOnDisk/ReadFromOffset0_OnDisk === RUN TestOffsetBasedSubscribe_DataOnDisk/ReadFromOffset5_OnDisk === RUN TestOffsetBasedSubscribe_DataOnDisk/ReadFromOffset11_InMemory --- PASS: TestOffsetBasedSubscribe_DataOnDisk (0.00s) --- PASS: TestOffsetBasedSubscribe_DataOnDisk/ReadFromOffset0_OnDisk (0.00s) --- PASS: TestOffsetBasedSubscribe_DataOnDisk/ReadFromOffset5_OnDisk (0.00s) --- PASS: TestOffsetBasedSubscribe_DataOnDisk/ReadFromOffset11_InMemory (0.00s) === RUN TestTimestampBasedSubscribe === RUN TestTimestampBasedSubscribe/ReadFromBeginning === RUN TestTimestampBasedSubscribe/ReadFromMiddleTimestamp --- PASS: TestTimestampBasedSubscribe (0.00s) --- PASS: TestTimestampBasedSubscribe/ReadFromBeginning (0.00s) --- PASS: TestTimestampBasedSubscribe/ReadFromMiddleTimestamp (0.00s) === RUN TestConcurrentSubscribers --- PASS: TestConcurrentSubscribers (0.00s) === RUN TestResumeFromDiskError --- PASS: TestResumeFromDiskError (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/mq/topic 0.055s ? github.com/seaweedfs/seaweedfs/weed/notification [no test files] ? github.com/seaweedfs/seaweedfs/weed/notification/aws_sqs [no test files] ? github.com/seaweedfs/seaweedfs/weed/notification/gocdk_pub_sub [no test files] ? github.com/seaweedfs/seaweedfs/weed/notification/google_pub_sub [no test files] === RUN TestConfigureSASLTLSRejectsPartialMTLSConfig === RUN TestConfigureSASLTLSRejectsPartialMTLSConfig/missing_key === RUN TestConfigureSASLTLSRejectsPartialMTLSConfig/missing_cert --- PASS: TestConfigureSASLTLSRejectsPartialMTLSConfig (0.00s) --- PASS: TestConfigureSASLTLSRejectsPartialMTLSConfig/missing_key (0.00s) --- PASS: TestConfigureSASLTLSRejectsPartialMTLSConfig/missing_cert (0.00s) === RUN TestConfigureSASLTLSConfiguresSCRAMSHA256 --- PASS: TestConfigureSASLTLSConfiguresSCRAMSHA256 (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/notification/kafka 0.019s ? github.com/seaweedfs/seaweedfs/weed/notification/log [no test files] === RUN TestFilterEventTypes === RUN TestFilterEventTypes/create_event_-_allowed === RUN TestFilterEventTypes/create_event_-_not_allowed === RUN TestFilterEventTypes/delete_event_-_allowed === RUN TestFilterEventTypes/update_event_-_allowed === RUN TestFilterEventTypes/rename_event_-_allowed === RUN TestFilterEventTypes/rename_event_same_name_different_parent_-_allowed === RUN TestFilterEventTypes/rename_event_-_not_allowed === RUN TestFilterEventTypes/all_events_allowed_when_empty --- PASS: TestFilterEventTypes (0.00s) --- PASS: TestFilterEventTypes/create_event_-_allowed (0.00s) --- PASS: TestFilterEventTypes/create_event_-_not_allowed (0.00s) --- PASS: TestFilterEventTypes/delete_event_-_allowed (0.00s) --- PASS: TestFilterEventTypes/update_event_-_allowed (0.00s) --- PASS: TestFilterEventTypes/rename_event_-_allowed (0.00s) --- PASS: TestFilterEventTypes/rename_event_same_name_different_parent_-_allowed (0.00s) --- PASS: TestFilterEventTypes/rename_event_-_not_allowed (0.00s) --- PASS: TestFilterEventTypes/all_events_allowed_when_empty (0.00s) === RUN TestFilterPathPrefixes === RUN TestFilterPathPrefixes/matches_single_prefix === RUN TestFilterPathPrefixes/matches_one_of_multiple_prefixes === RUN TestFilterPathPrefixes/no_match === RUN TestFilterPathPrefixes/empty_prefixes_allows_all === RUN TestFilterPathPrefixes/exact_prefix_match === RUN TestFilterPathPrefixes/partial_match_not_allowed === RUN TestFilterPathPrefixes/rename_matches_destination_prefix --- PASS: TestFilterPathPrefixes (0.00s) --- PASS: TestFilterPathPrefixes/matches_single_prefix (0.00s) --- PASS: TestFilterPathPrefixes/matches_one_of_multiple_prefixes (0.00s) --- PASS: TestFilterPathPrefixes/no_match (0.00s) --- PASS: TestFilterPathPrefixes/empty_prefixes_allows_all (0.00s) --- PASS: TestFilterPathPrefixes/exact_prefix_match (0.00s) --- PASS: TestFilterPathPrefixes/partial_match_not_allowed (0.00s) --- PASS: TestFilterPathPrefixes/rename_matches_destination_prefix (0.00s) === RUN TestFilterCombined === RUN TestFilterCombined/allowed_event_and_path === RUN TestFilterCombined/allowed_event_but_wrong_path === RUN TestFilterCombined/wrong_event_but_allowed_path === RUN TestFilterCombined/wrong_event_and_wrong_path --- PASS: TestFilterCombined (0.00s) --- PASS: TestFilterCombined/allowed_event_and_path (0.00s) --- PASS: TestFilterCombined/allowed_event_but_wrong_path (0.00s) --- PASS: TestFilterCombined/wrong_event_but_allowed_path (0.00s) --- PASS: TestFilterCombined/wrong_event_and_wrong_path (0.00s) === RUN TestHttpClientSendMessage --- PASS: TestHttpClientSendMessage (0.00s) === RUN TestHttpClientSendMessageWithoutToken --- PASS: TestHttpClientSendMessageWithoutToken (0.00s) === RUN TestHttpClientSendMessageServerError --- PASS: TestHttpClientSendMessageServerError (0.00s) === RUN TestHttpClientSendMessageNetworkError --- PASS: TestHttpClientSendMessageNetworkError (0.00s) === RUN TestHttpClientFollowsRedirectAsPost --- PASS: TestHttpClientFollowsRedirectAsPost (0.00s) === RUN TestHttpClientUsesCachedRedirect --- PASS: TestHttpClientUsesCachedRedirect (0.00s) === RUN TestHttpClientPreservesPostMethod --- PASS: TestHttpClientPreservesPostMethod (0.00s) === RUN TestHttpClientInvalidatesCacheOnError --- PASS: TestHttpClientInvalidatesCacheOnError (0.00s) === RUN TestHttpClientInvalidatesCacheOnNetworkError --- PASS: TestHttpClientInvalidatesCacheOnNetworkError (0.00s) === RUN TestConfigValidation === RUN TestConfigValidation/valid_config === RUN TestConfigValidation/empty_endpoint === RUN TestConfigValidation/invalid_URL === RUN TestConfigValidation/timeout_too_large === RUN TestConfigValidation/too_many_retries === RUN TestConfigValidation/too_many_workers === RUN TestConfigValidation/buffer_too_large --- PASS: TestConfigValidation (0.00s) --- PASS: TestConfigValidation/valid_config (0.00s) --- PASS: TestConfigValidation/empty_endpoint (0.00s) --- PASS: TestConfigValidation/invalid_URL (0.00s) --- PASS: TestConfigValidation/timeout_too_large (0.00s) --- PASS: TestConfigValidation/too_many_retries (0.00s) --- PASS: TestConfigValidation/too_many_workers (0.00s) --- PASS: TestConfigValidation/buffer_too_large (0.00s) === RUN TestWebhookMessageSerialization --- PASS: TestWebhookMessageSerialization (0.00s) === RUN TestQueueInitialize [watermill] 2026/07/20 21:29:54.366130 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.366543 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:54.366576 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.366650 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.366659 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:54.366669 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:54.366675 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:54.366680 router.go:419: level=INFO msg="All messages processed" I0720 21:29:54.366687 webhook_queue.go:170 webhook pubsub worker stopped --- PASS: TestQueueInitialize (0.10s) === RUN TestQueueSendMessage [watermill] 2026/07/20 21:29:54.471982 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.472025 pubsub.go:160: level=INFO msg="No subscribers to send message" message_uuid=b9bf53f0-9ab9-4635-92ba-c44964c02bc8 pubsub_uuid=VR4CiiZiBLuucf3DprBc9R topic=webhook_topic [watermill] 2026/07/20 21:29:54.472056 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:54.472124 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.472143 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.472150 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:54.472164 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:54.472170 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:54.472175 router.go:419: level=INFO msg="All messages processed" I0720 21:29:54.472181 webhook_queue.go:170 webhook pubsub worker stopped --- PASS: TestQueueSendMessage (0.10s) === RUN TestQueueHandleWebhook --- PASS: TestQueueHandleWebhook (0.00s) === RUN TestQueueEndToEnd [watermill] 2026/07/20 21:29:54.574314 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.574359 pubsub.go:160: level=INFO msg="No subscribers to send message" message_uuid=c7d6e341-c929-40cf-a736-16e6bb907574 pubsub_uuid=7MJ9BGgseJBZKqemVzoBJf topic=webhook_topic [watermill] 2026/07/20 21:29:54.574394 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:54.574417 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.574427 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.574434 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:54.574443 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:54.574448 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:54.574451 router.go:419: level=INFO msg="All messages processed" I0720 21:29:54.574457 webhook_queue.go:170 webhook pubsub worker stopped --- PASS: TestQueueEndToEnd (0.10s) === RUN TestQueueRetryMechanism [watermill] 2026/07/20 21:29:54.676691 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.676769 pubsub.go:160: level=INFO msg="No subscribers to send message" message_uuid=65ae1ace-064e-4692-b443-d746ea44ff34 pubsub_uuid=SgSi2AfqK82XGfu7nG7XTb topic=webhook_topic [watermill] 2026/07/20 21:29:54.676801 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:54.676819 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.676833 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.676840 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:54.676851 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:54.676856 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:54.676861 router.go:419: level=INFO msg="All messages processed" I0720 21:29:54.676867 webhook_queue.go:170 webhook pubsub worker stopped --- PASS: TestQueueRetryMechanism (0.10s) === RUN TestQueueNoDuplicateWebhooks === RUN TestQueueNoDuplicateWebhooks/1_worker [watermill] 2026/07/20 21:29:54.777649 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:54.777717 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:54.777740 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:55.483966 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:55.484023 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:55.484106 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:55.484123 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:55.484129 router.go:419: level=INFO msg="All messages processed" I0720 21:29:55.484156 webhook_queue.go:170 webhook pubsub worker stopped === RUN TestQueueNoDuplicateWebhooks/5_workers [watermill] 2026/07/20 21:29:55.587614 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:55.587707 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:55.587727 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:56.290665 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:56.290707 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:56.290727 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:56.290735 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:56.290739 router.go:419: level=INFO msg="All messages processed" I0720 21:29:56.290746 webhook_queue.go:170 webhook pubsub worker stopped === RUN TestQueueNoDuplicateWebhooks/10_workers [watermill] 2026/07/20 21:29:56.394357 router.go:280: level=INFO msg="Adding handler" handler_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:56.394416 router.go:434: level=INFO msg="Running router handlers" count=1 [watermill] 2026/07/20 21:29:56.394442 router.go:648: level=INFO msg="Starting handler" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:57.100641 router.go:483: level=INFO msg="Subscriber stopped" subscriber_name=webhook_handler topic=webhook_topic [watermill] 2026/07/20 21:29:57.100677 router.go:580: level=INFO msg="Closing router" [watermill] 2026/07/20 21:29:57.100693 router.go:413: level=INFO msg="Waiting for messages" timeout=1m0s [watermill] 2026/07/20 21:29:57.100700 router.go:591: level=INFO msg="Router closed" [watermill] 2026/07/20 21:29:57.100704 router.go:419: level=INFO msg="All messages processed" I0720 21:29:57.100710 webhook_queue.go:170 webhook pubsub worker stopped --- PASS: TestQueueNoDuplicateWebhooks (2.43s) --- PASS: TestQueueNoDuplicateWebhooks/1_worker (0.81s) --- PASS: TestQueueNoDuplicateWebhooks/5_workers (0.81s) --- PASS: TestQueueNoDuplicateWebhooks/10_workers (0.81s) === RUN TestQueueSendMessageWithFilter === RUN TestQueueSendMessageWithFilter/allowed_event_type === RUN TestQueueSendMessageWithFilter/filtered_event_type === RUN TestQueueSendMessageWithFilter/allowed_path_prefix === RUN TestQueueSendMessageWithFilter/filtered_path_prefix === RUN TestQueueSendMessageWithFilter/combined_filters_-_both_pass === RUN TestQueueSendMessageWithFilter/combined_filters_-_event_fails === RUN TestQueueSendMessageWithFilter/combined_filters_-_path_fails --- PASS: TestQueueSendMessageWithFilter (0.00s) --- PASS: TestQueueSendMessageWithFilter/allowed_event_type (0.00s) --- PASS: TestQueueSendMessageWithFilter/filtered_event_type (0.00s) --- PASS: TestQueueSendMessageWithFilter/allowed_path_prefix (0.00s) --- PASS: TestQueueSendMessageWithFilter/filtered_path_prefix (0.00s) --- PASS: TestQueueSendMessageWithFilter/combined_filters_-_both_pass (0.00s) --- PASS: TestQueueSendMessageWithFilter/combined_filters_-_event_fails (0.00s) --- PASS: TestQueueSendMessageWithFilter/combined_filters_-_path_fails (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/notification/webhook 2.894s === RUN TestAssignRetriesOnUnavailable --- PASS: TestAssignRetriesOnUnavailable (4.77s) === RUN TestAssignStopsOnContextCancel --- PASS: TestAssignStopsOnContextCancel (2.01s) === RUN TestLookupVolumeIdCacheInvalidation --- PASS: TestLookupVolumeIdCacheInvalidation (0.00s) === RUN TestLookupRaftLeaderMasterFromFollower --- PASS: TestLookupRaftLeaderMasterFromFollower (0.00s) === RUN TestLookupRaftLeaderMasterTriesNextPeer --- PASS: TestLookupRaftLeaderMasterTriesNextPeer (0.00s) === RUN TestLookupRaftLeaderMasterNoPeers --- PASS: TestLookupRaftLeaderMasterNoPeers (0.00s) === RUN TestLookupRaftLeaderMasterCanceledContext --- PASS: TestLookupRaftLeaderMasterCanceledContext (0.00s) === RUN TestCaching vid 123 locations = [{a.com:8080 0}] --- PASS: TestCaching (2.01s) === RUN TestCachingDelete --- PASS: TestCachingDelete (0.00s) === RUN TestCachingLargeVolumeId --- PASS: TestCachingLargeVolumeId (0.00s) === RUN TestCreateNeedleFromRequest needle: 0f084d17353afda0 Size:0, DataSize:0, Name:t.txt, Mime:text/plain; charset=utf-8 Compressed:true, originalSize: 1422 W0720 21:30:03.159632 upload_content.go:334 uploading 0 to http://localhost:8080/389,0f084d17353afda0: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF needle: 0f084d17353afda0 Size:0, DataSize:0, Name:t.txt, Mime:text/plain; charset=utf-8 Compressed:true, originalSize: 1422 W0720 21:30:03.637570 upload_content.go:334 uploading 1 to http://localhost:8080/389,0f084d17353afda0: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF needle: 0f084d17353afda0 Size:0, DataSize:0, Name:t.txt, Mime:text/plain; charset=utf-8 Compressed:true, originalSize: 1422 W0720 21:30:04.351575 upload_content.go:334 uploading 2 to http://localhost:8080/389,0f084d17353afda0: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF err: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF uploadResult: needle: 0f084d17353afda0 Size:0, DataSize:0, Name:t.txt, Mime:text/plain Compressed:true, dataSize:803 originalSize:1422 W0720 21:30:04.351815 upload_content.go:334 uploading 0 to http://localhost:8080/389,0f084d17353afda0: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF needle: 0f084d17353afda0 Size:0, DataSize:0, Name:t.txt, Mime:text/plain Compressed:true, dataSize:803 originalSize:1422 W0720 21:30:04.827432 upload_content.go:334 uploading 1 to http://localhost:8080/389,0f084d17353afda0: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF needle: 0f084d17353afda0 Size:0, DataSize:0, Name:t.txt, Mime:text/plain Compressed:true, dataSize:803 originalSize:1422 W0720 21:30:05.540719 upload_content.go:334 uploading 2 to http://localhost:8080/389,0f084d17353afda0: upload t.txt 803 bytes to http://localhost:8080/389,0f084d17353afda0: EOF --- PASS: TestCreateNeedleFromRequest (2.40s) === RUN TestUploadReaderInChunksReturnsPartialResultsOnError E0720 21:30:05.540882 upload_chunked.go:276 chunked upload failed: assign volume: simulated volume assignment failure (returning 1 partial chunks for cleanup) upload_chunked_test.go:72: ✓ Got expected error: assign volume: simulated volume assignment failure upload_chunked_test.go:78: ✓ Result is not nil (partial results returned) upload_chunked_test.go:84: ✓ Result contains 1 chunks (may be 0 if all assignments failed) upload_chunked_test.go:90: ✓ Md5Hash is available for partial data upload_chunked_test.go:97: ✓ TotalSize = 32768 bytes read before failure --- PASS: TestUploadReaderInChunksReturnsPartialResultsOnError (0.00s) === RUN TestUploadReaderInChunksSuccessPath upload_chunked_test.go:140: ✓ Upload completed without error upload_chunked_test.go:146: ✓ Result is not nil upload_chunked_test.go:152: ✓ Result contains 1 file chunk(s) upload_chunked_test.go:159: ✓ TotalSize=15 matches input data upload_chunked_test.go:166: ✓ Md5Hash is available upload_chunked_test.go:181: ✓ Chunk properties validated: FileId=test-fid,1234, Offset=0, Size=15 --- PASS: TestUploadReaderInChunksSuccessPath (0.00s) === RUN TestUploadReaderInChunksContextCancellation E0720 21:30:05.541165 upload_chunked.go:276 chunked upload failed: context canceled (returning 0 partial chunks for cleanup) upload_chunked_test.go:235: ✓ Got partial result on cancellation: chunks=0 --- PASS: TestUploadReaderInChunksContextCancellation (0.00s) === RUN TestUploadChunkToHoldersRollsBackOnPartialFailure E0720 21:30:05.542945 upload_content.go:564 unmarshal http://127.0.0.1:43231/3,01abcdef?type=replicate: boom W0720 21:30:05.542966 upload_content.go:334 uploading 0 to http://127.0.0.1:43231/3,01abcdef?type=replicate: unmarshal http://127.0.0.1:43231/3,01abcdef?type=replicate: invalid character 'b' looking for beginning of value E0720 21:30:06.017672 upload_content.go:564 unmarshal http://127.0.0.1:43231/3,01abcdef?type=replicate: boom W0720 21:30:06.017690 upload_content.go:334 uploading 1 to http://127.0.0.1:43231/3,01abcdef?type=replicate: unmarshal http://127.0.0.1:43231/3,01abcdef?type=replicate: invalid character 'b' looking for beginning of value E0720 21:30:06.731180 upload_content.go:564 unmarshal http://127.0.0.1:43231/3,01abcdef?type=replicate: boom W0720 21:30:06.731193 upload_content.go:334 uploading 2 to http://127.0.0.1:43231/3,01abcdef?type=replicate: unmarshal http://127.0.0.1:43231/3,01abcdef?type=replicate: invalid character 'b' looking for beginning of value --- PASS: TestUploadChunkToHoldersRollsBackOnPartialFailure (1.19s) === RUN TestUploadReaderInChunksReaderFailure E0720 21:30:06.731566 upload_chunked.go:276 chunked upload failed: read chunk at offset 8192 (got 1808 bytes): simulated read failure (returning 1 partial chunks for cleanup) upload_chunked_test.go:372: ✓ Got partial result on read failure: chunks=1, totalSize=8192 --- PASS: TestUploadReaderInChunksReaderFailure (0.00s) === RUN TestUploadReaderInChunksTagsTruncatedBody E0720 21:30:06.731612 upload_chunked.go:276 chunked upload failed: truncated request body: read chunk at offset 8192 (got 1808 bytes): unexpected EOF (returning 1 partial chunks for cleanup) --- PASS: TestUploadReaderInChunksTagsTruncatedBody (0.00s) === RUN TestShouldReassignUpload === RUN TestShouldReassignUpload/nil === RUN TestShouldReassignUpload/no_response_(transport) === RUN TestShouldReassignUpload/500_replica_write_failed === RUN TestShouldReassignUpload/503_service_unavailable === RUN TestShouldReassignUpload/400_bad_request === RUN TestShouldReassignUpload/401_unauthorized === RUN TestShouldReassignUpload/assign_rpc_transport === RUN TestShouldReassignUpload/other_permanent --- PASS: TestShouldReassignUpload (0.00s) --- PASS: TestShouldReassignUpload/nil (0.00s) --- PASS: TestShouldReassignUpload/no_response_(transport) (0.00s) --- PASS: TestShouldReassignUpload/500_replica_write_failed (0.00s) --- PASS: TestShouldReassignUpload/503_service_unavailable (0.00s) --- PASS: TestShouldReassignUpload/400_bad_request (0.00s) --- PASS: TestShouldReassignUpload/401_unauthorized (0.00s) --- PASS: TestShouldReassignUpload/assign_rpc_transport (0.00s) --- PASS: TestShouldReassignUpload/other_permanent (0.00s) === RUN TestUploadWithRetryDataReassignsOnVolumeSizeExceeded W0720 21:30:06.731844 upload_content.go:334 uploading 0 to http://volume-a/1,first: unmarshalled error http://volume-a/1,first: failed to write to local disk: append to volume 1 size 0 actualSize 0: Volume Size 33555976 Exceeded 33554432 W0720 21:30:07.207327 upload_content.go:334 uploading 1 to http://volume-a/1,first: unmarshalled error http://volume-a/1,first: failed to write to local disk: append to volume 1 size 0 actualSize 0: Volume Size 33555976 Exceeded 33554432 W0720 21:30:07.920677 upload_content.go:334 uploading 2 to http://volume-a/1,first: unmarshalled error http://volume-a/1,first: failed to write to local disk: append to volume 1 size 0 actualSize 0: Volume Size 33555976 Exceeded 33554432 I0720 21:30:07.920696 retry.go:76 retry uploadWithRetry: err: unmarshalled error http://volume-a/1,first: failed to write to local disk: append to volume 1 size 0 actualSize 0: Volume Size 33555976 Exceeded 33554432 I0720 21:30:08.923980 retry.go:70 retry uploadWithRetry successfully --- PASS: TestUploadWithRetryDataReassignsOnVolumeSizeExceeded (2.19s) === RUN TestUploadWithRetryDataReassignsOnReplicaWriteFailure W0720 21:30:08.924055 upload_content.go:334 uploading 0 to http://volume-a/1,first: unmarshalled error http://volume-a/1,first: failed to write to replicas for volume 1: [volume-down:8080]: dial tcp: lookup volume-down: no such host W0720 21:30:09.398430 upload_content.go:334 uploading 1 to http://volume-a/1,first: unmarshalled error http://volume-a/1,first: failed to write to replicas for volume 1: [volume-down:8080]: dial tcp: lookup volume-down: no such host W0720 21:30:10.110685 upload_content.go:334 uploading 2 to http://volume-a/1,first: unmarshalled error http://volume-a/1,first: failed to write to replicas for volume 1: [volume-down:8080]: dial tcp: lookup volume-down: no such host I0720 21:30:10.110708 retry.go:76 retry uploadWithRetry: err: unmarshalled error http://volume-a/1,first: failed to write to replicas for volume 1: [volume-down:8080]: dial tcp: lookup volume-down: no such host I0720 21:30:11.114017 retry.go:70 retry uploadWithRetry successfully --- PASS: TestUploadWithRetryDataReassignsOnReplicaWriteFailure (2.19s) === RUN TestUploadWithRetryBoundsAssignVolume I0720 21:30:11.314998 upload_content.go:266 assign volume failure count:1 expected_data_size:3: context deadline exceeded --- PASS: TestUploadWithRetryBoundsAssignVolume (0.20s) === RUN TestUploadRewindsBodyOnConnectionReset === RUN TestUploadRewindsBodyOnConnectionReset/connection_reset_by_peer === RUN TestUploadRewindsBodyOnConnectionReset/use_of_closed_network_connection --- PASS: TestUploadRewindsBodyOnConnectionReset (0.00s) --- PASS: TestUploadRewindsBodyOnConnectionReset/connection_reset_by_peer (0.00s) --- PASS: TestUploadRewindsBodyOnConnectionReset/use_of_closed_network_connection (0.00s) === RUN TestReplicationUploadPreservesUncompressedNeedle === RUN TestReplicationUploadPreservesUncompressedNeedle/unknown_binary_sampling === RUN TestReplicationUploadPreservesUncompressedNeedle/receiver_MIME_detection --- PASS: TestReplicationUploadPreservesUncompressedNeedle (0.00s) --- PASS: TestReplicationUploadPreservesUncompressedNeedle/unknown_binary_sampling (0.00s) --- PASS: TestReplicationUploadPreservesUncompressedNeedle/receiver_MIME_detection (0.00s) === RUN TestUploadToDeadReplicaRetriesThreeTimes W0720 21:30:11.416974 upload_content.go:334 uploading 0 to http://dead-replica:8080/3,01: upload test.bin 5 bytes to http://dead-replica:8080/3,01: dial tcp dead-replica:8080: i/o timeout W0720 21:30:11.994867 upload_content.go:334 uploading 1 to http://dead-replica:8080/3,01: upload test.bin 5 bytes to http://dead-replica:8080/3,01: dial tcp dead-replica:8080: i/o timeout W0720 21:30:12.807973 upload_content.go:334 uploading 2 to http://dead-replica:8080/3,01: upload test.bin 5 bytes to http://dead-replica:8080/3,01: dial tcp dead-replica:8080: i/o timeout upload_content_test.go:410: dead replica stalled the caller for 1.491710145s across 3 attempts --- PASS: TestUploadToDeadReplicaRetriesThreeTimes (1.49s) === RUN TestUploadToDeadReplicaSingleAttempt W0720 21:30:12.910645 upload_content.go:334 uploading 0 to http://dead-replica:8080/3,01: upload test.bin 5 bytes to http://dead-replica:8080/3,01: dial tcp dead-replica:8080: i/o timeout --- PASS: TestUploadToDeadReplicaSingleAttempt (0.10s) === RUN TestUploadRetryStopsOnContextCancel --- PASS: TestUploadRetryStopsOnContextCancel (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/operation 18.592s === RUN TestReadLogFileRefsMergeOrder --- PASS: TestReadLogFileRefsMergeOrder (0.02s) === RUN TestReadLogFileRefsPathFilter filer_pb_direct_read_test.go:209: Total events: 200, matching /data/filer00/: 100 --- PASS: TestReadLogFileRefsPathFilter (0.00s) === RUN TestDirectReadVsServerSideThroughput === RUN TestDirectReadVsServerSideThroughput/server_side_sequential filer_pb_direct_read_test.go:255: server-side: 6300 events 11.419s 552 events/sec (21 files sequential + 20µs send/event) === RUN TestDirectReadVsServerSideThroughput/client_direct_read_parallel_streaming filer_pb_direct_read_test.go:276: direct-read: 6300 events 40ms 155780 events/sec (3 filers parallel + streaming, no gRPC) === NAME TestDirectReadVsServerSideThroughput filer_pb_direct_read_test.go:281: Speedup: 282.4x (parallel + streaming + no gRPC vs server-side sequential) --- PASS: TestDirectReadVsServerSideThroughput (11.48s) --- PASS: TestDirectReadVsServerSideThroughput/server_side_sequential (11.42s) --- PASS: TestDirectReadVsServerSideThroughput/client_direct_read_parallel_streaming (0.04s) === RUN TestReadLogFileRefsMultiFilerGenuineErrorAborts --- PASS: TestReadLogFileRefsMultiFilerGenuineErrorAborts (0.00s) === RUN TestReadLogFileRefsMultiFilerNotFoundSkips I0720 21:30:05.880554 filer_pb_direct_read.go:228 skip log file filer=filer01 ts=1784579405880490435 after 0 entries: create reader: volume not found: filer01:1784579405880490435 --- PASS: TestReadLogFileRefsMultiFilerNotFoundSkips (0.00s) === RUN TestReadLogFileRefsSingleFilerOrder --- PASS: TestReadLogFileRefsSingleFilerOrder (0.00s) === RUN TestReadLogFileRefsSingleFilerProcessErrorStops --- PASS: TestReadLogFileRefsSingleFilerProcessErrorStops (0.00s) === RUN TestReadLogFileRefsCorruptSizePrefix --- PASS: TestReadLogFileRefsCorruptSizePrefix (0.00s) === RUN TestReadLogFileRefsAbortDoesNotJoinWedgedReader --- PASS: TestReadLogFileRefsAbortDoesNotJoinWedgedReader (0.00s) === RUN TestReadLogFileRefsSingleFilerNotFoundSkips I0720 21:30:05.881590 filer_pb_direct_read.go:228 skip log file filer=filer00 ts=1784579465881553061 after 0 entries: create reader: volume not found: filer00:1784579465881553061 --- PASS: TestReadLogFileRefsSingleFilerNotFoundSkips (0.00s) === RUN TestProcessOneLogEntrySkipsInvalidUTF8 E0720 21:30:05.881619 filer_pb_direct_read.go:242 unmarshal log entry: string field contains invalid UTF-8 --- PASS: TestProcessOneLogEntrySkipsInvalidUTF8 (0.00s) === RUN TestFilerSyncOffsetStaysFreshOnFilteredMarker filer_pb_tail_test.go:107: gauge timeline: [{src:offset ts:1000000000} {src:heartbeat ts:1784583005882264918} {src:heartbeat ts:1784583006882264918}] --- PASS: TestFilerSyncOffsetStaysFreshOnFilteredMarker (0.00s) === RUN TestFilerSyncBatchedFreshnessSignalDoesNotCrash --- PASS: TestFilerSyncBatchedFreshnessSignalDoesNotCrash (0.00s) === RUN TestShouldInvalidateConnection_MarshalErrorIsPerRequest --- PASS: TestShouldInvalidateConnection_MarshalErrorIsPerRequest (0.00s) === RUN TestShouldInvalidateConnection_CallerContextExpiryIsPerRequest --- PASS: TestShouldInvalidateConnection_CallerContextExpiryIsPerRequest (0.00s) === RUN TestShouldInvalidateConnection_StaleChannelStillInvalidates --- PASS: TestShouldInvalidateConnection_StaleChannelStillInvalidates (0.00s) === RUN TestShouldInvalidateConnection_ResourceExhaustedIsPerRequest --- PASS: TestShouldInvalidateConnection_ResourceExhaustedIsPerRequest (0.00s) === RUN TestShouldInvalidateConnection_GenuineInternalStillInvalidates --- PASS: TestShouldInvalidateConnection_GenuineInternalStillInvalidates (0.00s) === RUN TestShouldInvalidateConnection_TransportErrorsStillInvalidate --- PASS: TestShouldInvalidateConnection_TransportErrorsStillInvalidate (0.00s) === RUN TestIsClientSideMarshalError_RequiresGrpcStatus --- PASS: TestIsClientSideMarshalError_RequiresGrpcStatus (0.00s) === RUN TestResolveLocalGrpcSocket_RemotePortCollision === RUN TestResolveLocalGrpcSocket_RemotePortCollision/local_advertised_host_routes_to_socket === RUN TestResolveLocalGrpcSocket_RemotePortCollision/loopback_v4_routes_to_socket === RUN TestResolveLocalGrpcSocket_RemotePortCollision/localhost_routes_to_socket === RUN TestResolveLocalGrpcSocket_RemotePortCollision/loopback_v6_routes_to_socket === RUN TestResolveLocalGrpcSocket_RemotePortCollision/empty_host_(bare_port)_routes_to_socket === RUN TestResolveLocalGrpcSocket_RemotePortCollision/remote_host_with_same_port_stays_on_TCP === RUN TestResolveLocalGrpcSocket_RemotePortCollision/unrelated_host_with_same_port_stays_on_TCP === RUN TestResolveLocalGrpcSocket_RemotePortCollision/unregistered_port_stays_on_TCP === RUN TestResolveLocalGrpcSocket_RemotePortCollision/malformed_address_stays_on_TCP --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/local_advertised_host_routes_to_socket (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/loopback_v4_routes_to_socket (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/localhost_routes_to_socket (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/loopback_v6_routes_to_socket (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/empty_host_(bare_port)_routes_to_socket (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/remote_host_with_same_port_stays_on_TCP (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/unrelated_host_with_same_port_stays_on_TCP (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/unregistered_port_stays_on_TCP (0.00s) --- PASS: TestResolveLocalGrpcSocket_RemotePortCollision/malformed_address_stays_on_TCP (0.00s) === RUN TestJsonpMarshalUnmarshal marshalled: { "backendType": "aws", "backendId": "", "key": "", "offset": "0", "fileSize": "12", "modifiedTime": "0", "extension": "" } unmarshalled: backend_type:"aws" backend_id:"temp" file_size:12 --- PASS: TestJsonpMarshalUnmarshal (0.00s) === RUN TestServerAddresses_ToAddressMapOrSrv_shouldRemovePrefix --- PASS: TestServerAddresses_ToAddressMapOrSrv_shouldRemovePrefix (0.00s) === RUN TestServerAddresses_ToAddressMapOrSrv_shouldHandleIPPortList --- PASS: TestServerAddresses_ToAddressMapOrSrv_shouldHandleIPPortList (0.00s) === RUN TestServerAddress_ToHost === RUN TestServerAddress_ToHost/hostname_with_port === RUN TestServerAddress_ToHost/IPv4_with_port === RUN TestServerAddress_ToHost/IPv6_with_port === RUN TestServerAddress_ToHost/hostname_without_port === RUN TestServerAddress_ToHost/hostname_with_port.grpcPort === RUN TestServerAddress_ToHost/IPv4_with_port.grpcPort === RUN TestServerAddress_ToHost/IPv6_with_port.grpcPort === RUN TestServerAddress_ToHost/bracketed_IPv6_without_port --- PASS: TestServerAddress_ToHost (0.00s) --- PASS: TestServerAddress_ToHost/hostname_with_port (0.00s) --- PASS: TestServerAddress_ToHost/IPv4_with_port (0.00s) --- PASS: TestServerAddress_ToHost/IPv6_with_port (0.00s) --- PASS: TestServerAddress_ToHost/hostname_without_port (0.00s) --- PASS: TestServerAddress_ToHost/hostname_with_port.grpcPort (0.00s) --- PASS: TestServerAddress_ToHost/IPv4_with_port.grpcPort (0.00s) --- PASS: TestServerAddress_ToHost/IPv6_with_port.grpcPort (0.00s) --- PASS: TestServerAddress_ToHost/bracketed_IPv6_without_port (0.00s) === RUN TestIPv6ServerAddressFormatting === RUN TestIPv6ServerAddressFormatting/unbracketed_IPv6 === RUN TestIPv6ServerAddressFormatting/bracketed_IPv6 --- PASS: TestIPv6ServerAddressFormatting (0.00s) --- PASS: TestIPv6ServerAddressFormatting/unbracketed_IPv6 (0.00s) --- PASS: TestIPv6ServerAddressFormatting/bracketed_IPv6 (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/pb 11.561s === RUN TestReadDirAllEntriesWithSnapshotCarriesSnapshotAcrossPages --- PASS: TestReadDirAllEntriesWithSnapshotCarriesSnapshotAcrossPages (0.01s) === RUN TestReadDirAllEntriesWithSnapshotEmptyDirectory --- PASS: TestReadDirAllEntriesWithSnapshotEmptyDirectory (0.00s) === RUN TestScanMetadataEventSkeletonMatchesFullDecode --- PASS: TestScanMetadataEventSkeletonMatchesFullDecode (0.00s) === RUN TestScanMetadataEventSkeletonFallsBack --- PASS: TestScanMetadataEventSkeletonFallsBack (0.00s) === RUN TestScanMetadataEventSkeletonEmptyPayload --- PASS: TestScanMetadataEventSkeletonEmptyPayload (0.00s) === RUN TestIsDirectoryKeyObject === RUN TestIsDirectoryKeyObject/plain_directory === RUN TestIsDirectoryKeyObject/directory_marker_with_mime === RUN TestIsDirectoryKeyObject/directory_promoted_from_file_keeps_chunks === RUN TestIsDirectoryKeyObject/directory_promoted_from_small_file_keeps_content === RUN TestIsDirectoryKeyObject/directory_promoted_from_remote-tiered_file === RUN TestIsDirectoryKeyObject/directory_with_chunks_and_nil_attributes === RUN TestIsDirectoryKeyObject/regular_file_with_chunks === RUN TestIsDirectoryKeyObject/remote_mount_directory_has_no_remote_size --- PASS: TestIsDirectoryKeyObject (0.00s) --- PASS: TestIsDirectoryKeyObject/plain_directory (0.00s) --- PASS: TestIsDirectoryKeyObject/directory_marker_with_mime (0.00s) --- PASS: TestIsDirectoryKeyObject/directory_promoted_from_file_keeps_chunks (0.00s) --- PASS: TestIsDirectoryKeyObject/directory_promoted_from_small_file_keeps_content (0.00s) --- PASS: TestIsDirectoryKeyObject/directory_promoted_from_remote-tiered_file (0.00s) --- PASS: TestIsDirectoryKeyObject/directory_with_chunks_and_nil_attributes (0.00s) --- PASS: TestIsDirectoryKeyObject/regular_file_with_chunks (0.00s) --- PASS: TestIsDirectoryKeyObject/remote_mount_directory_has_no_remote_size (0.00s) === RUN TestVTWireCompatibility --- PASS: TestVTWireCompatibility (0.00s) === RUN TestUTF8Validation --- PASS: TestUTF8Validation (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/pb/filer_pb 0.063s ? github.com/seaweedfs/seaweedfs/weed/pb/iam_pb [no test files] === RUN TestDiskInfoSplitByPhysicalDisk_includesEmptyDiskFromPhysicalDisks --- PASS: TestDiskInfoSplitByPhysicalDisk_includesEmptyDiskFromPhysicalDisks (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_clampsNegativeFreeOnOverAllocation --- PASS: TestDiskInfoSplitByPhysicalDisk_clampsNegativeFreeOnOverAllocation (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_collapsesOnSingleDisk --- PASS: TestDiskInfoSplitByPhysicalDisk_collapsesOnSingleDisk (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_splitsByVolumeDiskId --- PASS: TestDiskInfoSplitByPhysicalDisk_splitsByVolumeDiskId (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_preservesAggregateCapacityWithRemainder --- PASS: TestDiskInfoSplitByPhysicalDisk_preservesAggregateCapacityWithRemainder (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_countsActiveAndRemoteExactly --- PASS: TestDiskInfoSplitByPhysicalDisk_countsActiveAndRemoteExactly (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_splitsByEcShardDiskId --- PASS: TestDiskInfoSplitByPhysicalDisk_splitsByEcShardDiskId (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_normalizesZeroToOuterDiskId --- PASS: TestDiskInfoSplitByPhysicalDisk_normalizesZeroToOuterDiskId (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_keepsRealDiskZeroWhenMixed --- PASS: TestDiskInfoSplitByPhysicalDisk_keepsRealDiskZeroWhenMixed (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_nilSafe --- PASS: TestDiskInfoSplitByPhysicalDisk_nilSafe (0.00s) === RUN TestDiskInfoSplitByPhysicalDisk_emptyDiskReturnsSelf --- PASS: TestDiskInfoSplitByPhysicalDisk_emptyDiskReturnsSelf (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/pb/master_pb 0.011s ? github.com/seaweedfs/seaweedfs/weed/pb/message_fbs [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/mount_pb [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/mount_peer_pb [no test files] === RUN TestPublishRecordResponseSerialization --- PASS: TestPublishRecordResponseSerialization (0.00s) === RUN TestSubscribeRecordResponseSerialization --- PASS: TestSubscribeRecordResponseSerialization (0.00s) === RUN TestPublishRecordResponseBackwardCompatibility --- PASS: TestPublishRecordResponseBackwardCompatibility (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/pb/mq_agent_pb 0.019s ? github.com/seaweedfs/seaweedfs/weed/pb/mq_pb [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/plugin_pb [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/remote_pb [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/s3_lifecycle_pb [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/s3_pb [no test files] === RUN TestOffsetTypeEnums === RUN TestOffsetTypeEnums/EXACT_OFFSET === RUN TestOffsetTypeEnums/RESET_TO_OFFSET --- PASS: TestOffsetTypeEnums (0.00s) --- PASS: TestOffsetTypeEnums/EXACT_OFFSET (0.00s) --- PASS: TestOffsetTypeEnums/RESET_TO_OFFSET (0.00s) === RUN TestPartitionOffsetSerialization --- PASS: TestPartitionOffsetSerialization (0.00s) === RUN TestPartitionOffsetBackwardCompatibility --- PASS: TestPartitionOffsetBackwardCompatibility (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/pb/schema_pb 0.020s ? github.com/seaweedfs/seaweedfs/weed/pb/volume_server_pb [no test files] ? github.com/seaweedfs/seaweedfs/weed/pb/worker_pb [no test files] === RUN TestAdminScriptFilerAddressKeepsGrpcPortOffConvention --- PASS: TestAdminScriptFilerAddressKeepsGrpcPortOffConvention (0.00s) === RUN TestAdminScriptShellOptionsRequireMasters --- PASS: TestAdminScriptShellOptionsRequireMasters (0.00s) === RUN TestAdminScriptDescriptorDefaults --- PASS: TestAdminScriptDescriptorDefaults (0.00s) === RUN TestAdminScriptDetectSkipsByRunInterval --- PASS: TestAdminScriptDetectSkipsByRunInterval (0.00s) === RUN TestAdminScriptDetectCreatesProposalWhenIntervalElapsed --- PASS: TestAdminScriptDetectCreatesProposalWhenIntervalElapsed (0.00s) === RUN TestBuildVolumeMetricsEmptyFilter --- PASS: TestBuildVolumeMetricsEmptyFilter (0.00s) === RUN TestBuildVolumeMetricsAllCollections --- PASS: TestBuildVolumeMetricsAllCollections (0.00s) === RUN TestBuildVolumeMetricsEachCollection --- PASS: TestBuildVolumeMetricsEachCollection (0.00s) === RUN TestBuildVolumeMetricsRegexFilter --- PASS: TestBuildVolumeMetricsRegexFilter (0.00s) === RUN TestBuildVolumeMetricsInvalidRegex --- PASS: TestBuildVolumeMetricsInvalidRegex (0.00s) === RUN TestWorkerBuildHelloUsesConfiguredConcurrency --- PASS: TestWorkerBuildHelloUsesConfiguredConcurrency (0.00s) === RUN TestWorkerBuildHelloIncludesMultipleCapabilities --- PASS: TestWorkerBuildHelloIncludesMultipleCapabilities (0.00s) === RUN TestWorkerCancelWorkByTargetID --- PASS: TestWorkerCancelWorkByTargetID (0.00s) === RUN TestWorkerHandleCancelRequestAck --- PASS: TestWorkerHandleCancelRequestAck (0.00s) === RUN TestWorkerSchemaRequestRequiresJobTypeWhenMultipleHandlers --- PASS: TestWorkerSchemaRequestRequiresJobTypeWhenMultipleHandlers (0.00s) === RUN TestWorkerHandleDetectionQueuesWhenAtCapacity --- PASS: TestWorkerHandleDetectionQueuesWhenAtCapacity (0.10s) === RUN TestWorkerHeartbeatReflectsActiveDetectionLoad --- PASS: TestWorkerHeartbeatReflectsActiveDetectionLoad (0.00s) === RUN TestWorkerHeartbeatReflectsActiveExecutionLoad --- PASS: TestWorkerHeartbeatReflectsActiveExecutionLoad (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/plugin/worker 0.156s ? github.com/seaweedfs/seaweedfs/weed/plugin/worker/handlers [no test files] === RUN TestAliasTimestampIntegration === RUN TestAliasTimestampIntegration/AliasWithLargeTimestamps === RUN TestAliasTimestampIntegration/AliasWithLargeTimestamps/Timestamp_1 === RUN TestAliasTimestampIntegration/AliasWithLargeTimestamps/Timestamp_2 === RUN TestAliasTimestampIntegration/AliasWithLargeTimestamps/Timestamp_3 === RUN TestAliasTimestampIntegration/AliasWithTimestampRangeQueries === RUN TestAliasTimestampIntegration/AliasWithTimestampPrecisionEdgeCases === RUN TestAliasTimestampIntegration/MultipleAliasesWithTimestamps === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_=_1756947416566456262 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_=_1756947416566456263 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>_1756947416566456261 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>_1756947416566456262 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>=_1756947416566456262 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>=_1756947416566456263 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<_1756947416566456263 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<_1756947416566456262 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<=_1756947416566456262 === RUN TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<=_1756947416566456261 === RUN TestAliasTimestampIntegration/ProductionScenarioReproduction --- PASS: TestAliasTimestampIntegration (0.00s) --- PASS: TestAliasTimestampIntegration/AliasWithLargeTimestamps (0.00s) --- PASS: TestAliasTimestampIntegration/AliasWithLargeTimestamps/Timestamp_1 (0.00s) --- PASS: TestAliasTimestampIntegration/AliasWithLargeTimestamps/Timestamp_2 (0.00s) --- PASS: TestAliasTimestampIntegration/AliasWithLargeTimestamps/Timestamp_3 (0.00s) --- PASS: TestAliasTimestampIntegration/AliasWithTimestampRangeQueries (0.00s) --- PASS: TestAliasTimestampIntegration/AliasWithTimestampPrecisionEdgeCases (0.00s) --- PASS: TestAliasTimestampIntegration/MultipleAliasesWithTimestamps (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_=_1756947416566456262 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_=_1756947416566456263 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>_1756947416566456261 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>_1756947416566456262 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>=_1756947416566456262 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_>=_1756947416566456263 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<_1756947416566456263 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<_1756947416566456262 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<=_1756947416566456262 (0.00s) --- PASS: TestAliasTimestampIntegration/CompatibilityWithExistingTimestampFixes/ts_<=_1756947416566456261 (0.00s) --- PASS: TestAliasTimestampIntegration/ProductionScenarioReproduction (0.00s) === RUN TestArithmeticOperations === RUN TestArithmeticOperations/Add_two_integers === RUN TestArithmeticOperations/Add_integer_and_float === RUN TestArithmeticOperations/Subtract_two_integers === RUN TestArithmeticOperations/Multiply_two_integers === RUN TestArithmeticOperations/Multiply_with_float === RUN TestArithmeticOperations/Divide_two_integers === RUN TestArithmeticOperations/Division_by_zero === RUN TestArithmeticOperations/Modulo_operation === RUN TestArithmeticOperations/Modulo_by_zero === RUN TestArithmeticOperations/Add_string_number_to_integer === RUN TestArithmeticOperations/Invalid_string_conversion === RUN TestArithmeticOperations/Add_boolean_to_integer === RUN TestArithmeticOperations/Add_with_null_left_operand === RUN TestArithmeticOperations/Add_with_null_right_operand --- PASS: TestArithmeticOperations (0.00s) --- PASS: TestArithmeticOperations/Add_two_integers (0.00s) --- PASS: TestArithmeticOperations/Add_integer_and_float (0.00s) --- PASS: TestArithmeticOperations/Subtract_two_integers (0.00s) --- PASS: TestArithmeticOperations/Multiply_two_integers (0.00s) --- PASS: TestArithmeticOperations/Multiply_with_float (0.00s) --- PASS: TestArithmeticOperations/Divide_two_integers (0.00s) --- PASS: TestArithmeticOperations/Division_by_zero (0.00s) --- PASS: TestArithmeticOperations/Modulo_operation (0.00s) --- PASS: TestArithmeticOperations/Modulo_by_zero (0.00s) --- PASS: TestArithmeticOperations/Add_string_number_to_integer (0.00s) --- PASS: TestArithmeticOperations/Invalid_string_conversion (0.00s) --- PASS: TestArithmeticOperations/Add_boolean_to_integer (0.00s) --- PASS: TestArithmeticOperations/Add_with_null_left_operand (0.00s) --- PASS: TestArithmeticOperations/Add_with_null_right_operand (0.00s) === RUN TestIndividualArithmeticFunctions --- PASS: TestIndividualArithmeticFunctions (0.00s) === RUN TestMathematicalFunctions === RUN TestMathematicalFunctions/ROUND_function_tests === RUN TestMathematicalFunctions/ROUND_function_tests/Round_float_to_integer === RUN TestMathematicalFunctions/ROUND_function_tests/Round_integer_stays_integer === RUN TestMathematicalFunctions/ROUND_function_tests/Round_with_precision_2 === RUN TestMathematicalFunctions/ROUND_function_tests/Round_negative_number === RUN TestMathematicalFunctions/ROUND_function_tests/Round_null_value === RUN TestMathematicalFunctions/CEIL_function_tests === RUN TestMathematicalFunctions/CEIL_function_tests/Ceil_positive_decimal === RUN TestMathematicalFunctions/CEIL_function_tests/Ceil_negative_decimal === RUN TestMathematicalFunctions/CEIL_function_tests/Ceil_integer === RUN TestMathematicalFunctions/CEIL_function_tests/Ceil_null_value === RUN TestMathematicalFunctions/FLOOR_function_tests === RUN TestMathematicalFunctions/FLOOR_function_tests/Floor_positive_decimal === RUN TestMathematicalFunctions/FLOOR_function_tests/Floor_negative_decimal === RUN TestMathematicalFunctions/FLOOR_function_tests/Floor_integer === RUN TestMathematicalFunctions/FLOOR_function_tests/Floor_null_value === RUN TestMathematicalFunctions/ABS_function_tests === RUN TestMathematicalFunctions/ABS_function_tests/Abs_positive_integer === RUN TestMathematicalFunctions/ABS_function_tests/Abs_negative_integer === RUN TestMathematicalFunctions/ABS_function_tests/Abs_positive_double === RUN TestMathematicalFunctions/ABS_function_tests/Abs_negative_double === RUN TestMathematicalFunctions/ABS_function_tests/Abs_positive_float === RUN TestMathematicalFunctions/ABS_function_tests/Abs_negative_float === RUN TestMathematicalFunctions/ABS_function_tests/Abs_zero === RUN TestMathematicalFunctions/ABS_function_tests/Abs_null_value --- PASS: TestMathematicalFunctions (0.00s) --- PASS: TestMathematicalFunctions/ROUND_function_tests (0.00s) --- PASS: TestMathematicalFunctions/ROUND_function_tests/Round_float_to_integer (0.00s) --- PASS: TestMathematicalFunctions/ROUND_function_tests/Round_integer_stays_integer (0.00s) --- PASS: TestMathematicalFunctions/ROUND_function_tests/Round_with_precision_2 (0.00s) --- PASS: TestMathematicalFunctions/ROUND_function_tests/Round_negative_number (0.00s) --- PASS: TestMathematicalFunctions/ROUND_function_tests/Round_null_value (0.00s) --- PASS: TestMathematicalFunctions/CEIL_function_tests (0.00s) --- PASS: TestMathematicalFunctions/CEIL_function_tests/Ceil_positive_decimal (0.00s) --- PASS: TestMathematicalFunctions/CEIL_function_tests/Ceil_negative_decimal (0.00s) --- PASS: TestMathematicalFunctions/CEIL_function_tests/Ceil_integer (0.00s) --- PASS: TestMathematicalFunctions/CEIL_function_tests/Ceil_null_value (0.00s) --- PASS: TestMathematicalFunctions/FLOOR_function_tests (0.00s) --- PASS: TestMathematicalFunctions/FLOOR_function_tests/Floor_positive_decimal (0.00s) --- PASS: TestMathematicalFunctions/FLOOR_function_tests/Floor_negative_decimal (0.00s) --- PASS: TestMathematicalFunctions/FLOOR_function_tests/Floor_integer (0.00s) --- PASS: TestMathematicalFunctions/FLOOR_function_tests/Floor_null_value (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_positive_integer (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_negative_integer (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_positive_double (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_negative_double (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_positive_float (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_negative_float (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_zero (0.00s) --- PASS: TestMathematicalFunctions/ABS_function_tests/Abs_null_value (0.00s) === RUN TestSQLEngine_ArithmeticOnlyQueryExecution === RUN TestSQLEngine_ArithmeticOnlyQueryExecution/Basic_arithmetic_only_query arithmetic_only_execution_test.go:75: SUCCESS: SELECT id+user_id, id*2 FROM user_events LIMIT 3 returned 3 rows with calculated values === RUN TestSQLEngine_ArithmeticOnlyQueryExecution/With_LIMIT_and_OFFSET_-_original_user_issue arithmetic_only_execution_test.go:75: SUCCESS: SELECT id+user_id, id*2 FROM user_events LIMIT 2 OFFSET 1 returned 2 rows with calculated values === RUN TestSQLEngine_ArithmeticOnlyQueryExecution/Multiple_arithmetic_expressions arithmetic_only_execution_test.go:75: SUCCESS: SELECT user_id+100, id-1000 FROM user_events LIMIT 1 returned 1 rows with calculated values --- PASS: TestSQLEngine_ArithmeticOnlyQueryExecution (0.00s) --- PASS: TestSQLEngine_ArithmeticOnlyQueryExecution/Basic_arithmetic_only_query (0.00s) --- PASS: TestSQLEngine_ArithmeticOnlyQueryExecution/With_LIMIT_and_OFFSET_-_original_user_issue (0.00s) --- PASS: TestSQLEngine_ArithmeticOnlyQueryExecution/Multiple_arithmetic_expressions (0.00s) === RUN TestSQLEngine_ArithmeticOnlyQueryBugReproduction arithmetic_only_execution_test.go:140: SUCCESS: Arithmetic-only query with OFFSET works correctly! arithmetic_only_execution_test.go:141: Query: SELECT id+user_id, id*amount, id*2 FROM user_events LIMIT 10 OFFSET 5 arithmetic_only_execution_test.go:142: Returned 5 rows with correct calculations --- PASS: TestSQLEngine_ArithmeticOnlyQueryBugReproduction (0.00s) === RUN TestArithmeticExpressionParsing === RUN TestArithmeticExpressionParsing/simple_addition === RUN TestArithmeticExpressionParsing/simple_subtraction === RUN TestArithmeticExpressionParsing/multiplication_with_spaces === RUN TestArithmeticExpressionParsing/string_concatenation === RUN TestArithmeticExpressionParsing/string_concatenation_with_spaces === RUN TestArithmeticExpressionParsing/not_arithmetic --- PASS: TestArithmeticExpressionParsing (0.00s) --- PASS: TestArithmeticExpressionParsing/simple_addition (0.00s) --- PASS: TestArithmeticExpressionParsing/simple_subtraction (0.00s) --- PASS: TestArithmeticExpressionParsing/multiplication_with_spaces (0.00s) --- PASS: TestArithmeticExpressionParsing/string_concatenation (0.00s) --- PASS: TestArithmeticExpressionParsing/string_concatenation_with_spaces (0.00s) --- PASS: TestArithmeticExpressionParsing/not_arithmetic (0.00s) === RUN TestArithmeticExpressionEvaluation === RUN TestArithmeticExpressionEvaluation/integer_addition === RUN TestArithmeticExpressionEvaluation/integer_subtraction === RUN TestArithmeticExpressionEvaluation/mixed_types_multiplication === RUN TestArithmeticExpressionEvaluation/string_concatenation === RUN TestArithmeticExpressionEvaluation/string_concatenation_with_spaces --- PASS: TestArithmeticExpressionEvaluation (0.00s) --- PASS: TestArithmeticExpressionEvaluation/integer_addition (0.00s) --- PASS: TestArithmeticExpressionEvaluation/integer_subtraction (0.00s) --- PASS: TestArithmeticExpressionEvaluation/mixed_types_multiplication (0.00s) --- PASS: TestArithmeticExpressionEvaluation/string_concatenation (0.00s) --- PASS: TestArithmeticExpressionEvaluation/string_concatenation_with_spaces (0.00s) === RUN TestSelectArithmeticExpression --- PASS: TestSelectArithmeticExpression (0.00s) === RUN TestArithmeticWithFunctions === RUN TestArithmeticWithFunctions/Simple_function_arithmetic arithmetic_with_functions_test.go:75: PASS Basic function call with addition: SELECT LENGTH('hello') + 10 FROM user_events LIMIT 1 → 15 === RUN TestArithmeticWithFunctions/Nested_functions_with_arithmetic arithmetic_with_functions_test.go:75: PASS Complex nested functions with arithmetic operation (user's original failing query): SELECT length(trim(' hello world ')) + 12 FROM user_events LIMIT 1 → 23 === RUN TestArithmeticWithFunctions/Function_subtraction arithmetic_with_functions_test.go:75: PASS Function call with subtraction: SELECT LENGTH('programming') - 5 FROM user_events LIMIT 1 → 6 === RUN TestArithmeticWithFunctions/Function_multiplication arithmetic_with_functions_test.go:75: PASS Function call with multiplication: SELECT LENGTH('test') * 3 FROM user_events LIMIT 1 → 12 === RUN TestArithmeticWithFunctions/Multiple_nested_functions arithmetic_with_functions_test.go:75: PASS Triple nested functions: SELECT LENGTH(UPPER(TRIM(' hello '))) FROM user_events LIMIT 1 → 5 --- PASS: TestArithmeticWithFunctions (0.00s) --- PASS: TestArithmeticWithFunctions/Simple_function_arithmetic (0.00s) --- PASS: TestArithmeticWithFunctions/Nested_functions_with_arithmetic (0.00s) --- PASS: TestArithmeticWithFunctions/Function_subtraction (0.00s) --- PASS: TestArithmeticWithFunctions/Function_multiplication (0.00s) --- PASS: TestArithmeticWithFunctions/Multiple_nested_functions (0.00s) === RUN TestConvertMQSchemaToTableInfo_NoSchema === RUN TestConvertMQSchemaToTableInfo_NoSchema/nil_schema === RUN TestConvertMQSchemaToTableInfo_NoSchema/schema_with_nil_RecordType --- PASS: TestConvertMQSchemaToTableInfo_NoSchema (0.00s) --- PASS: TestConvertMQSchemaToTableInfo_NoSchema/nil_schema (0.00s) --- PASS: TestConvertMQSchemaToTableInfo_NoSchema/schema_with_nil_RecordType (0.00s) === RUN TestCockroachDBParserSuccess === RUN TestCockroachDBParserSuccess/Basic_Function cockroach_parser_success_test.go:93: SUCCESS: Simple function call → 5 === RUN TestCockroachDBParserSuccess/Function_Arithmetic cockroach_parser_success_test.go:93: SUCCESS: Function with arithmetic operation (original user issue) → 15 === RUN TestCockroachDBParserSuccess/User_Original_Query cockroach_parser_success_test.go:93: SUCCESS: User's exact original failing query - now fixed! → 23 === RUN TestCockroachDBParserSuccess/String_Concatenation cockroach_parser_success_test.go:93: SUCCESS: Basic string concatenation → helloworld === RUN TestCockroachDBParserSuccess/Function_With_Concat cockroach_parser_success_test.go:93: SUCCESS: Function with string concatenation argument → 10 === RUN TestCockroachDBParserSuccess/Multiple_Arithmetic cockroach_parser_success_test.go:93: SUCCESS: Function with multiplication → 12 === RUN TestCockroachDBParserSuccess/Nested_Functions cockroach_parser_success_test.go:93: SUCCESS: Nested function calls → 5 === RUN TestCockroachDBParserSuccess/Column_Alias cockroach_parser_success_test.go:93: SUCCESS: Column alias functionality (AS keyword) → 4 === NAME TestCockroachDBParserSuccess cockroach_parser_success_test.go:101: CockroachDB Parser Integration: 8/8 tests passed! --- PASS: TestCockroachDBParserSuccess (0.00s) --- PASS: TestCockroachDBParserSuccess/Basic_Function (0.00s) --- PASS: TestCockroachDBParserSuccess/Function_Arithmetic (0.00s) --- PASS: TestCockroachDBParserSuccess/User_Original_Query (0.00s) --- PASS: TestCockroachDBParserSuccess/String_Concatenation (0.00s) --- PASS: TestCockroachDBParserSuccess/Function_With_Concat (0.00s) --- PASS: TestCockroachDBParserSuccess/Multiple_Arithmetic (0.00s) --- PASS: TestCockroachDBParserSuccess/Nested_Functions (0.00s) --- PASS: TestCockroachDBParserSuccess/Column_Alias (0.00s) === RUN TestCompleteSQLFixes === RUN TestCompleteSQLFixes/OriginalFailingProductionQueries === RUN TestCompleteSQLFixes/OriginalFailingProductionQueries/OriginalFailingQuery1 === RUN TestCompleteSQLFixes/OriginalFailingProductionQueries/OriginalFailingQuery2 === RUN TestCompleteSQLFixes/OriginalFailingProductionQueries/CurrentDataQuery === RUN TestCompleteSQLFixes/AllFixesWorkTogether === RUN TestCompleteSQLFixes/BackwardCompatibilityVerified === RUN TestCompleteSQLFixes/PerformanceAndStability === RUN TestCompleteSQLFixes/EdgeCasesAndErrorHandling --- PASS: TestCompleteSQLFixes (0.00s) --- PASS: TestCompleteSQLFixes/OriginalFailingProductionQueries (0.00s) --- PASS: TestCompleteSQLFixes/OriginalFailingProductionQueries/OriginalFailingQuery1 (0.00s) --- PASS: TestCompleteSQLFixes/OriginalFailingProductionQueries/OriginalFailingQuery2 (0.00s) --- PASS: TestCompleteSQLFixes/OriginalFailingProductionQueries/CurrentDataQuery (0.00s) --- PASS: TestCompleteSQLFixes/AllFixesWorkTogether (0.00s) --- PASS: TestCompleteSQLFixes/BackwardCompatibilityVerified (0.00s) --- PASS: TestCompleteSQLFixes/PerformanceAndStability (0.00s) --- PASS: TestCompleteSQLFixes/EdgeCasesAndErrorHandling (0.00s) === RUN TestSQLFixesSummary === RUN TestSQLFixesSummary/Summary complete_sql_fixes_test.go:251: ALL SQL FIXES VERIFIED: complete_sql_fixes_test.go:252: Timestamp precision for large int64 values complete_sql_fixes_test.go:253: SQL alias resolution in WHERE clauses complete_sql_fixes_test.go:254: Scan boundary fixes for equality queries complete_sql_fixes_test.go:255: Range query fixes for equal boundaries complete_sql_fixes_test.go:256: Hybrid scanner time range handling complete_sql_fixes_test.go:257: Backward compatibility maintained complete_sql_fixes_test.go:258: Production stability verified --- PASS: TestSQLFixesSummary (0.00s) --- PASS: TestSQLFixesSummary/Summary (0.00s) === RUN TestComprehensiveSQLSuite === RUN TestComprehensiveSQLSuite/Basic_Select_All comprehensive_sql_test.go:319: PASS: Success: Basic select all columns === RUN TestComprehensiveSQLSuite/Basic_Select_Column comprehensive_sql_test.go:319: PASS: Success: Basic select single column === RUN TestComprehensiveSQLSuite/Basic_Select_Multiple_Columns comprehensive_sql_test.go:319: PASS: Success: Basic select multiple columns === RUN TestComprehensiveSQLSuite/Arithmetic_Multiply_FIXED comprehensive_sql_test.go:319: PASS: Success: FIXED: Arithmetic multiplication works === RUN TestComprehensiveSQLSuite/Arithmetic_Add comprehensive_sql_test.go:319: PASS: Success: Arithmetic addition works === RUN TestComprehensiveSQLSuite/Arithmetic_Subtract comprehensive_sql_test.go:319: PASS: Success: Arithmetic subtraction works === RUN TestComprehensiveSQLSuite/Arithmetic_Divide comprehensive_sql_test.go:319: PASS: Success: Arithmetic division works === RUN TestComprehensiveSQLSuite/Arithmetic_Complex comprehensive_sql_test.go:319: PASS: Success: Complex arithmetic expression works === RUN TestComprehensiveSQLSuite/String_Concatenation comprehensive_sql_test.go:319: PASS: Success: String concatenation === RUN TestComprehensiveSQLSuite/String_Column_Concat comprehensive_sql_test.go:319: PASS: Success: Column string concatenation === RUN TestComprehensiveSQLSuite/Function_LENGTH comprehensive_sql_test.go:319: PASS: Success: LENGTH function with literal === RUN TestComprehensiveSQLSuite/Function_LENGTH_Column comprehensive_sql_test.go:319: PASS: Success: LENGTH function with column === RUN TestComprehensiveSQLSuite/Function_UPPER comprehensive_sql_test.go:319: PASS: Success: UPPER function === RUN TestComprehensiveSQLSuite/Function_Nested comprehensive_sql_test.go:319: PASS: Success: Nested functions === RUN TestComprehensiveSQLSuite/Function_Arithmetic comprehensive_sql_test.go:319: PASS: Success: Function with arithmetic === RUN TestComprehensiveSQLSuite/Function_Arithmetic_Complex comprehensive_sql_test.go:319: PASS: Success: Function with complex arithmetic === RUN TestComprehensiveSQLSuite/Table_Simple comprehensive_sql_test.go:319: PASS: Success: Simple table reference === RUN TestComprehensiveSQLSuite/Table_With_Database comprehensive_sql_test.go:319: PASS: Success: Table with database qualifier === RUN TestComprehensiveSQLSuite/Table_Quoted comprehensive_sql_test.go:319: PASS: Success: Quoted table name === RUN TestComprehensiveSQLSuite/Where_Simple comprehensive_sql_test.go:319: PASS: Success: Simple WHERE clause === RUN TestComprehensiveSQLSuite/Where_String comprehensive_sql_test.go:319: PASS: Success: WHERE clause with string === RUN TestComprehensiveSQLSuite/Limit_Only comprehensive_sql_test.go:319: PASS: Success: LIMIT clause only === RUN TestComprehensiveSQLSuite/Limit_Offset comprehensive_sql_test.go:319: PASS: Success: LIMIT with OFFSET === RUN TestComprehensiveSQLSuite/DateTime_CURRENT_DATE comprehensive_sql_test.go:319: PASS: Success: CURRENT_DATE function === RUN TestComprehensiveSQLSuite/DateTime_NOW comprehensive_sql_test.go:319: PASS: Success: NOW() function === RUN TestComprehensiveSQLSuite/DateTime_EXTRACT comprehensive_sql_test.go:319: PASS: Success: EXTRACT function === RUN TestComprehensiveSQLSuite/Empty_String comprehensive_sql_test.go:319: PASS: Success: Empty string literal === RUN TestComprehensiveSQLSuite/Multiple_Spaces comprehensive_sql_test.go:319: PASS: Success: Query with multiple spaces === RUN TestComprehensiveSQLSuite/Mixed_Case comprehensive_sql_test.go:319: PASS: Success: Mixed case SQL === RUN TestComprehensiveSQLSuite/Show_Databases comprehensive_sql_test.go:319: PASS: Success: SHOW DATABASES statement === RUN TestComprehensiveSQLSuite/Show_Tables comprehensive_sql_test.go:319: PASS: Success: SHOW TABLES statement === NAME TestComprehensiveSQLSuite comprehensive_sql_test.go:327: ================================================================================ comprehensive_sql_test.go:328: COMPREHENSIVE SQL TEST SUITE SUMMARY comprehensive_sql_test.go:329: ================================================================================ comprehensive_sql_test.go:330: Total Tests: 31 comprehensive_sql_test.go:331: Successful: 31 comprehensive_sql_test.go:332: Panics: 0 comprehensive_sql_test.go:333: Errors: 0 comprehensive_sql_test.go:334: ================================================================================ --- PASS: TestComprehensiveSQLSuite (0.00s) --- PASS: TestComprehensiveSQLSuite/Basic_Select_All (0.00s) --- PASS: TestComprehensiveSQLSuite/Basic_Select_Column (0.00s) --- PASS: TestComprehensiveSQLSuite/Basic_Select_Multiple_Columns (0.00s) --- PASS: TestComprehensiveSQLSuite/Arithmetic_Multiply_FIXED (0.00s) --- PASS: TestComprehensiveSQLSuite/Arithmetic_Add (0.00s) --- PASS: TestComprehensiveSQLSuite/Arithmetic_Subtract (0.00s) --- PASS: TestComprehensiveSQLSuite/Arithmetic_Divide (0.00s) --- PASS: TestComprehensiveSQLSuite/Arithmetic_Complex (0.00s) --- PASS: TestComprehensiveSQLSuite/String_Concatenation (0.00s) --- PASS: TestComprehensiveSQLSuite/String_Column_Concat (0.00s) --- PASS: TestComprehensiveSQLSuite/Function_LENGTH (0.00s) --- PASS: TestComprehensiveSQLSuite/Function_LENGTH_Column (0.00s) --- PASS: TestComprehensiveSQLSuite/Function_UPPER (0.00s) --- PASS: TestComprehensiveSQLSuite/Function_Nested (0.00s) --- PASS: TestComprehensiveSQLSuite/Function_Arithmetic (0.00s) --- PASS: TestComprehensiveSQLSuite/Function_Arithmetic_Complex (0.00s) --- PASS: TestComprehensiveSQLSuite/Table_Simple (0.00s) --- PASS: TestComprehensiveSQLSuite/Table_With_Database (0.00s) --- PASS: TestComprehensiveSQLSuite/Table_Quoted (0.00s) --- PASS: TestComprehensiveSQLSuite/Where_Simple (0.00s) --- PASS: TestComprehensiveSQLSuite/Where_String (0.00s) --- PASS: TestComprehensiveSQLSuite/Limit_Only (0.00s) --- PASS: TestComprehensiveSQLSuite/Limit_Offset (0.00s) --- PASS: TestComprehensiveSQLSuite/DateTime_CURRENT_DATE (0.00s) --- PASS: TestComprehensiveSQLSuite/DateTime_NOW (0.00s) --- PASS: TestComprehensiveSQLSuite/DateTime_EXTRACT (0.00s) --- PASS: TestComprehensiveSQLSuite/Empty_String (0.00s) --- PASS: TestComprehensiveSQLSuite/Multiple_Spaces (0.00s) --- PASS: TestComprehensiveSQLSuite/Mixed_Case (0.00s) --- PASS: TestComprehensiveSQLSuite/Show_Databases (0.00s) --- PASS: TestComprehensiveSQLSuite/Show_Tables (0.00s) === RUN TestDateTimeFunctions === RUN TestDateTimeFunctions/CURRENT_DATE_function_tests === RUN TestDateTimeFunctions/CURRENT_TIMESTAMP_function_tests === RUN TestDateTimeFunctions/NOW_function_tests === RUN TestDateTimeFunctions/CURRENT_TIME_function_tests --- PASS: TestDateTimeFunctions (0.00s) --- PASS: TestDateTimeFunctions/CURRENT_DATE_function_tests (0.00s) --- PASS: TestDateTimeFunctions/CURRENT_TIMESTAMP_function_tests (0.00s) --- PASS: TestDateTimeFunctions/NOW_function_tests (0.00s) --- PASS: TestDateTimeFunctions/CURRENT_TIME_function_tests (0.00s) === RUN TestExtractFunction === RUN TestExtractFunction/Extract_YEAR === RUN TestExtractFunction/Extract_MONTH === RUN TestExtractFunction/Extract_DAY === RUN TestExtractFunction/Extract_HOUR === RUN TestExtractFunction/Extract_MINUTE === RUN TestExtractFunction/Extract_SECOND === RUN TestExtractFunction/Extract_QUARTER_from_June === RUN TestExtractFunction/Extract_from_string_date === RUN TestExtractFunction/Extract_from_Unix_timestamp === RUN TestExtractFunction/Extract_from_null_value === RUN TestExtractFunction/Extract_invalid_part === RUN TestExtractFunction/Extract_from_invalid_string --- PASS: TestExtractFunction (0.00s) --- PASS: TestExtractFunction/Extract_YEAR (0.00s) --- PASS: TestExtractFunction/Extract_MONTH (0.00s) --- PASS: TestExtractFunction/Extract_DAY (0.00s) --- PASS: TestExtractFunction/Extract_HOUR (0.00s) --- PASS: TestExtractFunction/Extract_MINUTE (0.00s) --- PASS: TestExtractFunction/Extract_SECOND (0.00s) --- PASS: TestExtractFunction/Extract_QUARTER_from_June (0.00s) --- PASS: TestExtractFunction/Extract_from_string_date (0.00s) --- PASS: TestExtractFunction/Extract_from_Unix_timestamp (0.00s) --- PASS: TestExtractFunction/Extract_from_null_value (0.00s) --- PASS: TestExtractFunction/Extract_invalid_part (0.00s) --- PASS: TestExtractFunction/Extract_from_invalid_string (0.00s) === RUN TestDateTruncFunction === RUN TestDateTruncFunction/Truncate_to_second === RUN TestDateTruncFunction/Truncate_to_minute === RUN TestDateTruncFunction/Truncate_to_hour === RUN TestDateTruncFunction/Truncate_to_day === RUN TestDateTruncFunction/Truncate_to_month === RUN TestDateTruncFunction/Truncate_to_quarter === RUN TestDateTruncFunction/Truncate_to_year === RUN TestDateTruncFunction/Truncate_with_plural_precision === RUN TestDateTruncFunction/Truncate_from_string_date === RUN TestDateTruncFunction/Truncate_null_value === RUN TestDateTruncFunction/Invalid_precision --- PASS: TestDateTruncFunction (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_second (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_minute (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_hour (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_day (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_month (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_quarter (0.00s) --- PASS: TestDateTruncFunction/Truncate_to_year (0.00s) --- PASS: TestDateTruncFunction/Truncate_with_plural_precision (0.00s) --- PASS: TestDateTruncFunction/Truncate_from_string_date (0.00s) --- PASS: TestDateTruncFunction/Truncate_null_value (0.00s) --- PASS: TestDateTruncFunction/Invalid_precision (0.00s) === RUN TestDateTimeConstantsInSQL === RUN TestDateTimeConstantsInSQL/CURRENT_TIME_in_SQL_query datetime_functions_test.go:468: CURRENT_TIME returned valid time: 21:30:02 === RUN TestDateTimeConstantsInSQL/CURRENT_DATE_in_SQL_query datetime_functions_test.go:494: CURRENT_DATE returned: 2026-07-20 --- PASS: TestDateTimeConstantsInSQL (0.00s) --- PASS: TestDateTimeConstantsInSQL/CURRENT_TIME_in_SQL_query (0.00s) --- PASS: TestDateTimeConstantsInSQL/CURRENT_DATE_in_SQL_query (0.00s) === RUN TestFunctionArgumentCountHandling === RUN TestFunctionArgumentCountHandling/Zero-argument_function_should_fail_appropriately === RUN TestFunctionArgumentCountHandling/Single-argument_function_should_still_work === RUN TestFunctionArgumentCountHandling/Any_zero-argument_function_should_fail === RUN TestFunctionArgumentCountHandling/Wrong_argument_count_for_single-arg_function_should_fail --- PASS: TestFunctionArgumentCountHandling (0.00s) --- PASS: TestFunctionArgumentCountHandling/Zero-argument_function_should_fail_appropriately (0.00s) --- PASS: TestFunctionArgumentCountHandling/Single-argument_function_should_still_work (0.00s) --- PASS: TestFunctionArgumentCountHandling/Any_zero-argument_function_should_fail (0.00s) --- PASS: TestFunctionArgumentCountHandling/Wrong_argument_count_for_single-arg_function_should_fail (0.00s) === RUN TestExtractFunctionSQL === RUN TestExtractFunctionSQL/Extract_YEAR_from_current_date === RUN TestExtractFunctionSQL/Extract_MONTH_from_current_date === RUN TestExtractFunctionSQL/Extract_DAY_from_current_date === RUN TestExtractFunctionSQL/Extract_HOUR_from_current_timestamp === RUN TestExtractFunctionSQL/Extract_MINUTE_from_current_timestamp === RUN TestExtractFunctionSQL/Extract_QUARTER_from_current_date === RUN TestExtractFunctionSQL/Multiple_EXTRACT_functions === RUN TestExtractFunctionSQL/EXTRACT_with_invalid_date_part === RUN TestExtractFunctionSQL/EXTRACT_with_wrong_number_of_arguments === RUN TestExtractFunctionSQL/EXTRACT_with_too_many_arguments --- PASS: TestExtractFunctionSQL (0.00s) --- PASS: TestExtractFunctionSQL/Extract_YEAR_from_current_date (0.00s) --- PASS: TestExtractFunctionSQL/Extract_MONTH_from_current_date (0.00s) --- PASS: TestExtractFunctionSQL/Extract_DAY_from_current_date (0.00s) --- PASS: TestExtractFunctionSQL/Extract_HOUR_from_current_timestamp (0.00s) --- PASS: TestExtractFunctionSQL/Extract_MINUTE_from_current_timestamp (0.00s) --- PASS: TestExtractFunctionSQL/Extract_QUARTER_from_current_date (0.00s) --- PASS: TestExtractFunctionSQL/Multiple_EXTRACT_functions (0.00s) --- PASS: TestExtractFunctionSQL/EXTRACT_with_invalid_date_part (0.00s) --- PASS: TestExtractFunctionSQL/EXTRACT_with_wrong_number_of_arguments (0.00s) --- PASS: TestExtractFunctionSQL/EXTRACT_with_too_many_arguments (0.00s) === RUN TestDateTruncFunctionSQL === RUN TestDateTruncFunctionSQL/DATE_TRUNC_to_day === RUN TestDateTruncFunctionSQL/DATE_TRUNC_to_hour === RUN TestDateTruncFunctionSQL/DATE_TRUNC_to_month === RUN TestDateTruncFunctionSQL/DATE_TRUNC_with_invalid_precision === RUN TestDateTruncFunctionSQL/DATE_TRUNC_with_wrong_number_of_arguments --- PASS: TestDateTruncFunctionSQL (0.00s) --- PASS: TestDateTruncFunctionSQL/DATE_TRUNC_to_day (0.00s) --- PASS: TestDateTruncFunctionSQL/DATE_TRUNC_to_hour (0.00s) --- PASS: TestDateTruncFunctionSQL/DATE_TRUNC_to_month (0.00s) --- PASS: TestDateTruncFunctionSQL/DATE_TRUNC_with_invalid_precision (0.00s) --- PASS: TestDateTruncFunctionSQL/DATE_TRUNC_with_wrong_number_of_arguments (0.00s) === RUN TestFastPathPredicateValidation === RUN TestFastPathPredicateValidation/No_WHERE_clause fast_path_predicate_validation_test.go:142: No WHERE clause: onlyTimePredicates=true, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/Time-only_predicate_(greater_than) fast_path_predicate_validation_test.go:142: Time-only predicate (greater than): onlyTimePredicates=true, startTimeNs=1640995200000000000, stopTimeNs=0 === RUN TestFastPathPredicateValidation/Time-only_predicate_(less_than) fast_path_predicate_validation_test.go:142: Time-only predicate (less than): onlyTimePredicates=true, startTimeNs=0, stopTimeNs=1640995200000000000 === RUN TestFastPathPredicateValidation/Time-only_predicate_(range_with_AND) fast_path_predicate_validation_test.go:142: Time-only predicate (range with AND): onlyTimePredicates=true, startTimeNs=1640995200000000000, stopTimeNs=1641081600000000000 === RUN TestFastPathPredicateValidation/Mixed_predicate_(time_+_non-time) fast_path_predicate_validation_test.go:142: Mixed predicate (time + non-time): onlyTimePredicates=false, startTimeNs=1640995200000000000, stopTimeNs=0 === RUN TestFastPathPredicateValidation/Non-time_predicate_only fast_path_predicate_validation_test.go:142: Non-time predicate only: onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/Multiple_non-time_predicates fast_path_predicate_validation_test.go:142: Multiple non-time predicates: onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/OR_with_time_predicate_(unsafe) fast_path_predicate_validation_test.go:142: OR with time predicate (unsafe): onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/OR_with_only_time_predicates_(still_unsafe) fast_path_predicate_validation_test.go:142: OR with only time predicates (still unsafe): onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/String_column_comparison fast_path_predicate_validation_test.go:142: String column comparison: onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/Numeric_column_comparison fast_path_predicate_validation_test.go:142: Numeric column comparison: onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0 === RUN TestFastPathPredicateValidation/Internal_timestamp_column fast_path_predicate_validation_test.go:142: Internal timestamp column: onlyTimePredicates=true, startTimeNs=1640995200000000000, stopTimeNs=0 --- PASS: TestFastPathPredicateValidation (0.00s) --- PASS: TestFastPathPredicateValidation/No_WHERE_clause (0.00s) --- PASS: TestFastPathPredicateValidation/Time-only_predicate_(greater_than) (0.00s) --- PASS: TestFastPathPredicateValidation/Time-only_predicate_(less_than) (0.00s) --- PASS: TestFastPathPredicateValidation/Time-only_predicate_(range_with_AND) (0.00s) --- PASS: TestFastPathPredicateValidation/Mixed_predicate_(time_+_non-time) (0.00s) --- PASS: TestFastPathPredicateValidation/Non-time_predicate_only (0.00s) --- PASS: TestFastPathPredicateValidation/Multiple_non-time_predicates (0.00s) --- PASS: TestFastPathPredicateValidation/OR_with_time_predicate_(unsafe) (0.00s) --- PASS: TestFastPathPredicateValidation/OR_with_only_time_predicates_(still_unsafe) (0.00s) --- PASS: TestFastPathPredicateValidation/String_column_comparison (0.00s) --- PASS: TestFastPathPredicateValidation/Numeric_column_comparison (0.00s) --- PASS: TestFastPathPredicateValidation/Internal_timestamp_column (0.00s) === RUN TestFastPathAggregationSafety === RUN TestFastPathAggregationSafety/No_WHERE_-_should_use_fast_path fast_path_predicate_validation_test.go:215: No WHERE - should use fast path: canAttemptFastPath=true (onlyTimePredicates=true, startTimeNs=0, stopTimeNs=0) === RUN TestFastPathAggregationSafety/Time-only_WHERE_-_should_use_fast_path fast_path_predicate_validation_test.go:215: Time-only WHERE - should use fast path: canAttemptFastPath=true (onlyTimePredicates=true, startTimeNs=1640995200000000000, stopTimeNs=0) === RUN TestFastPathAggregationSafety/Mixed_WHERE_-_should_NOT_use_fast_path fast_path_predicate_validation_test.go:215: Mixed WHERE - should NOT use fast path: canAttemptFastPath=false (onlyTimePredicates=false, startTimeNs=1640995200000000000, stopTimeNs=0) === RUN TestFastPathAggregationSafety/Non-time_WHERE_-_should_NOT_use_fast_path fast_path_predicate_validation_test.go:215: Non-time WHERE - should NOT use fast path: canAttemptFastPath=false (onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0) === RUN TestFastPathAggregationSafety/OR_expression_-_should_NOT_use_fast_path fast_path_predicate_validation_test.go:215: OR expression - should NOT use fast path: canAttemptFastPath=false (onlyTimePredicates=false, startTimeNs=0, stopTimeNs=0) --- PASS: TestFastPathAggregationSafety (0.00s) --- PASS: TestFastPathAggregationSafety/No_WHERE_-_should_use_fast_path (0.00s) --- PASS: TestFastPathAggregationSafety/Time-only_WHERE_-_should_use_fast_path (0.00s) --- PASS: TestFastPathAggregationSafety/Mixed_WHERE_-_should_NOT_use_fast_path (0.00s) --- PASS: TestFastPathAggregationSafety/Non-time_WHERE_-_should_NOT_use_fast_path (0.00s) --- PASS: TestFastPathAggregationSafety/OR_expression_-_should_NOT_use_fast_path (0.00s) === RUN TestTimestampColumnDetection === RUN TestTimestampColumnDetection/_ts fast_path_predicate_validation_test.go:269: Column '_ts': isTimestamp=true === RUN TestTimestampColumnDetection/_ts_ns fast_path_predicate_validation_test.go:269: Column '_ts_ns': isTimestamp=true === RUN TestTimestampColumnDetection/user_id fast_path_predicate_validation_test.go:269: Column 'user_id': isTimestamp=false === RUN TestTimestampColumnDetection/id fast_path_predicate_validation_test.go:269: Column 'id': isTimestamp=false === RUN TestTimestampColumnDetection/status fast_path_predicate_validation_test.go:269: Column 'status': isTimestamp=false === RUN TestTimestampColumnDetection/event_type fast_path_predicate_validation_test.go:269: Column 'event_type': isTimestamp=false --- PASS: TestTimestampColumnDetection (0.00s) --- PASS: TestTimestampColumnDetection/_ts (0.00s) --- PASS: TestTimestampColumnDetection/_ts_ns (0.00s) --- PASS: TestTimestampColumnDetection/user_id (0.00s) --- PASS: TestTimestampColumnDetection/id (0.00s) --- PASS: TestTimestampColumnDetection/status (0.00s) --- PASS: TestTimestampColumnDetection/event_type (0.00s) === RUN TestSQLEngine_HybridSelectBasic hybrid_test.go:67: Found live_log data source from unflushed messages --- PASS: TestSQLEngine_HybridSelectBasic (0.00s) === RUN TestSQLEngine_HybridSelectWithLimit --- PASS: TestSQLEngine_HybridSelectWithLimit (0.00s) === RUN TestSQLEngine_HybridSelectDifferentTables hybrid_test.go:129: Table user_events: 1 columns, 10 rows with hybrid data sources hybrid_test.go:129: Table system_logs: 1 columns, 4 rows with hybrid data sources --- PASS: TestSQLEngine_HybridSelectDifferentTables (0.00s) === RUN TestSQLEngine_HybridDataSource hybrid_test.go:178: Found live event: live_login from live_log hybrid_test.go:178: Found live event: live_action from live_log hybrid_test.go:183: Found archived event: archived_login from parquet_archive hybrid_test.go:183: Found archived event: archived_logout from parquet_archive --- PASS: TestSQLEngine_HybridDataSource (0.00s) === RUN TestSQLEngine_HybridSystemLogs hybrid_test.go:240: Live system log: level=INFO hybrid_test.go:240: Live system log: level=WARN hybrid_test.go:248: Archived system log: level=ERROR hybrid_test.go:248: Archived system log: level=INFO --- PASS: TestSQLEngine_HybridSystemLogs (0.00s) === RUN TestSQLEngine_HybridSelectWithTimeImplications hybrid_test.go:304: Hybrid query results: 4 live messages, 6 archived messages --- PASS: TestSQLEngine_HybridSelectWithTimeImplications (0.00s) === RUN TestMockBrokerClient_BasicFunctionality --- PASS: TestMockBrokerClient_BasicFunctionality (0.00s) === RUN TestMockBrokerClient_FailureScenarios --- PASS: TestMockBrokerClient_FailureScenarios (0.00s) === RUN TestMockBrokerClient_TopicManagement --- PASS: TestMockBrokerClient_TopicManagement (0.00s) === RUN TestSQLEngineWithMockBrokerClient_ErrorHandling mock_test.go:147: ExecuteSQL returned error (acceptable): topic default.nonexistent_topic not found or no schema available: mock broker failure: mock broker unavailable --- PASS: TestSQLEngineWithMockBrokerClient_ErrorHandling (0.00s) === RUN TestNoSchemaError --- PASS: TestNoSchemaError (0.00s) === RUN TestParseSQL_OFFSET_EdgeCases === RUN TestParseSQL_OFFSET_EdgeCases/Valid_LIMIT_OFFSET_with_WHERE === RUN TestParseSQL_OFFSET_EdgeCases/LIMIT_OFFSET_with_mixed_case === RUN TestParseSQL_OFFSET_EdgeCases/LIMIT_OFFSET_with_extra_spaces --- PASS: TestParseSQL_OFFSET_EdgeCases (0.00s) --- PASS: TestParseSQL_OFFSET_EdgeCases/Valid_LIMIT_OFFSET_with_WHERE (0.00s) --- PASS: TestParseSQL_OFFSET_EdgeCases/LIMIT_OFFSET_with_mixed_case (0.00s) --- PASS: TestParseSQL_OFFSET_EdgeCases/LIMIT_OFFSET_with_extra_spaces (0.00s) === RUN TestSQLEngine_OFFSET_EdgeCases === RUN TestSQLEngine_OFFSET_EdgeCases/OFFSET_larger_than_result_set === RUN TestSQLEngine_OFFSET_EdgeCases/OFFSET_with_LIMIT_0 === RUN TestSQLEngine_OFFSET_EdgeCases/High_OFFSET_with_small_LIMIT --- PASS: TestSQLEngine_OFFSET_EdgeCases (0.00s) --- PASS: TestSQLEngine_OFFSET_EdgeCases/OFFSET_larger_than_result_set (0.00s) --- PASS: TestSQLEngine_OFFSET_EdgeCases/OFFSET_with_LIMIT_0 (0.00s) --- PASS: TestSQLEngine_OFFSET_EdgeCases/High_OFFSET_with_small_LIMIT (0.00s) === RUN TestSQLEngine_OFFSET_ErrorCases === RUN TestSQLEngine_OFFSET_ErrorCases/Negative_OFFSET_value offset_test.go:149: Parser accepts negative OFFSET, execution should validate === RUN TestSQLEngine_OFFSET_ErrorCases/Very_large_OFFSET_value --- PASS: TestSQLEngine_OFFSET_ErrorCases (0.00s) --- PASS: TestSQLEngine_OFFSET_ErrorCases/Negative_OFFSET_value (0.00s) --- PASS: TestSQLEngine_OFFSET_ErrorCases/Very_large_OFFSET_value (0.00s) === RUN TestSQLEngine_OFFSET_Consistency === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_0 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_1 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_2 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_3 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_4 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_5 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_6 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_7 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_8 === RUN TestSQLEngine_OFFSET_Consistency/OFFSET_9 --- PASS: TestSQLEngine_OFFSET_Consistency (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_0 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_1 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_2 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_3 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_4 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_5 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_6 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_7 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_8 (0.00s) --- PASS: TestSQLEngine_OFFSET_Consistency/OFFSET_9 (0.00s) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_10_OFFSET_5_returns_correct_count offset_test.go:240: LIMIT 10 OFFSET 5 returned 5 rows (within limit) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_5_OFFSET_0 offset_test.go:287: LIMIT 5 OFFSET 0: returned 5 rows (within limit 5) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_5_OFFSET_2 offset_test.go:287: LIMIT 5 OFFSET 2: returned 5 rows (within limit 5) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_8_OFFSET_3 offset_test.go:287: LIMIT 8 OFFSET 3: returned 7 rows (within limit 8) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_15_OFFSET_1 offset_test.go:287: LIMIT 15 OFFSET 1: returned 9 rows (within limit 15) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_3_OFFSET_7 offset_test.go:287: LIMIT 3 OFFSET 7: returned 3 rows (within limit 3) === RUN TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_12_OFFSET_4 offset_test.go:287: LIMIT 12 OFFSET 4: returned 6 rows (within limit 12) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_10_OFFSET_5_returns_correct_count (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_5_OFFSET_0 (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_5_OFFSET_2 (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_8_OFFSET_3 (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_15_OFFSET_1 (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_3_OFFSET_7 (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_BugFix/LIMIT_12_OFFSET_4 (0.00s) === RUN TestSQLEngine_OFFSET_DataCollectionBuffer === RUN TestSQLEngine_OFFSET_DataCollectionBuffer/Large_OFFSET_with_small_LIMIT === RUN TestSQLEngine_OFFSET_DataCollectionBuffer/Medium_OFFSET_with_medium_LIMIT === RUN TestSQLEngine_OFFSET_DataCollectionBuffer/Progressive_OFFSET_test offset_test.go:351: OFFSET 0: returned 3 rows offset_test.go:351: OFFSET 1: returned 3 rows offset_test.go:351: OFFSET 2: returned 3 rows offset_test.go:351: OFFSET 3: returned 3 rows offset_test.go:351: OFFSET 4: returned 3 rows offset_test.go:351: OFFSET 5: returned 3 rows --- PASS: TestSQLEngine_OFFSET_DataCollectionBuffer (0.00s) --- PASS: TestSQLEngine_OFFSET_DataCollectionBuffer/Large_OFFSET_with_small_LIMIT (0.00s) --- PASS: TestSQLEngine_OFFSET_DataCollectionBuffer/Medium_OFFSET_with_medium_LIMIT (0.00s) --- PASS: TestSQLEngine_OFFSET_DataCollectionBuffer/Progressive_OFFSET_test (0.00s) === RUN TestSQLEngine_LIMIT_OFFSET_ArithmeticExpressions === RUN TestSQLEngine_LIMIT_OFFSET_ArithmeticExpressions/Arithmetic_expressions_with_LIMIT_OFFSET offset_test.go:391: LIMIT 10: returned 10 rows offset_test.go:392: LIMIT 10 OFFSET 5: returned 5 rows offset_test.go:399: Insufficient data (10 rows) to fully test LIMIT 10 OFFSET 5 scenario offset_test.go:409: Row 0: id=417224, user_id=7810, id+user_id=425034 offset_test.go:409: Row 1: id=424297, user_id=8897, id+user_id=433194 offset_test.go:409: Row 2: id=431189, user_id=3400, id+user_id=434589 offset_test.go:409: Row 3: id=413249, user_id=5175, id+user_id=418424 offset_test.go:409: Row 4: id=120612, user_id=5429, id+user_id=126041 === RUN TestSQLEngine_LIMIT_OFFSET_ArithmeticExpressions/Multiplication_expressions offset_test.go:441: Row 0: id=82460, id*2=164920 ✓ offset_test.go:441: Row 1: id=841256, id*2=1682512 ✓ offset_test.go:441: Row 2: id=55537, id*2=111074 ✓ --- PASS: TestSQLEngine_LIMIT_OFFSET_ArithmeticExpressions (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_ArithmeticExpressions/Arithmetic_expressions_with_LIMIT_OFFSET (0.00s) --- PASS: TestSQLEngine_LIMIT_OFFSET_ArithmeticExpressions/Multiplication_expressions (0.00s) === RUN TestSQLEngine_OFFSET_WithAggregation === RUN TestSQLEngine_OFFSET_WithAggregation/COUNT_with_OFFSET === RUN TestSQLEngine_OFFSET_WithAggregation/COUNT_with_OFFSET_1 --- PASS: TestSQLEngine_OFFSET_WithAggregation (0.00s) --- PASS: TestSQLEngine_OFFSET_WithAggregation/COUNT_with_OFFSET (0.00s) --- PASS: TestSQLEngine_OFFSET_WithAggregation/COUNT_with_OFFSET_1 (0.00s) === RUN TestBasicParsing === RUN TestBasicParsing/Query_1 parsing_debug_test.go:20: Testing SQL: SELECT * FROM user_events parsing_debug_test.go:28: Parsed statement type: *engine.SelectStatement parsing_debug_test.go:31: SelectStatement details: parsing_debug_test.go:32: SelectExprs count: 1 parsing_debug_test.go:33: From count: 1 parsing_debug_test.go:34: WHERE clause exists: false parsing_debug_test.go:39: WHERE clause is NIL - this is the bug! === RUN TestBasicParsing/Query_2 parsing_debug_test.go:20: Testing SQL: SELECT id FROM user_events parsing_debug_test.go:28: Parsed statement type: *engine.SelectStatement parsing_debug_test.go:31: SelectStatement details: parsing_debug_test.go:32: SelectExprs count: 1 parsing_debug_test.go:33: From count: 1 parsing_debug_test.go:34: WHERE clause exists: false parsing_debug_test.go:39: WHERE clause is NIL - this is the bug! === RUN TestBasicParsing/Query_3 parsing_debug_test.go:20: Testing SQL: SELECT id FROM user_events WHERE id = 123 parsing_debug_test.go:28: Parsed statement type: *engine.SelectStatement parsing_debug_test.go:31: SelectStatement details: parsing_debug_test.go:32: SelectExprs count: 1 parsing_debug_test.go:33: From count: 1 parsing_debug_test.go:34: WHERE clause exists: true parsing_debug_test.go:37: WHERE expression type: *engine.ComparisonExpr === RUN TestBasicParsing/Query_4 parsing_debug_test.go:20: Testing SQL: SELECT id FROM user_events WHERE id > 123 parsing_debug_test.go:28: Parsed statement type: *engine.SelectStatement parsing_debug_test.go:31: SelectStatement details: parsing_debug_test.go:32: SelectExprs count: 1 parsing_debug_test.go:33: From count: 1 parsing_debug_test.go:34: WHERE clause exists: true parsing_debug_test.go:37: WHERE expression type: *engine.ComparisonExpr === RUN TestBasicParsing/Query_5 parsing_debug_test.go:20: Testing SQL: SELECT id FROM user_events WHERE status = 'active' parsing_debug_test.go:28: Parsed statement type: *engine.SelectStatement parsing_debug_test.go:31: SelectStatement details: parsing_debug_test.go:32: SelectExprs count: 1 parsing_debug_test.go:33: From count: 1 parsing_debug_test.go:34: WHERE clause exists: true parsing_debug_test.go:37: WHERE expression type: *engine.ComparisonExpr --- PASS: TestBasicParsing (0.00s) --- PASS: TestBasicParsing/Query_1 (0.00s) --- PASS: TestBasicParsing/Query_2 (0.00s) --- PASS: TestBasicParsing/Query_3 (0.00s) --- PASS: TestBasicParsing/Query_4 (0.00s) --- PASS: TestBasicParsing/Query_5 (0.00s) === RUN TestCockroachParserDirectly parsing_debug_test.go:53: Testing CockroachDB parser directly with: SELECT id FROM user_events WHERE id > 123 parsing_debug_test.go:61: Our ParseSQL returned: *engine.SelectStatement parsing_debug_test.go:68: Our ParseSQL extracted WHERE clause: *engine.ComparisonExpr --- PASS: TestCockroachParserDirectly (0.00s) === RUN TestParseMethodComparison parsing_debug_test.go:77: Comparing parsing methods for: SELECT id FROM user_events WHERE id > 123 parsing_debug_test.go:81: Global ParseSQL: *engine.SelectStatement, error: parsing_debug_test.go:84: WHERE clause: true parsing_debug_test.go:92: ExecuteSQL error (helps identify parsing path): --- PASS: TestParseMethodComparison (0.00s) === RUN TestPostgreSQLOnlySupport === RUN TestPostgreSQLOnlySupport/MySQL_Backticks_Table postgresql_only_test.go:89: CORRECTLY REJECTED: MySQL backticks for table names should be rejected === RUN TestPostgreSQLOnlySupport/MySQL_Backticks_Column postgresql_only_test.go:89: CORRECTLY REJECTED: MySQL backticks for column names should be rejected === RUN TestPostgreSQLOnlySupport/PostgreSQL_Double_Quotes_OK postgresql_only_test.go:103: CORRECTLY ACCEPTED: PostgreSQL double quotes for identifiers should work === RUN TestPostgreSQLOnlySupport/PostgreSQL_EXTRACT_OK postgresql_only_test.go:103: CORRECTLY ACCEPTED: PostgreSQL EXTRACT function should work === RUN TestPostgreSQLOnlySupport/Single_Quotes_String_Literal_OK postgresql_only_test.go:103: CORRECTLY ACCEPTED: Single quotes for string literals should work === NAME TestPostgreSQLOnlySupport postgresql_only_test.go:109: PostgreSQL-only compliance: 5/5 tests passed --- PASS: TestPostgreSQLOnlySupport (0.00s) --- PASS: TestPostgreSQLOnlySupport/MySQL_Backticks_Table (0.00s) --- PASS: TestPostgreSQLOnlySupport/MySQL_Backticks_Column (0.00s) --- PASS: TestPostgreSQLOnlySupport/PostgreSQL_Double_Quotes_OK (0.00s) --- PASS: TestPostgreSQLOnlySupport/PostgreSQL_EXTRACT_OK (0.00s) --- PASS: TestPostgreSQLOnlySupport/Single_Quotes_String_Literal_OK (0.00s) === RUN TestParseSQL_COUNT_Functions === RUN TestParseSQL_COUNT_Functions/COUNT(*)_basic === RUN TestParseSQL_COUNT_Functions/COUNT(column_name) === RUN TestParseSQL_COUNT_Functions/Multiple_aggregate_functions --- PASS: TestParseSQL_COUNT_Functions (0.00s) --- PASS: TestParseSQL_COUNT_Functions/COUNT(*)_basic (0.00s) --- PASS: TestParseSQL_COUNT_Functions/COUNT(column_name) (0.00s) --- PASS: TestParseSQL_COUNT_Functions/Multiple_aggregate_functions (0.00s) === RUN TestParseSQL_SELECT_Expressions === RUN TestParseSQL_SELECT_Expressions/SELECT_*_FROM_table === RUN TestParseSQL_SELECT_Expressions/SELECT_column_FROM_table === RUN TestParseSQL_SELECT_Expressions/SELECT_multiple_columns --- PASS: TestParseSQL_SELECT_Expressions (0.00s) --- PASS: TestParseSQL_SELECT_Expressions/SELECT_*_FROM_table (0.00s) --- PASS: TestParseSQL_SELECT_Expressions/SELECT_column_FROM_table (0.00s) --- PASS: TestParseSQL_SELECT_Expressions/SELECT_multiple_columns (0.00s) === RUN TestParseSQL_WHERE_Clauses === RUN TestParseSQL_WHERE_Clauses/WHERE_with_simple_comparison === RUN TestParseSQL_WHERE_Clauses/WHERE_with_AND_condition === RUN TestParseSQL_WHERE_Clauses/WHERE_with_OR_condition --- PASS: TestParseSQL_WHERE_Clauses (0.00s) --- PASS: TestParseSQL_WHERE_Clauses/WHERE_with_simple_comparison (0.00s) --- PASS: TestParseSQL_WHERE_Clauses/WHERE_with_AND_condition (0.00s) --- PASS: TestParseSQL_WHERE_Clauses/WHERE_with_OR_condition (0.00s) === RUN TestParseSQL_LIMIT_Clauses === RUN TestParseSQL_LIMIT_Clauses/LIMIT_with_number === RUN TestParseSQL_LIMIT_Clauses/LIMIT_with_OFFSET === RUN TestParseSQL_LIMIT_Clauses/LIMIT_with_OFFSET_zero === RUN TestParseSQL_LIMIT_Clauses/LIMIT_with_large_OFFSET --- PASS: TestParseSQL_LIMIT_Clauses (0.00s) --- PASS: TestParseSQL_LIMIT_Clauses/LIMIT_with_number (0.00s) --- PASS: TestParseSQL_LIMIT_Clauses/LIMIT_with_OFFSET (0.00s) --- PASS: TestParseSQL_LIMIT_Clauses/LIMIT_with_OFFSET_zero (0.00s) --- PASS: TestParseSQL_LIMIT_Clauses/LIMIT_with_large_OFFSET (0.00s) === RUN TestParseSQL_SHOW_Statements === RUN TestParseSQL_SHOW_Statements/SHOW_DATABASES === RUN TestParseSQL_SHOW_Statements/SHOW_TABLES === RUN TestParseSQL_SHOW_Statements/SHOW_TABLES_FROM_database --- PASS: TestParseSQL_SHOW_Statements (0.00s) --- PASS: TestParseSQL_SHOW_Statements/SHOW_DATABASES (0.00s) --- PASS: TestParseSQL_SHOW_Statements/SHOW_TABLES (0.00s) --- PASS: TestParseSQL_SHOW_Statements/SHOW_TABLES_FROM_database (0.00s) === RUN TestRealNamespaceDiscovery real_namespace_test.go:24: Discovered 0 namespaces (no fallback data): real_namespace_test.go:26: (No namespaces found - requires real SeaweedFS MQ cluster) --- PASS: TestRealNamespaceDiscovery (0.00s) === RUN TestRealTopicDiscovery real_namespace_test.go:53: Discovered 0 topics in 'default' namespace (no fallback data): real_namespace_test.go:55: (No topics found - requires real SeaweedFS MQ cluster with 'default' namespace) --- PASS: TestRealTopicDiscovery (0.00s) === RUN TestNamespaceDiscoveryNoFallback real_namespace_test.go:79: ListNamespaces failed as expected: failed to get filer client: failed to discover filer: failed to list filers from master: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp [::1]:18888: connect: connection refused" real_namespace_test.go:99: No fallback behavior - returns empty lists when filer unavailable --- PASS: TestNamespaceDiscoveryNoFallback (0.00s) === RUN TestRealWorldWhereClauseFailure real_world_where_clause_test.go:44: TESTING REAL-WORLD WHERE CLAUSE SCENARIOS real_world_where_clause_test.go:45: ============================================ === RUN TestRealWorldWhereClauseFailure/Where_ID_Greater_Than_Large_Number real_world_where_clause_test.go:65: Query: SELECT id FROM user_events WHERE id > 10000000 real_world_where_clause_test.go:66: Total rows returned: 0 real_world_where_clause_test.go:131: No rows returned - this could be correct if no data matches === RUN TestRealWorldWhereClauseFailure/Where_ID_Greater_Than_Small_Number real_world_where_clause_test.go:65: Query: SELECT id FROM user_events WHERE id > 100000 real_world_where_clause_test.go:66: Total rows returned: 7 real_world_where_clause_test.go:69: Sample IDs returned: real_world_where_clause_test.go:78: Row 1: id = 841256 real_world_where_clause_test.go:78: Row 2: id = 686003 real_world_where_clause_test.go:78: Row 3: id = 417224 real_world_where_clause_test.go:78: Row 4: id = 424297 real_world_where_clause_test.go:78: Row 5: id = 431189 real_world_where_clause_test.go:120: Analysis: real_world_where_clause_test.go:121: Rows matching WHERE condition: 7 real_world_where_clause_test.go:122: Rows NOT matching WHERE condition: 0 real_world_where_clause_test.go:128: PASS: WHERE id > 100000 should filter results - All returned rows match the WHERE condition === RUN TestRealWorldWhereClauseFailure/Where_ID_Less_Than real_world_where_clause_test.go:65: Query: SELECT id FROM user_events WHERE id < 100000 real_world_where_clause_test.go:66: Total rows returned: 3 real_world_where_clause_test.go:69: Sample IDs returned: real_world_where_clause_test.go:78: Row 1: id = 82460 real_world_where_clause_test.go:78: Row 2: id = 55537 real_world_where_clause_test.go:78: Row 3: id = 65143 real_world_where_clause_test.go:120: Analysis: real_world_where_clause_test.go:121: Rows matching WHERE condition: 3 real_world_where_clause_test.go:122: Rows NOT matching WHERE condition: 0 real_world_where_clause_test.go:128: PASS: WHERE id < 100000 should filter results - All returned rows match the WHERE condition --- PASS: TestRealWorldWhereClauseFailure (0.00s) --- PASS: TestRealWorldWhereClauseFailure/Where_ID_Greater_Than_Large_Number (0.00s) --- PASS: TestRealWorldWhereClauseFailure/Where_ID_Greater_Than_Small_Number (0.00s) --- PASS: TestRealWorldWhereClauseFailure/Where_ID_Less_Than (0.00s) === RUN TestWhereClauseWithLimitOffset real_world_where_clause_test.go:144: Testing exact failing query: SELECT id FROM user_events WHERE id > 10000000 LIMIT 10 OFFSET 5 real_world_where_clause_test.go:159: Returned 0 rows (LIMIT 10 worked) real_world_where_clause_test.go:181: WHERE clause working correctly --- PASS: TestWhereClauseWithLimitOffset (0.00s) === RUN TestWhatShouldHaveBeenTested real_world_where_clause_test.go:189: THE TEST THAT SHOULD HAVE CAUGHT THE WHERE CLAUSE ISSUE real_world_where_clause_test.go:190: ======================================================== real_world_where_clause_test.go:199: All rows: 10 real_world_where_clause_test.go:200: WHERE id > 999999999: 0 rows real_world_where_clause_test.go:213: WHERE 1 = 0 (impossible): 0 rows --- PASS: TestWhatShouldHaveBeenTested (0.00s) === RUN TestSchemaAwareParsing === RUN TestSchemaAwareParsing/JSON_Message_Parsing schema_parsing_test.go:74: JSON parsing correctly converted types: int32=1234, string='login', double=75.5, bool=true === RUN TestSchemaAwareParsing/Raw_Data_Type_Conversion schema_parsing_test.go:118: Raw data type conversions working correctly === RUN TestSchemaAwareParsing/Invalid_JSON_Graceful_Handling schema_parsing_test.go:129: Invalid JSON handled gracefully with error --- PASS: TestSchemaAwareParsing (0.00s) --- PASS: TestSchemaAwareParsing/JSON_Message_Parsing (0.00s) --- PASS: TestSchemaAwareParsing/Raw_Data_Type_Conversion (0.00s) --- PASS: TestSchemaAwareParsing/Invalid_JSON_Graceful_Handling (0.00s) === RUN TestSchemaAwareParsingIntegration schema_parsing_test.go:160: Schema-aware parsing integrates correctly with SQL engine --- PASS: TestSchemaAwareParsingIntegration (0.00s) === RUN TestSQLEngine_SelectBasic --- PASS: TestSQLEngine_SelectBasic (0.00s) === RUN TestSQLEngine_SelectWithLimit --- PASS: TestSQLEngine_SelectWithLimit (0.00s) === RUN TestSQLEngine_SelectSpecificColumns --- PASS: TestSQLEngine_SelectSpecificColumns (0.00s) === RUN TestSQLEngine_SelectFromNonExistentTable select_test.go:83: Skipping non-existent table test - table name parsing issue needs investigation --- SKIP: TestSQLEngine_SelectFromNonExistentTable (0.00s) === RUN TestSQLEngine_SelectWithOffset --- PASS: TestSQLEngine_SelectWithOffset (0.00s) === RUN TestSQLEngine_SelectWithLimitAndOffset --- PASS: TestSQLEngine_SelectWithLimitAndOffset (0.00s) === RUN TestSQLEngine_SelectWithOffsetExceedsRows --- PASS: TestSQLEngine_SelectWithOffsetExceedsRows (0.00s) === RUN TestSQLEngine_SelectWithOffsetZero --- PASS: TestSQLEngine_SelectWithOffsetZero (0.00s) === RUN TestSQLEngine_SelectDifferentTables select_test.go:211: Table user_events: 4 columns, 10 rows select_test.go:211: Table system_logs: 3 columns, 4 rows --- PASS: TestSQLEngine_SelectDifferentTables (0.00s) === RUN TestSQLAliasResolution === RUN TestSQLAliasResolution/ResolveColumnAlias === RUN TestSQLAliasResolution/SingleAliasInWhere === RUN TestSQLAliasResolution/MultipleAliasesInWhere === RUN TestSQLAliasResolution/RangeQueryWithAliases === RUN TestSQLAliasResolution/MixedAliasAndDirectColumn === RUN TestSQLAliasResolution/AliasCompatibilityWithTimestampFixes === RUN TestSQLAliasResolution/EdgeCasesAndErrorHandling === RUN TestSQLAliasResolution/ComparisonOperators === RUN TestSQLAliasResolution/ComparisonOperators/=_1000 === RUN TestSQLAliasResolution/ComparisonOperators/=_999 === RUN TestSQLAliasResolution/ComparisonOperators/>_999 === RUN TestSQLAliasResolution/ComparisonOperators/>_1000 === RUN TestSQLAliasResolution/ComparisonOperators/>=_1000 === RUN TestSQLAliasResolution/ComparisonOperators/>=_1001 === RUN TestSQLAliasResolution/ComparisonOperators/<_1001 === RUN TestSQLAliasResolution/ComparisonOperators/<_1000 === RUN TestSQLAliasResolution/ComparisonOperators/<=_1000 === RUN TestSQLAliasResolution/ComparisonOperators/<=_999 === RUN TestSQLAliasResolution/BackwardCompatibility --- PASS: TestSQLAliasResolution (0.00s) --- PASS: TestSQLAliasResolution/ResolveColumnAlias (0.00s) --- PASS: TestSQLAliasResolution/SingleAliasInWhere (0.00s) --- PASS: TestSQLAliasResolution/MultipleAliasesInWhere (0.00s) --- PASS: TestSQLAliasResolution/RangeQueryWithAliases (0.00s) --- PASS: TestSQLAliasResolution/MixedAliasAndDirectColumn (0.00s) --- PASS: TestSQLAliasResolution/AliasCompatibilityWithTimestampFixes (0.00s) --- PASS: TestSQLAliasResolution/EdgeCasesAndErrorHandling (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/=_1000 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/=_999 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/>_999 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/>_1000 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/>=_1000 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/>=_1001 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/<_1001 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/<_1000 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/<=_1000 (0.00s) --- PASS: TestSQLAliasResolution/ComparisonOperators/<=_999 (0.00s) --- PASS: TestSQLAliasResolution/BackwardCompatibility (0.00s) === RUN TestAliasIntegrationWithProductionScenarios === RUN TestAliasIntegrationWithProductionScenarios/OriginalFailingQuery === RUN TestAliasIntegrationWithProductionScenarios/ComplexProductionQuery === RUN TestAliasIntegrationWithProductionScenarios/PerformanceRegression --- PASS: TestAliasIntegrationWithProductionScenarios (0.00s) --- PASS: TestAliasIntegrationWithProductionScenarios/OriginalFailingQuery (0.00s) --- PASS: TestAliasIntegrationWithProductionScenarios/ComplexProductionQuery (0.00s) --- PASS: TestAliasIntegrationWithProductionScenarios/PerformanceRegression (0.00s) === RUN TestSQLFeatureDiagnostic sql_feature_diagnostic_test.go:14: SEAWEEDFS SQL ENGINE FEATURE DIAGNOSTIC sql_feature_diagnostic_test.go:15: ================================================================================ sql_feature_diagnostic_test.go:18: 1. TESTING LIMIT FUNCTIONALITY: sql_feature_diagnostic_test.go:35: LIMIT 0: PASS - Got 0 rows sql_feature_diagnostic_test.go:35: LIMIT 1: PASS - Got 1 rows sql_feature_diagnostic_test.go:35: LIMIT 3: PASS - Got 3 rows sql_feature_diagnostic_test.go:35: LIMIT 5: PASS - Got 5 rows sql_feature_diagnostic_test.go:35: LIMIT 10: PASS - Got 10 rows sql_feature_diagnostic_test.go:35: LIMIT 100: PASS - Got 10 rows sql_feature_diagnostic_test.go:43: 2. TESTING OFFSET FUNCTIONALITY: sql_feature_diagnostic_test.go:58: OFFSET 0: PASS - Got 3 rows sql_feature_diagnostic_test.go:58: OFFSET 1: PASS - Got 3 rows sql_feature_diagnostic_test.go:58: OFFSET 2: PASS - Got 3 rows sql_feature_diagnostic_test.go:58: OFFSET 5: PASS - Got 3 rows sql_feature_diagnostic_test.go:56: OFFSET 10: PASS - Beyond data range, got 0 rows sql_feature_diagnostic_test.go:56: OFFSET 100: PASS - Beyond data range, got 0 rows sql_feature_diagnostic_test.go:64: 3. TESTING WHERE CLAUSE FUNCTIONALITY: sql_feature_diagnostic_test.go:90: Specific ID match: PASS - WHERE clause working, got 1 rows sql_feature_diagnostic_test.go:90: Greater than comparison: PASS - WHERE clause working, got 7 rows sql_feature_diagnostic_test.go:90: String equality: PASS - WHERE clause working, got 1 rows sql_feature_diagnostic_test.go:90: Non-existent ID: PASS - WHERE clause working, got 0 rows sql_feature_diagnostic_test.go:90: Always false condition: PASS - WHERE clause working, got 0 rows sql_feature_diagnostic_test.go:96: 4. TESTING COMBINED LIMIT + OFFSET + WHERE: sql_feature_diagnostic_test.go:106: Combined query: Got 2 rows (LIMIT=2 part works, WHERE filtering unknown) sql_feature_diagnostic_test.go:110: ================================================================================ sql_feature_diagnostic_test.go:111: FEATURE SUMMARY: sql_feature_diagnostic_test.go:112: LIMIT: FULLY WORKING - Correctly limits result rows sql_feature_diagnostic_test.go:113: OFFSET: FULLY WORKING - Correctly skips rows sql_feature_diagnostic_test.go:114: WHERE: FULLY WORKING - All comparison operators working sql_feature_diagnostic_test.go:115: SELECT: WORKING - Supports *, columns, functions, arithmetic sql_feature_diagnostic_test.go:116: Functions: WORKING - String and datetime functions work sql_feature_diagnostic_test.go:117: Arithmetic: WORKING - +, -, *, / operations work sql_feature_diagnostic_test.go:118: ================================================================================ --- PASS: TestSQLFeatureDiagnostic (0.00s) === RUN TestSQLWhereClauseIssue sql_feature_diagnostic_test.go:125: DEMONSTRATING WHERE CLAUSE ISSUE: sql_feature_diagnostic_test.go:130: Total rows in test data: 10 sql_feature_diagnostic_test.go:134: First row ID: 82460 sql_feature_diagnostic_test.go:144: WHERE id = 82460 returned 1 rows sql_feature_diagnostic_test.go:152: WHERE clause working correctly sql_feature_diagnostic_test.go:162: WHERE 1 = 0 returned 0 rows sql_feature_diagnostic_test.go:167: Impossible WHERE condition correctly returns no rows --- PASS: TestSQLWhereClauseIssue (0.00s) === RUN TestSQLFilteringLimitOffset === RUN TestSQLFilteringLimitOffset/Where_Equals_Integer sql_filtering_limit_offset_test.go:295: PASS: WHERE with equals operator (integer) - Correct row count: 1 === RUN TestSQLFilteringLimitOffset/Where_Equals_String sql_filtering_limit_offset_test.go:298: PASS: WHERE with equals operator (string) - Row count: 1 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Not_Equals sql_filtering_limit_offset_test.go:298: PASS: WHERE with not equals operator - Row count: 2 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Greater_Than sql_filtering_limit_offset_test.go:298: PASS: WHERE with greater than operator - Row count: 7 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Less_Than sql_filtering_limit_offset_test.go:298: PASS: WHERE with less than operator - Row count: 3 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Greater_Equal sql_filtering_limit_offset_test.go:298: PASS: WHERE with greater than or equal operator - Row count: 8 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Less_Equal sql_filtering_limit_offset_test.go:298: PASS: WHERE with less than or equal operator - Row count: 3 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Column_Comparison sql_filtering_limit_offset_test.go:295: PASS: WHERE filtering with specific columns selected - Correct row count: 1 === RUN TestSQLFilteringLimitOffset/Where_With_Function sql_filtering_limit_offset_test.go:298: PASS: WHERE with function in SELECT - Row count: 1 (not validated) === RUN TestSQLFilteringLimitOffset/Where_With_Arithmetic sql_filtering_limit_offset_test.go:295: PASS: WHERE with arithmetic in SELECT - Correct row count: 1 === RUN TestSQLFilteringLimitOffset/Limit_1 sql_filtering_limit_offset_test.go:295: PASS: LIMIT 1 row - Correct row count: 1 === RUN TestSQLFilteringLimitOffset/Limit_5 sql_filtering_limit_offset_test.go:295: PASS: LIMIT 5 rows - Correct row count: 5 === RUN TestSQLFilteringLimitOffset/Limit_0 sql_filtering_limit_offset_test.go:295: PASS: LIMIT 0 rows (should return no results) - Correct row count: 0 === RUN TestSQLFilteringLimitOffset/Limit_Large sql_filtering_limit_offset_test.go:298: PASS: LIMIT with large number - Row count: 10 (not validated) === RUN TestSQLFilteringLimitOffset/Limit_With_Columns sql_filtering_limit_offset_test.go:295: PASS: LIMIT with specific columns - Correct row count: 3 === RUN TestSQLFilteringLimitOffset/Limit_With_Functions sql_filtering_limit_offset_test.go:295: PASS: LIMIT with functions - Correct row count: 2 === RUN TestSQLFilteringLimitOffset/Offset_0 sql_filtering_limit_offset_test.go:295: PASS: OFFSET 0 (same as no offset) - Correct row count: 5 === RUN TestSQLFilteringLimitOffset/Offset_1 sql_filtering_limit_offset_test.go:295: PASS: OFFSET 1 row - Correct row count: 3 === RUN TestSQLFilteringLimitOffset/Offset_5 sql_filtering_limit_offset_test.go:295: PASS: OFFSET 5 rows - Correct row count: 2 === RUN TestSQLFilteringLimitOffset/Offset_Large sql_filtering_limit_offset_test.go:298: PASS: OFFSET with large number - Row count: 0 (not validated) === RUN TestSQLFilteringLimitOffset/Limit_Offset_Pagination_Page1 sql_filtering_limit_offset_test.go:295: PASS: Pagination: Page 1 (LIMIT 3, OFFSET 0) - Correct row count: 3 === RUN TestSQLFilteringLimitOffset/Limit_Offset_Pagination_Page2 sql_filtering_limit_offset_test.go:295: PASS: Pagination: Page 2 (LIMIT 3, OFFSET 3) - Correct row count: 3 === RUN TestSQLFilteringLimitOffset/Limit_Offset_Pagination_Page3 sql_filtering_limit_offset_test.go:295: PASS: Pagination: Page 3 (LIMIT 3, OFFSET 6) - Correct row count: 3 === RUN TestSQLFilteringLimitOffset/Where_Limit sql_filtering_limit_offset_test.go:298: PASS: WHERE clause with LIMIT - Row count: 1 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Limit_Offset sql_filtering_limit_offset_test.go:298: PASS: WHERE clause with LIMIT and OFFSET - Row count: 0 (not validated) === RUN TestSQLFilteringLimitOffset/Where_Complex_Limit sql_filtering_limit_offset_test.go:298: PASS: Complex WHERE with functions and arithmetic, plus LIMIT - Row count: 3 (not validated) === RUN TestSQLFilteringLimitOffset/Where_No_Match sql_filtering_limit_offset_test.go:295: PASS: WHERE clause that matches no rows - Correct row count: 0 === RUN TestSQLFilteringLimitOffset/Limit_Offset_Beyond_Data sql_filtering_limit_offset_test.go:295: PASS: OFFSET beyond available data - Correct row count: 0 === RUN TestSQLFilteringLimitOffset/Where_Empty_String sql_filtering_limit_offset_test.go:298: PASS: WHERE with empty string value - Row count: 0 (not validated) === RUN TestSQLFilteringLimitOffset/Small_Result_Set sql_filtering_limit_offset_test.go:295: PASS: Optimized query: specific WHERE + LIMIT 1 - Correct row count: 1 === RUN TestSQLFilteringLimitOffset/Batch_Processing sql_filtering_limit_offset_test.go:298: PASS: Batch processing pattern: moderate LIMIT - Row count: 10 (not validated) === NAME TestSQLFilteringLimitOffset sql_filtering_limit_offset_test.go:307: ================================================================================ sql_filtering_limit_offset_test.go:308: SQL FILTERING, LIMIT & OFFSET TEST SUITE SUMMARY sql_filtering_limit_offset_test.go:309: ================================================================================ sql_filtering_limit_offset_test.go:310: Total Tests: 31 sql_filtering_limit_offset_test.go:311: Successful: 31 sql_filtering_limit_offset_test.go:312: Errors: 0 sql_filtering_limit_offset_test.go:313: Row Count Mismatches: 0 sql_filtering_limit_offset_test.go:314: ================================================================================ --- PASS: TestSQLFilteringLimitOffset (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Equals_Integer (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Equals_String (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Not_Equals (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Greater_Than (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Less_Than (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Greater_Equal (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Less_Equal (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Column_Comparison (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_With_Function (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_With_Arithmetic (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_1 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_5 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_0 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_Large (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_With_Columns (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_With_Functions (0.00s) --- PASS: TestSQLFilteringLimitOffset/Offset_0 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Offset_1 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Offset_5 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Offset_Large (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_Offset_Pagination_Page1 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_Offset_Pagination_Page2 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_Offset_Pagination_Page3 (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Limit (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Limit_Offset (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Complex_Limit (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_No_Match (0.00s) --- PASS: TestSQLFilteringLimitOffset/Limit_Offset_Beyond_Data (0.00s) --- PASS: TestSQLFilteringLimitOffset/Where_Empty_String (0.00s) --- PASS: TestSQLFilteringLimitOffset/Small_Result_Set (0.00s) --- PASS: TestSQLFilteringLimitOffset/Batch_Processing (0.00s) === RUN TestSQLFilteringAccuracy sql_filtering_limit_offset_test.go:335: Testing SQL filtering accuracy with specific data verification sql_filtering_limit_offset_test.go:350: PASS: Exact ID filtering works correctly sql_filtering_limit_offset_test.go:363: PASS: LIMIT 3 returns exactly 3 rows sql_filtering_limit_offset_test.go:380: PASS: OFFSET 1 correctly skips first row --- PASS: TestSQLFilteringAccuracy (0.00s) === RUN TestSQLFilteringEdgeCases === RUN TestSQLFilteringEdgeCases/Zero_Limit sql_filtering_limit_offset_test.go:441: PASS: LIMIT 0 should return empty result set - Rows: 0 === RUN TestSQLFilteringEdgeCases/Large_Offset sql_filtering_limit_offset_test.go:441: PASS: Very large OFFSET should handle gracefully - Rows: 0 === RUN TestSQLFilteringEdgeCases/Where_False_Condition sql_filtering_limit_offset_test.go:431: UNEXPECTED SUCCESS: WHERE with always-false condition (may indicate feature is implemented) === RUN TestSQLFilteringEdgeCases/Complex_Where sql_filtering_limit_offset_test.go:431: UNEXPECTED SUCCESS: Complex WHERE with AND condition (may indicate feature is implemented) --- PASS: TestSQLFilteringEdgeCases (0.00s) --- PASS: TestSQLFilteringEdgeCases/Zero_Limit (0.00s) --- PASS: TestSQLFilteringEdgeCases/Large_Offset (0.00s) --- PASS: TestSQLFilteringEdgeCases/Where_False_Condition (0.00s) --- PASS: TestSQLFilteringEdgeCases/Complex_Where (0.00s) === RUN TestSQLEngine_StringConcatenationWithLiterals === RUN TestSQLEngine_StringConcatenationWithLiterals/Simple_concatenation_with_literals string_concatenation_test.go:108: Query: SELECT 'test' || action || 'end' FROM user_events LIMIT 1 string_concatenation_test.go:109: Columns: ['test'||action||'end'] string_concatenation_test.go:115: Row 0: [testloginend] === RUN TestSQLEngine_StringConcatenationWithLiterals/User's_original_complex_concatenation string_concatenation_test.go:92: Expected column 0 to be ''test'||action||'xxx'||action||'~~~'||status', got ''test'||action||'xxx'||action||' ~~~ '||status' string_concatenation_test.go:108: Query: SELECT 'test' || action || 'xxx' || action || ' ~~~ ' || status FROM user_events LIMIT 1 string_concatenation_test.go:109: Columns: ['test'||action||'xxx'||action||' ~~~ '||status] string_concatenation_test.go:115: Row 0: [testloginxxxlogin ~~~ active] === RUN TestSQLEngine_StringConcatenationWithLiterals/Mixed_columns_and_literals string_concatenation_test.go:108: Query: SELECT status || '=' || action, 'prefix:' || user_type FROM user_events LIMIT 1 string_concatenation_test.go:109: Columns: [status||'='||action 'prefix:'||user_type] string_concatenation_test.go:115: Row 0: [active=login prefix:premium] === RUN TestSQLEngine_StringConcatenationWithLiterals/Concatenation_with_spaces_in_literals string_concatenation_test.go:92: Expected column 0 to be ''['||status||']'', got '' [ '||status||' ] '' string_concatenation_test.go:108: Query: SELECT ' [ ' || status || ' ] ' FROM user_events LIMIT 2 string_concatenation_test.go:109: Columns: [' [ '||status||' ] '] string_concatenation_test.go:115: Row 0: [ [ active ] ] string_concatenation_test.go:115: Row 1: [ [ pending ] ] --- PASS: TestSQLEngine_StringConcatenationWithLiterals (0.00s) --- PASS: TestSQLEngine_StringConcatenationWithLiterals/Simple_concatenation_with_literals (0.00s) --- PASS: TestSQLEngine_StringConcatenationWithLiterals/User's_original_complex_concatenation (0.00s) --- PASS: TestSQLEngine_StringConcatenationWithLiterals/Mixed_columns_and_literals (0.00s) --- PASS: TestSQLEngine_StringConcatenationWithLiterals/Concatenation_with_spaces_in_literals (0.00s) === RUN TestSQLEngine_StringConcatenationBugReproduction string_concatenation_test.go:180: SUCCESS: Complex string concatenation works correctly! string_concatenation_test.go:181: Query: SELECT UPPER(status), id*2, 'test' || action || 'xxx' || action || ' ~~~ ' || status FROM user_events LIMIT 2 string_concatenation_test.go:188: Row 0: [ACTIVE 164920 testloginxxxlogin ~~~ active] string_concatenation_test.go:188: Row 1: [PENDING 1682512 testclickxxxclick ~~~ pending] --- PASS: TestSQLEngine_StringConcatenationBugReproduction (0.00s) === RUN TestStringFunctions === RUN TestStringFunctions/LENGTH_function_tests === RUN TestStringFunctions/LENGTH_function_tests/Length_of_string === RUN TestStringFunctions/LENGTH_function_tests/Length_of_empty_string === RUN TestStringFunctions/LENGTH_function_tests/Length_of_number === RUN TestStringFunctions/LENGTH_function_tests/Length_of_null_value === RUN TestStringFunctions/UPPER/LOWER_function_tests === RUN TestStringFunctions/TRIM_function_tests === RUN TestStringFunctions/TRIM_function_tests/TRIM_whitespace === RUN TestStringFunctions/TRIM_function_tests/LTRIM_whitespace === RUN TestStringFunctions/TRIM_function_tests/RTRIM_whitespace === RUN TestStringFunctions/TRIM_function_tests/TRIM_with_tabs_and_newlines === RUN TestStringFunctions/SUBSTRING_function_tests === RUN TestStringFunctions/CONCAT_function_tests === RUN TestStringFunctions/REPLACE_function_tests === RUN TestStringFunctions/POSITION_function_tests === RUN TestStringFunctions/LEFT/RIGHT_function_tests === RUN TestStringFunctions/REVERSE_function_tests --- PASS: TestStringFunctions (0.00s) --- PASS: TestStringFunctions/LENGTH_function_tests (0.00s) --- PASS: TestStringFunctions/LENGTH_function_tests/Length_of_string (0.00s) --- PASS: TestStringFunctions/LENGTH_function_tests/Length_of_empty_string (0.00s) --- PASS: TestStringFunctions/LENGTH_function_tests/Length_of_number (0.00s) --- PASS: TestStringFunctions/LENGTH_function_tests/Length_of_null_value (0.00s) --- PASS: TestStringFunctions/UPPER/LOWER_function_tests (0.00s) --- PASS: TestStringFunctions/TRIM_function_tests (0.00s) --- PASS: TestStringFunctions/TRIM_function_tests/TRIM_whitespace (0.00s) --- PASS: TestStringFunctions/TRIM_function_tests/LTRIM_whitespace (0.00s) --- PASS: TestStringFunctions/TRIM_function_tests/RTRIM_whitespace (0.00s) --- PASS: TestStringFunctions/TRIM_function_tests/TRIM_with_tabs_and_newlines (0.00s) --- PASS: TestStringFunctions/SUBSTRING_function_tests (0.00s) --- PASS: TestStringFunctions/CONCAT_function_tests (0.00s) --- PASS: TestStringFunctions/REPLACE_function_tests (0.00s) --- PASS: TestStringFunctions/POSITION_function_tests (0.00s) --- PASS: TestStringFunctions/LEFT/RIGHT_function_tests (0.00s) --- PASS: TestStringFunctions/REVERSE_function_tests (0.00s) === RUN TestStringFunctionsSQL === RUN TestStringFunctionsSQL/UPPER_function === RUN TestStringFunctionsSQL/LOWER_function === RUN TestStringFunctionsSQL/LENGTH_function === RUN TestStringFunctionsSQL/TRIM_function === RUN TestStringFunctionsSQL/LTRIM_function === RUN TestStringFunctionsSQL/RTRIM_function === RUN TestStringFunctionsSQL/Multiple_string_functions === RUN TestStringFunctionsSQL/String_function_with_wrong_argument_count === RUN TestStringFunctionsSQL/String_function_with_no_arguments --- PASS: TestStringFunctionsSQL (0.00s) --- PASS: TestStringFunctionsSQL/UPPER_function (0.00s) --- PASS: TestStringFunctionsSQL/LOWER_function (0.00s) --- PASS: TestStringFunctionsSQL/LENGTH_function (0.00s) --- PASS: TestStringFunctionsSQL/TRIM_function (0.00s) --- PASS: TestStringFunctionsSQL/LTRIM_function (0.00s) --- PASS: TestStringFunctionsSQL/RTRIM_function (0.00s) --- PASS: TestStringFunctionsSQL/Multiple_string_functions (0.00s) --- PASS: TestStringFunctionsSQL/String_function_with_wrong_argument_count (0.00s) --- PASS: TestStringFunctionsSQL/String_function_with_no_arguments (0.00s) === RUN TestSQLEngine_StringFunctionsAndLiterals === RUN TestSQLEngine_StringFunctionsAndLiterals/String_functions_-_UPPER_and_LENGTH string_literal_function_test.go:48: Status: 'active', UPPER: 'ACTIVE', LENGTH: '6' string_literal_function_test.go:160: Query: SELECT status, UPPER(status), LENGTH(status) FROM user_events LIMIT 3 string_literal_function_test.go:161: Columns: [status UPPER(status) LENGTH(status)] string_literal_function_test.go:167: Row 0: [active ACTIVE 6] string_literal_function_test.go:167: Row 1: [pending PENDING 7] string_literal_function_test.go:167: Row 2: [ ] === RUN TestSQLEngine_StringFunctionsAndLiterals/String_literal_in_SELECT string_literal_function_test.go:160: Query: SELECT id, user_id, 'good' FROM user_events LIMIT 2 string_literal_function_test.go:161: Columns: [id user_id 'good'] string_literal_function_test.go:167: Row 0: [82460 9465 good] string_literal_function_test.go:167: Row 1: [841256 2336 good] === RUN TestSQLEngine_StringFunctionsAndLiterals/Mixed:_columns,_functions,_arithmetic,_and_literals string_literal_function_test.go:160: Query: SELECT id, UPPER(status), id*2, 'test' FROM user_events LIMIT 2 string_literal_function_test.go:161: Columns: [id UPPER(status) id*2 'test'] string_literal_function_test.go:167: Row 0: [82460 ACTIVE 164920 test] string_literal_function_test.go:167: Row 1: [841256 PENDING 1682512 test] === RUN TestSQLEngine_StringFunctionsAndLiterals/User's_original_failing_query_-_fixed string_literal_function_test.go:160: Query: SELECT status, action, user_type, UPPER(action), LENGTH(action) FROM user_events LIMIT 2 string_literal_function_test.go:161: Columns: [status action user_type UPPER(action) LENGTH(action)] string_literal_function_test.go:167: Row 0: [active login premium LOGIN 5] string_literal_function_test.go:167: Row 1: [pending click standard CLICK 5] --- PASS: TestSQLEngine_StringFunctionsAndLiterals (0.00s) --- PASS: TestSQLEngine_StringFunctionsAndLiterals/String_functions_-_UPPER_and_LENGTH (0.00s) --- PASS: TestSQLEngine_StringFunctionsAndLiterals/String_literal_in_SELECT (0.00s) --- PASS: TestSQLEngine_StringFunctionsAndLiterals/Mixed:_columns,_functions,_arithmetic,_and_literals (0.00s) --- PASS: TestSQLEngine_StringFunctionsAndLiterals/User's_original_failing_query_-_fixed (0.00s) === RUN TestSQLEngine_StringFunctionErrorHandling string_literal_function_test.go:186: UPPER function works correctly string_literal_function_test.go:197: LENGTH function works correctly --- PASS: TestSQLEngine_StringFunctionErrorHandling (0.00s) === RUN TestTimestampIntegrationScenarios === RUN TestTimestampIntegrationScenarios/EndToEndTimestampEquality === RUN TestTimestampIntegrationScenarios/EndToEndTimestampEquality/original_failing_1 === RUN TestTimestampIntegrationScenarios/EndToEndTimestampEquality/original_failing_2 === RUN TestTimestampIntegrationScenarios/EndToEndTimestampEquality/current_data === RUN TestTimestampIntegrationScenarios/ComplexRangeQueries === RUN TestTimestampIntegrationScenarios/ComplexRangeQueries/RangeWithDifferentBounds === RUN TestTimestampIntegrationScenarios/ComplexRangeQueries/RangeWithSameBounds === RUN TestTimestampIntegrationScenarios/ComplexRangeQueries/OpenEndedRange === RUN TestTimestampIntegrationScenarios/ProductionScenarioReproduction --- PASS: TestTimestampIntegrationScenarios (0.00s) --- PASS: TestTimestampIntegrationScenarios/EndToEndTimestampEquality (0.00s) --- PASS: TestTimestampIntegrationScenarios/EndToEndTimestampEquality/original_failing_1 (0.00s) --- PASS: TestTimestampIntegrationScenarios/EndToEndTimestampEquality/original_failing_2 (0.00s) --- PASS: TestTimestampIntegrationScenarios/EndToEndTimestampEquality/current_data (0.00s) --- PASS: TestTimestampIntegrationScenarios/ComplexRangeQueries (0.00s) --- PASS: TestTimestampIntegrationScenarios/ComplexRangeQueries/RangeWithDifferentBounds (0.00s) --- PASS: TestTimestampIntegrationScenarios/ComplexRangeQueries/RangeWithSameBounds (0.00s) --- PASS: TestTimestampIntegrationScenarios/ComplexRangeQueries/OpenEndedRange (0.00s) --- PASS: TestTimestampIntegrationScenarios/ProductionScenarioReproduction (0.00s) === RUN TestRegressionPrevention === RUN TestRegressionPrevention/SmallTimestamps === RUN TestRegressionPrevention/NonTimestampColumns === RUN TestRegressionPrevention/StringComparisons --- PASS: TestRegressionPrevention (0.00s) --- PASS: TestRegressionPrevention/SmallTimestamps (0.00s) --- PASS: TestRegressionPrevention/NonTimestampColumns (0.00s) --- PASS: TestRegressionPrevention/StringComparisons (0.00s) === RUN TestTimestampQueryFixes === RUN TestTimestampQueryFixes/Fix1_PrecisionLoss === RUN TestTimestampQueryFixes/Fix2_TimeFilterExtraction === RUN TestTimestampQueryFixes/Fix3_RangeBoundaryFix === RUN TestTimestampQueryFixes/Fix4_DifferentRangeBoundaries === RUN TestTimestampQueryFixes/Fix5_PredicateAccuracy === RUN TestTimestampQueryFixes/Fix6_ComparisonOperators === RUN TestTimestampQueryFixes/Fix7_EdgeCases --- PASS: TestTimestampQueryFixes (0.00s) --- PASS: TestTimestampQueryFixes/Fix1_PrecisionLoss (0.00s) --- PASS: TestTimestampQueryFixes/Fix2_TimeFilterExtraction (0.00s) --- PASS: TestTimestampQueryFixes/Fix3_RangeBoundaryFix (0.00s) --- PASS: TestTimestampQueryFixes/Fix4_DifferentRangeBoundaries (0.00s) --- PASS: TestTimestampQueryFixes/Fix5_PredicateAccuracy (0.00s) --- PASS: TestTimestampQueryFixes/Fix6_ComparisonOperators (0.00s) --- PASS: TestTimestampQueryFixes/Fix7_EdgeCases (0.00s) === RUN TestOriginalFailingQueries === RUN TestOriginalFailingQueries/OriginalQuery1 === RUN TestOriginalFailingQueries/OriginalQuery2 === RUN TestOriginalFailingQueries/CurrentDataQuery --- PASS: TestOriginalFailingQueries (0.00s) --- PASS: TestOriginalFailingQueries/OriginalQuery1 (0.00s) --- PASS: TestOriginalFailingQueries/OriginalQuery2 (0.00s) --- PASS: TestOriginalFailingQueries/CurrentDataQuery (0.00s) === RUN TestWhereParsing === RUN TestWhereParsing/Simple_Equals where_clause_debug_test.go:78: PASS: WHERE clause parsed successfully for: Simple equality WHERE clause where_clause_debug_test.go:79: WHERE expression type: *engine.ComparisonExpr === RUN TestWhereParsing/Greater_Than where_clause_debug_test.go:78: PASS: WHERE clause parsed successfully for: Greater than WHERE clause where_clause_debug_test.go:79: WHERE expression type: *engine.ComparisonExpr === RUN TestWhereParsing/String_Equals where_clause_debug_test.go:78: PASS: WHERE clause parsed successfully for: String equality WHERE clause where_clause_debug_test.go:79: WHERE expression type: *engine.ComparisonExpr === RUN TestWhereParsing/Impossible_Condition where_clause_debug_test.go:78: PASS: WHERE clause parsed successfully for: Impossible WHERE condition (should parse but return no rows) where_clause_debug_test.go:79: WHERE expression type: *engine.ComparisonExpr --- PASS: TestWhereParsing (0.00s) --- PASS: TestWhereParsing/Simple_Equals (0.00s) --- PASS: TestWhereParsing/Greater_Than (0.00s) --- PASS: TestWhereParsing/String_Equals (0.00s) --- PASS: TestWhereParsing/Impossible_Condition (0.00s) === RUN TestPredicateBuilding === RUN TestPredicateBuilding/Simple_Equals_Match where_clause_debug_test.go:172: PASS: Simple equality - should match - Predicate worked correctly (match=true) === RUN TestPredicateBuilding/Simple_Equals_NoMatch where_clause_debug_test.go:172: PASS: Simple equality - should not match - Predicate worked correctly (match=false) === RUN TestPredicateBuilding/Greater_Than_Match where_clause_debug_test.go:172: PASS: Greater than - should match - Predicate worked correctly (match=true) === RUN TestPredicateBuilding/Greater_Than_NoMatch where_clause_debug_test.go:172: PASS: Greater than - should not match - Predicate worked correctly (match=false) === RUN TestPredicateBuilding/String_Equals_Match where_clause_debug_test.go:172: PASS: String equality - should match - Predicate worked correctly (match=true) === RUN TestPredicateBuilding/String_Equals_NoMatch where_clause_debug_test.go:172: PASS: String equality - should not match - Predicate worked correctly (match=false) === RUN TestPredicateBuilding/Impossible_Condition where_clause_debug_test.go:172: PASS: Impossible condition - should never match - Predicate worked correctly (match=false) --- PASS: TestPredicateBuilding (0.00s) --- PASS: TestPredicateBuilding/Simple_Equals_Match (0.00s) --- PASS: TestPredicateBuilding/Simple_Equals_NoMatch (0.00s) --- PASS: TestPredicateBuilding/Greater_Than_Match (0.00s) --- PASS: TestPredicateBuilding/Greater_Than_NoMatch (0.00s) --- PASS: TestPredicateBuilding/String_Equals_Match (0.00s) --- PASS: TestPredicateBuilding/String_Equals_NoMatch (0.00s) --- PASS: TestPredicateBuilding/Impossible_Condition (0.00s) === RUN TestWhereClauseEndToEnd where_clause_debug_test.go:185: END-TO-END WHERE CLAUSE VALIDATION where_clause_debug_test.go:186: =================================== where_clause_debug_test.go:194: Baseline (no WHERE): 10 rows where_clause_debug_test.go:202: WHERE 1 = 0: 0 rows where_clause_debug_test.go:210: Impossible WHERE condition correctly returns 0 rows where_clause_debug_test.go:222: WHERE id = 82460: 1 rows where_clause_debug_test.go:227: Specific ID WHERE clause working correctly where_clause_debug_test.go:239: WHERE id > 10000000: 0 rows where_clause_debug_test.go:256: WHERE id > 10000000 correctly filtered results --- PASS: TestWhereClauseEndToEnd (0.00s) === RUN TestSpecificWhereClauseBug where_clause_debug_test.go:298: REPRODUCING EXACT WHERE CLAUSE BUG where_clause_debug_test.go:299: ================================== where_clause_debug_test.go:309: Query: SELECT id FROM user_events WHERE id > 10000000 LIMIT 10 OFFSET 5 where_clause_debug_test.go:310: Returned 0 rows: where_clause_debug_test.go:326: WHERE clause working correctly - all IDs > 10,000,000 --- PASS: TestSpecificWhereClauseBug (0.00s) === RUN TestWhereClauseValidation where_validation_test.go:13: WHERE CLAUSE VALIDATION TESTS where_validation_test.go:14: ============================== where_validation_test.go:22: Baseline data - Total rows: 10 where_validation_test.go:24: Sample IDs: 82460, 841256, 55537 where_validation_test.go:38: WHERE id = 82460: 1 rows where_validation_test.go:40: Specific ID filtering works correctly where_validation_test.go:59: Data range: min ID = 55537, max ID = 841256 where_validation_test.go:69: WHERE id > 448396: 2 rows where_validation_test.go:83: Range filtering works correctly - all returned IDs > 448396 where_validation_test.go:95: WHERE status = 'active': 1 rows where_validation_test.go:107: String filtering works correctly where_validation_test.go:111: TESTING REAL-WORLD CASE: where_validation_test.go:118: Real-world query returned: 0 rows where_validation_test.go:131: Real-world case FIXED: No violations found --- PASS: TestWhereClauseValidation (0.00s) === RUN TestWhereClauseComparisonOperators where_validation_test.go:164: Testing comparison operators with ID = 841256 where_validation_test.go:175: WHERE id = 841256: 1 rows (equals) where_validation_test.go:175: WHERE id != 841256: 9 rows (not equals) where_validation_test.go:175: WHERE id > 841256: 0 rows (greater than) where_validation_test.go:175: WHERE id < 841256: 9 rows (less than) where_validation_test.go:175: WHERE id >= 841256: 1 rows (greater or equal) where_validation_test.go:175: WHERE id <= 841256: 10 rows (less or equal) --- PASS: TestWhereClauseComparisonOperators (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/query/engine 0.133s === RUN TestGjson { "quiz": { "sport": { "q1": { "question": "Which one is correct team name in NBA?", "options": [ "New York Bulls", "Los Angeles Kings", "Golden State Warriros", "Huston Rocket" ], "answer": "Huston Rocket" } }, "maths": { "q1": { "question": "5 + 7 = ?", "options": [ "10", "11", "12", "13" ], "answer": "12" }, "q2": { "question": "12 - 8 = ?", "options": [ "1", "2", "3", "4" ], "answer": "4" } } } } +++++++++++ 12 5 { "sport": { "q1": { "question": "Which one is correct team name in NBA?", "options": [ "New York Bulls", "Los Angeles Kings", "Golden State Warriros", "Huston Rocket" ], "answer": "Huston Rocket" } }, "maths": { "q1": { "question": "5 + 7 = ?", "options": [ "10", "11", "12", "13" ], "answer": "12" }, "q2": { "question": "12 - 8 = ?", "options": [ "1", "2", "3", "4" ], "answer": "4" } } } 0 0 ----------- { "fruit": "Apple", "size": "Large", "quiz": "Red" } +++++++++++ 51 3 Red 13 3 Apple ----------- --- PASS: TestGjson (0.00s) === RUN TestJsonQueryRow {fruit:"Bl\"ue",size:6} --- PASS: TestJsonQueryRow (0.00s) === RUN TestJsonQueryNumber {fruit:"Bl\"ue",quiz:"green"} --- PASS: TestJsonQueryNumber (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/query/json 0.016s ? github.com/seaweedfs/seaweedfs/weed/query/sqltypes [no test files] ? github.com/seaweedfs/seaweedfs/weed/remote_storage [no test files] === RUN TestAzureStorageClientBasic azure_storage_client_test.go:26: Skipping Azure storage test: AZURE_STORAGE_ACCOUNT or AZURE_STORAGE_ACCESS_KEY not set --- SKIP: TestAzureStorageClientBasic (0.00s) === RUN TestToMetadata === RUN TestToMetadata/basic_metadata === RUN TestToMetadata/metadata_with_dashes === RUN TestToMetadata/non-metadata_keys_ignored === RUN TestToMetadata/keys_starting_with_digits === RUN TestToMetadata/uppercase_and_mixed_case_keys === RUN TestToMetadata/keys_with_invalid_characters === RUN TestToMetadata/collision_prevention === RUN TestToMetadata/empty_input --- PASS: TestToMetadata (0.00s) --- PASS: TestToMetadata/basic_metadata (0.00s) --- PASS: TestToMetadata/metadata_with_dashes (0.00s) --- PASS: TestToMetadata/non-metadata_keys_ignored (0.00s) --- PASS: TestToMetadata/keys_starting_with_digits (0.00s) --- PASS: TestToMetadata/uppercase_and_mixed_case_keys (0.00s) --- PASS: TestToMetadata/keys_with_invalid_characters (0.00s) --- PASS: TestToMetadata/collision_prevention (0.00s) --- PASS: TestToMetadata/empty_input (0.00s) === RUN TestAzureRemoteStorageMaker --- PASS: TestAzureRemoteStorageMaker (0.00s) === RUN TestAzureStorageClientErrors --- PASS: TestAzureStorageClientErrors (12.92s) === RUN TestAzureRemoteStorageClientImplementsInterface --- PASS: TestAzureRemoteStorageClientImplementsInterface (0.00s) === RUN TestAzureErrRemoteObjectNotFoundIsAccessible --- PASS: TestAzureErrRemoteObjectNotFoundIsAccessible (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/remote_storage/azure 12.981s === RUN TestGCSRemoteStorageClientImplementsInterface --- PASS: TestGCSRemoteStorageClientImplementsInterface (0.00s) === RUN TestGCSErrRemoteObjectNotFoundIsAccessible --- PASS: TestGCSErrRemoteObjectNotFoundIsAccessible (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/remote_storage/gcs 0.044s === RUN TestS3MakeUsesAnonymousCredentialsWhenKeysAreEmpty --- PASS: TestS3MakeUsesAnonymousCredentialsWhenKeysAreEmpty (0.00s) === RUN TestS3MakeUsesStaticCredentialsWhenKeysAreProvided --- PASS: TestS3MakeUsesStaticCredentialsWhenKeysAreProvided (0.00s) === RUN TestS3RemoteStorageClientImplementsInterface --- PASS: TestS3RemoteStorageClientImplementsInterface (0.00s) === RUN TestS3ErrRemoteObjectNotFoundIsAccessible --- PASS: TestS3ErrRemoteObjectNotFoundIsAccessible (0.00s) === RUN TestS3WriteFilePassesMimeAsContentType --- PASS: TestS3WriteFilePassesMimeAsContentType (0.00s) === RUN TestS3WriteFileOmitsContentTypeWhenMimeMissing --- PASS: TestS3WriteFileOmitsContentTypeWhenMimeMissing (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/remote_storage/s3 0.060s === RUN TestReplicateRenameUsesTargetKeyForNonFilerSink --- PASS: TestReplicateRenameUsesTargetKeyForNonFilerSink (0.00s) === RUN TestReplicateRenameWithoutMoverUsesCreateThenDelete --- PASS: TestReplicateRenameWithoutMoverUsesCreateThenDelete (0.00s) === RUN TestReplicateRenameUsesMoveEntryWhenSupported --- PASS: TestReplicateRenameUsesMoveEntryWhenSupported (0.00s) === RUN TestReplicateInPlaceUpdateUsesUpdateEntry --- PASS: TestReplicateInPlaceUpdateUsesUpdateEntry (0.00s) === RUN TestReplicateInPlaceUpdateFallbackCreates --- PASS: TestReplicateInPlaceUpdateFallbackCreates (0.00s) === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/equal === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/descendant === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/sibling_prefix === RUN TestPathIsEqualOrUnderUsesDirectoryBoundaries/root --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/equal (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/descendant (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/sibling_prefix (0.00s) --- PASS: TestPathIsEqualOrUnderUsesDirectoryBoundaries/root (0.00s) === RUN TestReplicateRenameOutToSiblingPrefixBecomesDelete --- PASS: TestReplicateRenameOutToSiblingPrefixBecomesDelete (0.00s) === RUN TestReplicateRenameFromExcludedDirBecomesCreate --- PASS: TestReplicateRenameFromExcludedDirBecomesCreate (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication 0.130s === RUN TestDetectSSEType === RUN TestDetectSSEType/no_chunks_no_extended === RUN TestDetectSSEType/plaintext_chunks === RUN TestDetectSSEType/uniform_SSE-S3_chunks === RUN TestDetectSSEType/uniform_SSE-KMS_chunks === RUN TestDetectSSEType/mixed_chunk_SSE_types === RUN TestDetectSSEType/inline_SSE-S3_via_extended === RUN TestDetectSSEType/inline_SSE-KMS_via_extended === RUN TestDetectSSEType/inline_SSE-C_via_extended === RUN TestDetectSSEType/conflicting_extended_metadata === RUN TestDetectSSEType/chunks_take_precedence_over_extended --- PASS: TestDetectSSEType (0.00s) --- PASS: TestDetectSSEType/no_chunks_no_extended (0.00s) --- PASS: TestDetectSSEType/plaintext_chunks (0.00s) --- PASS: TestDetectSSEType/uniform_SSE-S3_chunks (0.00s) --- PASS: TestDetectSSEType/uniform_SSE-KMS_chunks (0.00s) --- PASS: TestDetectSSEType/mixed_chunk_SSE_types (0.00s) --- PASS: TestDetectSSEType/inline_SSE-S3_via_extended (0.00s) --- PASS: TestDetectSSEType/inline_SSE-KMS_via_extended (0.00s) --- PASS: TestDetectSSEType/inline_SSE-C_via_extended (0.00s) --- PASS: TestDetectSSEType/conflicting_extended_metadata (0.00s) --- PASS: TestDetectSSEType/chunks_take_precedence_over_extended (0.00s) === RUN TestMaybeDecryptReader_Plaintext --- PASS: TestMaybeDecryptReader_Plaintext (0.00s) === RUN TestMaybeDecryptReader_NilEntry --- PASS: TestMaybeDecryptReader_NilEntry (0.00s) === RUN TestMaybeDecryptReader_SSEC_Error --- PASS: TestMaybeDecryptReader_SSEC_Error (0.00s) === RUN TestMaybeDecryptContent_Plaintext --- PASS: TestMaybeDecryptContent_Plaintext (0.00s) === RUN TestMaybeDecryptContent_NilEntry --- PASS: TestMaybeDecryptContent_NilEntry (0.00s) === RUN TestMaybeDecryptContent_Empty --- PASS: TestMaybeDecryptContent_Empty (0.00s) === RUN TestMaybeDecryptContent_SSEC_Error --- PASS: TestMaybeDecryptContent_SSEC_Error (0.00s) === RUN TestMaybeDecryptContent_MixedExtended_Error --- PASS: TestMaybeDecryptContent_MixedExtended_Error (0.00s) === RUN TestMaybeDecryptReader_SSES3 I0720 21:30:02.302959 s3_sse_s3.go:530 SSE-S3 KeyManager: Loaded KEK from s3.sse.kek config I0720 21:30:02.303189 s3_sse_s3.go:567 SSE-S3 KeyManager: No KEK configured. SSE-S3 encryption is disabled. Set s3.sse.kek or s3.sse.key in security.toml to enable it. --- PASS: TestMaybeDecryptReader_SSES3 (0.00s) === RUN TestMaybeDecryptContent_SSES3 I0720 21:30:02.303223 s3_sse_s3.go:530 SSE-S3 KeyManager: Loaded KEK from s3.sse.kek config I0720 21:30:02.303245 s3_sse_s3.go:567 SSE-S3 KeyManager: No KEK configured. SSE-S3 encryption is disabled. Set s3.sse.kek or s3.sse.key in security.toml to enable it. --- PASS: TestMaybeDecryptContent_SSES3 (0.00s) === RUN TestMaybeDecryptReader_SSEKMS --- PASS: TestMaybeDecryptReader_SSEKMS (0.00s) === RUN TestMaybeDecryptContent_SSEKMS --- PASS: TestMaybeDecryptContent_SSEKMS (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication/repl_util 0.130s ? github.com/seaweedfs/seaweedfs/weed/replication/sink [no test files] === RUN TestAzureSinkInterface --- PASS: TestAzureSinkInterface (0.00s) === RUN TestAzureSinkInitialization azure_sink_test.go:99: Skipping Azure sink test: AZURE_STORAGE_ACCOUNT or AZURE_STORAGE_ACCESS_KEY not set --- SKIP: TestAzureSinkInitialization (0.00s) === RUN TestAzureSinkInitializeFromConfig azure_sink_test.go:132: Skipping Azure sink config test: AZURE_STORAGE_ACCOUNT or AZURE_STORAGE_ACCESS_KEY not set --- SKIP: TestAzureSinkInitializeFromConfig (0.00s) === RUN TestCleanKey === RUN TestCleanKey//test/file.txt === RUN TestCleanKey/test/file.txt === RUN TestCleanKey// === RUN TestCleanKey/#00 === RUN TestCleanKey//a/b/c --- PASS: TestCleanKey (0.00s) --- PASS: TestCleanKey//test/file.txt (0.00s) --- PASS: TestCleanKey/test/file.txt (0.00s) --- PASS: TestCleanKey// (0.00s) --- PASS: TestCleanKey/#00 (0.00s) --- PASS: TestCleanKey//a/b/c (0.00s) === RUN TestAzureSinkEntryOperations azure_sink_test.go:186: Skipping Azure sink entry test: credentials not set --- SKIP: TestAzureSinkEntryOperations (0.00s) === RUN TestAzureSinkPrecondition azure_sink_test.go:281: Skipping Azure sink precondition test: credentials not set --- SKIP: TestAzureSinkPrecondition (0.00s) === RUN TestAzureSinkIdempotentCreate azure_sink_test.go:350: Skipping Azure sink idempotent create test: credentials not set --- SKIP: TestAzureSinkIdempotentCreate (0.00s) === RUN TestAzureSinkInvalidCredentials azure_sink_test.go:465: Invalid credentials correctly rejected at initialization --- SKIP: TestAzureSinkInvalidCredentials (12.92s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication/sink/azuresink 12.989s ? github.com/seaweedfs/seaweedfs/weed/replication/sink/b2sink [no test files] === RUN TestTargetPathToSourcePath === RUN TestTargetPathToSourcePath/basic_mapping === RUN TestTargetPathToSourcePath/incremental_sink_is_unmappable === RUN TestTargetPathToSourcePath/trailing_slash_roots === RUN TestTargetPathToSourcePath/root_target_mapping === RUN TestTargetPathToSourcePath/target_root_itself === RUN TestTargetPathToSourcePath/outside_target_root --- PASS: TestTargetPathToSourcePath (0.01s) --- PASS: TestTargetPathToSourcePath/basic_mapping (0.00s) --- PASS: TestTargetPathToSourcePath/incremental_sink_is_unmappable (0.00s) --- PASS: TestTargetPathToSourcePath/trailing_slash_roots (0.00s) --- PASS: TestTargetPathToSourcePath/root_target_mapping (0.00s) --- PASS: TestTargetPathToSourcePath/target_root_itself (0.00s) --- PASS: TestTargetPathToSourcePath/outside_target_root (0.00s) === RUN TestValidateReplicatedChunkSize === RUN TestValidateReplicatedChunkSize/healthy === RUN TestValidateReplicatedChunkSize/legitimately_empty_file === RUN TestValidateReplicatedChunkSize/zero-byte_read_for_non-empty_source === RUN TestValidateReplicatedChunkSize/short_read === RUN TestValidateReplicatedChunkSize/over-read_(server_returned_more_than_metadata) --- PASS: TestValidateReplicatedChunkSize (0.00s) --- PASS: TestValidateReplicatedChunkSize/healthy (0.00s) --- PASS: TestValidateReplicatedChunkSize/legitimately_empty_file (0.00s) --- PASS: TestValidateReplicatedChunkSize/zero-byte_read_for_non-empty_source (0.00s) --- PASS: TestValidateReplicatedChunkSize/short_read (0.00s) --- PASS: TestValidateReplicatedChunkSize/over-read_(server_returned_more_than_metadata) (0.00s) === RUN TestFetchAndWriteRejectsZeroByteSource I0720 21:30:02.242440 fetch_write.go:407 permanent size mismatch replicating 74,047d16a94aa581 for /dst/index.bin: chunk size mismatch: read 74,047d16a94aa581 got 0 bytes, source metadata says 5171 --- PASS: TestFetchAndWriteRejectsZeroByteSource (0.00s) === RUN TestIsRetryableNetworkError === RUN TestIsRetryableNetworkError/nil === RUN TestIsRetryableNetworkError/eof === RUN TestIsRetryableNetworkError/unexpected_eof === RUN TestIsRetryableNetworkError/volume_idle_timeout_json === RUN TestIsRetryableNetworkError/volume_idle_timeout_capitalized === RUN TestIsRetryableNetworkError/connection_reset === RUN TestIsRetryableNetworkError/connection_reset_capitalized === RUN TestIsRetryableNetworkError/broken_pipe === RUN TestIsRetryableNetworkError/broken_pipe_capitalized === RUN TestIsRetryableNetworkError/net.Error_timeout === RUN TestIsRetryableNetworkError/size_mismatch_is_permanent === RUN TestIsRetryableNetworkError/unrelated_error --- PASS: TestIsRetryableNetworkError (0.00s) --- PASS: TestIsRetryableNetworkError/nil (0.00s) --- PASS: TestIsRetryableNetworkError/eof (0.00s) --- PASS: TestIsRetryableNetworkError/unexpected_eof (0.00s) --- PASS: TestIsRetryableNetworkError/volume_idle_timeout_json (0.00s) --- PASS: TestIsRetryableNetworkError/volume_idle_timeout_capitalized (0.00s) --- PASS: TestIsRetryableNetworkError/connection_reset (0.00s) --- PASS: TestIsRetryableNetworkError/connection_reset_capitalized (0.00s) --- PASS: TestIsRetryableNetworkError/broken_pipe (0.00s) --- PASS: TestIsRetryableNetworkError/broken_pipe_capitalized (0.00s) --- PASS: TestIsRetryableNetworkError/net.Error_timeout (0.00s) --- PASS: TestIsRetryableNetworkError/size_mismatch_is_permanent (0.00s) --- PASS: TestIsRetryableNetworkError/unrelated_error (0.00s) === RUN TestReplicateChunksPreservesSizeMismatchSentinel I0720 21:30:02.243226 fetch_write.go:407 permanent size mismatch replicating 74,047d16a94aa581 for /dst/index.bin: chunk size mismatch: read 74,047d16a94aa581 got 0 bytes, source metadata says 5171 --- PASS: TestReplicateChunksPreservesSizeMismatchSentinel (0.00s) === RUN TestSourceSupersedes === RUN TestSourceSupersedes/not-found_sentinel === RUN TestSourceSupersedes/not-found_wrapped === RUN TestSourceSupersedes/not-found_as_string_(gRPC) === RUN TestSourceSupersedes/nil_entry,_nil_error === RUN TestSourceSupersedes/network_error === RUN TestSourceSupersedes/source_strictly_newer === RUN TestSourceSupersedes/source_same_version === RUN TestSourceSupersedes/source_older_(out-of-order_replay) --- PASS: TestSourceSupersedes (0.00s) --- PASS: TestSourceSupersedes/not-found_sentinel (0.00s) --- PASS: TestSourceSupersedes/not-found_wrapped (0.00s) --- PASS: TestSourceSupersedes/not-found_as_string_(gRPC) (0.00s) --- PASS: TestSourceSupersedes/nil_entry,_nil_error (0.00s) --- PASS: TestSourceSupersedes/network_error (0.00s) --- PASS: TestSourceSupersedes/source_strictly_newer (0.00s) --- PASS: TestSourceSupersedes/source_same_version (0.00s) --- PASS: TestSourceSupersedes/source_older_(out-of-order_replay) (0.00s) === RUN TestSourceSupersedesEpochMtime --- PASS: TestSourceSupersedesEpochMtime (0.00s) === RUN TestManifestResolveRetryGateUnverifiableSupersessionBounded I0720 21:30:02.243387 fetch_write.go:223 resolve manifest 3,01abc for /backup/2026-07-10/buckets/x/f.pt: LookupFileId volume id 3: not found I0720 21:30:02.243392 fetch_write.go:223 resolve manifest 3,01abc for /backup/2026-07-10/buckets/x/f.pt: LookupFileId volume id 3: not found I0720 21:30:02.243396 fetch_write.go:220 resolve manifest 3,01abc for /backup/2026-07-10/buckets/x/f.pt: supersession unverifiable, propagating after 3 attempts: LookupFileId volume id 3: not found --- PASS: TestManifestResolveRetryGateUnverifiableSupersessionBounded (0.00s) === RUN TestManifestResolveRetryGateNonTransientPropagates I0720 21:30:02.243412 fetch_write.go:215 resolve manifest 3,01abc for /backup/buckets/x/f.pt: non-transient error, propagating: fail to unmarshal manifest 3,01abc: proto: cannot parse invalid wire-format data I0720 21:30:02.243417 fetch_write.go:215 resolve manifest 3,01abc for /backup/buckets/x/f.pt: non-transient error, propagating: invalid fileId abc I0720 21:30:02.243420 fetch_write.go:223 resolve manifest 3,01abc for /backup/buckets/x/f.pt: LookupFileId volume id 3: not found --- PASS: TestManifestResolveRetryGateNonTransientPropagates (0.00s) === RUN TestGetEntryMtimeNs --- PASS: TestGetEntryMtimeNs (0.00s) === RUN TestOnCorruptChunkRefusesWhenSupersessionUnconfirmed E0720 21:30:02.243447 filer_sink.go:441 refuse to replicate entry with corrupt chunk /buckets/x/config: corrupt chunk: read 0 bytes, source says 107 --- PASS: TestOnCorruptChunkRefusesWhenSupersessionUnconfirmed (0.01s) === RUN TestOnReplicateChunkErrorPropagatesWhenSupersessionUnconfirmed --- PASS: TestOnReplicateChunkErrorPropagatesWhenSupersessionUnconfirmed (0.00s) === RUN TestChooseUpdateAction === RUN TestChooseUpdateAction/nil_existing_entry === RUN TestChooseUpdateAction/destination_older,_catching_up === RUN TestChooseUpdateAction/same_mtime === RUN TestChooseUpdateAction/destination_newer_and_complete === RUN TestChooseUpdateAction/destination_newer_and_larger === RUN TestChooseUpdateAction/newer_smaller_dest,_older_larger_event_->_skip === RUN TestChooseUpdateAction/same_second,_destination_newer_and_complete === RUN TestChooseUpdateAction/same_second,_destination_older === RUN TestChooseUpdateAction/behind,_covers_less_than_oldEntry_->_repair === RUN TestChooseUpdateAction/behind,_matches_oldEntry_coverage_->_normal === RUN TestChooseUpdateAction/behind,_exceeds_oldEntry_coverage_->_normal === RUN TestChooseUpdateAction/behind,_equal_total_but_range_gap_->_repair === RUN TestChooseUpdateAction/older,_stale_backup_chunk_->_repair === RUN TestChooseUpdateAction/equal-mtime_stale_->_deferred_(normal) === RUN TestChooseUpdateAction/older,_rsync_chunks_->_normal === RUN TestChooseUpdateAction/in-sync_(fast_path)_->_normal --- PASS: TestChooseUpdateAction (0.00s) --- PASS: TestChooseUpdateAction/nil_existing_entry (0.00s) --- PASS: TestChooseUpdateAction/destination_older,_catching_up (0.00s) --- PASS: TestChooseUpdateAction/same_mtime (0.00s) --- PASS: TestChooseUpdateAction/destination_newer_and_complete (0.00s) --- PASS: TestChooseUpdateAction/destination_newer_and_larger (0.00s) --- PASS: TestChooseUpdateAction/newer_smaller_dest,_older_larger_event_->_skip (0.00s) --- PASS: TestChooseUpdateAction/same_second,_destination_newer_and_complete (0.00s) --- PASS: TestChooseUpdateAction/same_second,_destination_older (0.00s) --- PASS: TestChooseUpdateAction/behind,_covers_less_than_oldEntry_->_repair (0.00s) --- PASS: TestChooseUpdateAction/behind,_matches_oldEntry_coverage_->_normal (0.00s) --- PASS: TestChooseUpdateAction/behind,_exceeds_oldEntry_coverage_->_normal (0.00s) --- PASS: TestChooseUpdateAction/behind,_equal_total_but_range_gap_->_repair (0.00s) --- PASS: TestChooseUpdateAction/older,_stale_backup_chunk_->_repair (0.00s) --- PASS: TestChooseUpdateAction/equal-mtime_stale_->_deferred_(normal) (0.00s) --- PASS: TestChooseUpdateAction/older,_rsync_chunks_->_normal (0.00s) --- PASS: TestChooseUpdateAction/in-sync_(fast_path)_->_normal (0.00s) === RUN TestCoversReference === RUN TestCoversReference/empty_reference === RUN TestCoversReference/reference_no_chunks === RUN TestCoversReference/exact === RUN TestCoversReference/superset === RUN TestCoversReference/truncated_tail === RUN TestCoversReference/equal_total,_range_gap === RUN TestCoversReference/contiguous_parts_cover === RUN TestCoversReference/overlapping_parts_cover === RUN TestCoversReference/existing_empty,_reference_has_bytes --- PASS: TestCoversReference (0.00s) --- PASS: TestCoversReference/empty_reference (0.00s) --- PASS: TestCoversReference/reference_no_chunks (0.00s) --- PASS: TestCoversReference/exact (0.00s) --- PASS: TestCoversReference/superset (0.00s) --- PASS: TestCoversReference/truncated_tail (0.00s) --- PASS: TestCoversReference/equal_total,_range_gap (0.00s) --- PASS: TestCoversReference/contiguous_parts_cover (0.00s) --- PASS: TestCoversReference/overlapping_parts_cover (0.00s) --- PASS: TestCoversReference/existing_empty,_reference_has_bytes (0.00s) === RUN TestDestinationSatisfiesSource === RUN TestDestinationSatisfiesSource/nil_existing === RUN TestDestinationSatisfiesSource/identical_complete_->_skip === RUN TestDestinationSatisfiesSource/truncated_(no_md5)_->_replicate === RUN TestDestinationSatisfiesSource/md5-backed_truncated_->_replicate === RUN TestDestinationSatisfiesSource/md5_equal_and_complete_->_skip === RUN TestDestinationSatisfiesSource/complete,_dest_newer_->_skip === RUN TestDestinationSatisfiesSource/complete,_dest_older_->_replicate === RUN TestDestinationSatisfiesSource/older,_stale_backup_chunk_->_replicate === RUN TestDestinationSatisfiesSource/equal-mtime_stale_->_skip_(deferred) === RUN TestDestinationSatisfiesSource/rsync_complete_->_skip === RUN TestDestinationSatisfiesSource/in-sync_(fast_path)_->_skip === RUN TestDestinationSatisfiesSource/newer_smaller_dest,_older_larger_source_->_skip --- PASS: TestDestinationSatisfiesSource (0.00s) --- PASS: TestDestinationSatisfiesSource/nil_existing (0.00s) --- PASS: TestDestinationSatisfiesSource/identical_complete_->_skip (0.00s) --- PASS: TestDestinationSatisfiesSource/truncated_(no_md5)_->_replicate (0.00s) --- PASS: TestDestinationSatisfiesSource/md5-backed_truncated_->_replicate (0.00s) --- PASS: TestDestinationSatisfiesSource/md5_equal_and_complete_->_skip (0.00s) --- PASS: TestDestinationSatisfiesSource/complete,_dest_newer_->_skip (0.00s) --- PASS: TestDestinationSatisfiesSource/complete,_dest_older_->_replicate (0.00s) --- PASS: TestDestinationSatisfiesSource/older,_stale_backup_chunk_->_replicate (0.00s) --- PASS: TestDestinationSatisfiesSource/equal-mtime_stale_->_skip_(deferred) (0.00s) --- PASS: TestDestinationSatisfiesSource/rsync_complete_->_skip (0.00s) --- PASS: TestDestinationSatisfiesSource/in-sync_(fast_path)_->_skip (0.00s) --- PASS: TestDestinationSatisfiesSource/newer_smaller_dest,_older_larger_source_->_skip (0.00s) === RUN TestDestinationMatchesReference === RUN TestDestinationMatchesReference/both_empty === RUN TestDestinationMatchesReference/exact_in_order === RUN TestDestinationMatchesReference/count_mismatch === RUN TestDestinationMatchesReference/reordered === RUN TestDestinationMatchesReference/stale_identity === RUN TestDestinationMatchesReference/out-of-band_chunk === RUN TestDestinationMatchesReference/shifted_offset,_same_identity === RUN TestDestinationMatchesReference/larger_size,_same_identity --- PASS: TestDestinationMatchesReference (0.00s) --- PASS: TestDestinationMatchesReference/both_empty (0.00s) --- PASS: TestDestinationMatchesReference/exact_in_order (0.00s) --- PASS: TestDestinationMatchesReference/count_mismatch (0.00s) --- PASS: TestDestinationMatchesReference/reordered (0.00s) --- PASS: TestDestinationMatchesReference/stale_identity (0.00s) --- PASS: TestDestinationMatchesReference/out-of-band_chunk (0.00s) --- PASS: TestDestinationMatchesReference/shifted_offset,_same_identity (0.00s) --- PASS: TestDestinationMatchesReference/larger_size,_same_identity (0.00s) === RUN TestHasStaleBackupChunk === RUN TestHasStaleBackupChunk/nil_existing === RUN TestHasStaleBackupChunk/in_sync === RUN TestHasStaleBackupChunk/stale_backup_chunk === RUN TestHasStaleBackupChunk/rsync_chunks_ignored === RUN TestHasStaleBackupChunk/synced_backup_+_rsync === RUN TestHasStaleBackupChunk/stale_backup_among_rsync --- PASS: TestHasStaleBackupChunk (0.00s) --- PASS: TestHasStaleBackupChunk/nil_existing (0.00s) --- PASS: TestHasStaleBackupChunk/in_sync (0.00s) --- PASS: TestHasStaleBackupChunk/stale_backup_chunk (0.00s) --- PASS: TestHasStaleBackupChunk/rsync_chunks_ignored (0.00s) --- PASS: TestHasStaleBackupChunk/synced_backup_+_rsync (0.00s) --- PASS: TestHasStaleBackupChunk/stale_backup_among_rsync (0.00s) === RUN TestUpdatedEntryKey === RUN TestUpdatedEntryKey/content_update,_no_new_parent === RUN TestUpdatedEntryKey/rename_same_dir === RUN TestUpdatedEntryKey/rename_to_subdir === RUN TestUpdatedEntryKey/root_parent --- PASS: TestUpdatedEntryKey (0.00s) --- PASS: TestUpdatedEntryKey/content_update,_no_new_parent (0.00s) --- PASS: TestUpdatedEntryKey/rename_same_dir (0.00s) --- PASS: TestUpdatedEntryKey/rename_to_subdir (0.00s) --- PASS: TestUpdatedEntryKey/root_parent (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication/sink/filersink 0.057s ? github.com/seaweedfs/seaweedfs/weed/replication/sink/gcssink [no test files] === RUN TestSanitizeFsKeyKeepsColons --- PASS: TestSanitizeFsKeyKeepsColons (0.00s) === RUN TestCreateEntry_OverwriteReadOnlyFile --- PASS: TestCreateEntry_OverwriteReadOnlyFile (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication/sink/localsink 0.070s === RUN TestBuildTaggingString_ShouldStripTagPrefix --- PASS: TestBuildTaggingString_ShouldStripTagPrefix (0.00s) === RUN TestBuildTaggingString_ShouldURLEncodeValues --- PASS: TestBuildTaggingString_ShouldURLEncodeValues (0.00s) === RUN TestS3SinkCreateEntryPassesMimeAsContentType --- PASS: TestS3SinkCreateEntryPassesMimeAsContentType (0.00s) === RUN TestS3SinkCreateEntryOmitsContentTypeWhenMimeMissing --- PASS: TestS3SinkCreateEntryOmitsContentTypeWhenMimeMissing (0.00s) === RUN TestBuildTaggingString_EmptyWhenNoTags --- PASS: TestBuildTaggingString_EmptyWhenNoTags (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication/sink/s3sink 0.040s === RUN TestDownloadFile_NoOffset --- PASS: TestDownloadFile_NoOffset (0.00s) === RUN TestDownloadFile_WithOffset --- PASS: TestDownloadFile_WithOffset (0.00s) === RUN TestDownloadFile_RejectsIgnoredRange --- PASS: TestDownloadFile_RejectsIgnoredRange (0.00s) === RUN TestDownloadFile_ContentDisposition --- PASS: TestDownloadFile_ContentDisposition (0.00s) === RUN TestDownloadFile_PartialReadThenResume --- PASS: TestDownloadFile_PartialReadThenResume (0.00s) === RUN TestDownloadFile_GzipPartialReadThenResume --- PASS: TestDownloadFile_GzipPartialReadThenResume (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/replication/source 0.139s ? github.com/seaweedfs/seaweedfs/weed/replication/sub [no test files] === RUN TestAccountForUnscopedIdentity --- PASS: TestAccountForUnscopedIdentity (0.00s) === RUN TestUnscopedIdentitiesGetDistinctAccounts I0720 21:30:02.301713 config_loader.go:73 Using explicit credential store: memory --- PASS: TestUnscopedIdentitiesGetDistinctAccounts (0.00s) === RUN TestCheckAccessByOwnershipDeniesNonOwner --- PASS: TestCheckAccessByOwnershipDeniesNonOwner (0.00s) === RUN TestUnscopedIdentityAccountResolvesByName I0720 21:30:02.303659 config_loader.go:73 Using explicit credential store: memory --- PASS: TestUnscopedIdentityAccountResolvesByName (0.00s) === RUN TestUnscopedIdentityReusesConfiguredAccount I0720 21:30:02.303788 config_loader.go:73 Using explicit credential store: memory --- PASS: TestUnscopedIdentityReusesConfiguredAccount (0.00s) === RUN TestAuthorizeCopySource_AuthDisabled --- PASS: TestAuthorizeCopySource_AuthDisabled (0.00s) === RUN TestAuthorizeCopySource_NilIdentityDenies --- PASS: TestAuthorizeCopySource_NilIdentityDenies (0.00s) === RUN TestAuthorizeCopySource_AdminAllowed --- PASS: TestAuthorizeCopySource_AdminAllowed (0.00s) === RUN TestAuthorizeCopySource_PrefixScopedIdentity === RUN TestAuthorizeCopySource_PrefixScopedIdentity/source_within_scope_is_allowed === RUN TestAuthorizeCopySource_PrefixScopedIdentity/source_outside_scope_is_denied === RUN TestAuthorizeCopySource_PrefixScopedIdentity/source_in_different_bucket_is_denied --- PASS: TestAuthorizeCopySource_PrefixScopedIdentity (0.00s) --- PASS: TestAuthorizeCopySource_PrefixScopedIdentity/source_within_scope_is_allowed (0.00s) --- PASS: TestAuthorizeCopySource_PrefixScopedIdentity/source_outside_scope_is_denied (0.00s) --- PASS: TestAuthorizeCopySource_PrefixScopedIdentity/source_in_different_bucket_is_denied (0.00s) === RUN TestAuthorizeCopySource_IAMIntegrationGetsSourceResource --- PASS: TestAuthorizeCopySource_IAMIntegrationGetsSourceResource (0.00s) === RUN TestAuthorizeCopySource_IAMIntegrationAllow --- PASS: TestAuthorizeCopySource_IAMIntegrationAllow (0.00s) === RUN TestAuthorizeCopySource_VersionIdPropagated --- PASS: TestAuthorizeCopySource_VersionIdPropagated (0.00s) === RUN TestAuthorizeCopySource_PresignedURLSessionTokenPreserved --- PASS: TestAuthorizeCopySource_PresignedURLSessionTokenPreserved (0.00s) === RUN TestAuthorizeCopySource_PreservesCopySourceHeader --- PASS: TestAuthorizeCopySource_PreservesCopySourceHeader (0.00s) === RUN TestPutPolicy_SyncsToIAMManager --- PASS: TestPutPolicy_SyncsToIAMManager (0.00s) === RUN TestDeletePolicy_SyncsToIAMManager --- PASS: TestDeletePolicy_SyncsToIAMManager (0.00s) === RUN TestSetIAMIntegration_FlushesLoadedPolicies --- PASS: TestSetIAMIntegration_FlushesLoadedPolicies (0.00s) === RUN TestResyncIAMManager_ReflectsCurrentPolicies --- PASS: TestResyncIAMManager_ReflectsCurrentPolicies (0.00s) === RUN TestIamConfigWithoutIdentitiesIsNotStatic --- PASS: TestIamConfigWithoutIdentitiesIsNotStatic (0.00s) === RUN TestConfigWithIdentitiesIsStatic --- PASS: TestConfigWithIdentitiesIsStatic (0.00s) === RUN TestReloadStaticConfigMarksNewIdentitiesWithoutFreezingDynamic --- PASS: TestReloadStaticConfigMarksNewIdentitiesWithoutFreezingDynamic (0.00s) === RUN TestReloadStaticConfigUpdatesExistingSecretKey --- PASS: TestReloadStaticConfigUpdatesExistingSecretKey (0.01s) === RUN TestReloadStaticConfigUpdatesServiceAccountSecret --- PASS: TestReloadStaticConfigUpdatesServiceAccountSecret (0.00s) === RUN TestOnIamConfigChangeLegacyIdentityDeletionReloadsConfiguration --- PASS: TestOnIamConfigChangeLegacyIdentityDeletionReloadsConfiguration (0.00s) === RUN TestOnIamConfigChangeReloadsOnIamIdentityDirectoryChanges --- PASS: TestOnIamConfigChangeReloadsOnIamIdentityDirectoryChanges (0.00s) === RUN TestJWTStreamingUnsignedAuth I0720 21:30:02.322385 config_loader.go:73 Using explicit credential store: memory --- PASS: TestJWTStreamingUnsignedAuth (0.00s) === RUN TestJWTStreamingUnsignedChunkedReader --- PASS: TestJWTStreamingUnsignedChunkedReader (0.00s) === RUN TestPresignedPutSignedContentTypeWithRawSemicolon --- PASS: TestPresignedPutSignedContentTypeWithRawSemicolon (0.00s) === RUN TestReverseProxySignatureVerification === RUN TestReverseProxySignatureVerification/non-standard_port,_externalUrl_matches_proxy_address I0720 21:30:02.324996 auth_credentials.go:230 S3 signature verification will use external host: "127.0.0.1:40519" (from "http://127.0.0.1:40519") I0720 21:30:02.325006 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/externalUrl_with_non-standard_port,_client_signs_against_external_host I0720 21:30:02.326545 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com:9000" (from "http://api.example.com:9000") I0720 21:30:02.326552 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/externalUrl_with_HTTPS_default_port_stripped,_client_signs_without_port I0720 21:30:02.327276 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com" (from "https://api.example.com:443") I0720 21:30:02.327282 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/externalUrl_with_HTTP_default_port_stripped,_client_signs_without_port I0720 21:30:02.328188 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com" (from "http://api.example.com:80") I0720 21:30:02.328195 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/proxy_forwards_X-Forwarded-Host_correctly,_no_externalUrl_needed I0720 21:30:02.330030 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/proxy_without_X-Forwarded-Host,_no_externalUrl:_host_mismatch I0720 21:30:02.330701 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/nginx_$host/$server_port_forwarding,_backend_on_same_hostname I0720 21:30:02.331593 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/portless_X-Forwarded-Host_on_default_port,_backend_on_same_hostname I0720 21:30:02.334893 config_loader.go:73 Using explicit credential store: memory === RUN TestReverseProxySignatureVerification/proxy_without_X-Forwarded-Host,_externalUrl_saves_the_day I0720 21:30:02.335613 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com:9000" (from "http://api.example.com:9000") I0720 21:30:02.335618 config_loader.go:73 Using explicit credential store: memory --- PASS: TestReverseProxySignatureVerification (0.01s) --- PASS: TestReverseProxySignatureVerification/non-standard_port,_externalUrl_matches_proxy_address (0.00s) --- PASS: TestReverseProxySignatureVerification/externalUrl_with_non-standard_port,_client_signs_against_external_host (0.00s) --- PASS: TestReverseProxySignatureVerification/externalUrl_with_HTTPS_default_port_stripped,_client_signs_without_port (0.00s) --- PASS: TestReverseProxySignatureVerification/externalUrl_with_HTTP_default_port_stripped,_client_signs_without_port (0.00s) --- PASS: TestReverseProxySignatureVerification/proxy_forwards_X-Forwarded-Host_correctly,_no_externalUrl_needed (0.00s) --- PASS: TestReverseProxySignatureVerification/proxy_without_X-Forwarded-Host,_no_externalUrl:_host_mismatch (0.00s) --- PASS: TestReverseProxySignatureVerification/nginx_$host/$server_port_forwarding,_backend_on_same_hostname (0.00s) --- PASS: TestReverseProxySignatureVerification/portless_X-Forwarded-Host_on_default_port,_backend_on_same_hostname (0.00s) --- PASS: TestReverseProxySignatureVerification/proxy_without_X-Forwarded-Host,_externalUrl_saves_the_day (0.00s) === RUN TestActionScopedToBucket --- PASS: TestActionScopedToBucket (0.00s) === RUN TestReproIssue7912 I0720 21:30:02.336316 config_loader.go:73 Using explicit credential store: memory === RUN TestReproIssue7912/Unknown_access_key W0720 21:30:02.336455 auth_signature_v4.go:255 InvalidAccessKeyId: attempted key 'unknown_key' not found. Available keys: 2, Auth enabled: true === RUN TestReproIssue7912/Positive_test_case:_properly_signed_credentials === RUN TestReproIssue7912/Nil_identity_tests_for_guards === RUN TestReproIssue7912/AuthSignatureOnly_path === RUN TestReproIssue7912/Wrong_secret_key === RUN TestReproIssue7912/Anonymous_request_to_protected_bucket === RUN TestReproIssue7912/Non-S3_request_should_be_denied === RUN TestReproIssue7912/Any_other_credentials W0720 21:30:02.336859 auth_signature_v4.go:255 InvalidAccessKeyId: attempted key 'some_other_key' not found. Available keys: 2, Auth enabled: true === RUN TestReproIssue7912/Streaming_unsigned_payload_bypass_attempt --- PASS: TestReproIssue7912 (0.00s) --- PASS: TestReproIssue7912/Unknown_access_key (0.00s) --- PASS: TestReproIssue7912/Positive_test_case:_properly_signed_credentials (0.00s) --- PASS: TestReproIssue7912/Nil_identity_tests_for_guards (0.00s) --- PASS: TestReproIssue7912/AuthSignatureOnly_path (0.00s) --- PASS: TestReproIssue7912/Wrong_secret_key (0.00s) --- PASS: TestReproIssue7912/Anonymous_request_to_protected_bucket (0.00s) --- PASS: TestReproIssue7912/Non-S3_request_should_be_denied (0.00s) --- PASS: TestReproIssue7912/Any_other_credentials (0.00s) --- PASS: TestReproIssue7912/Streaming_unsigned_payload_bypass_attempt (0.00s) === RUN TestAnonymousStreamingUnsignedUpload I0720 21:30:02.336936 config_loader.go:73 Using explicit credential store: memory --- PASS: TestAnonymousStreamingUnsignedUpload (0.00s) === RUN TestExternalUrlSignatureVerification === RUN TestExternalUrlSignatureVerification/non-standard_port_with_externalUrl I0720 21:30:02.337046 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com:9000" (from "https://api.example.com:9000") I0720 21:30:02.337050 config_loader.go:73 Using explicit credential store: memory === RUN TestExternalUrlSignatureVerification/HTTPS_default_port_443_with_externalUrl_(port_stripped_by_SDK_and_parseExternalUrlToHost) I0720 21:30:02.337162 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com" (from "https://api.example.com:443") I0720 21:30:02.337166 config_loader.go:73 Using explicit credential store: memory === RUN TestExternalUrlSignatureVerification/HTTPS_without_explicit_port_with_externalUrl I0720 21:30:02.339188 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com" (from "https://api.example.com") I0720 21:30:02.339193 config_loader.go:73 Using explicit credential store: memory === RUN TestExternalUrlSignatureVerification/HTTP_default_port_80_with_externalUrl I0720 21:30:02.339336 auth_credentials.go:230 S3 signature verification will use external host: "api.example.com" (from "http://api.example.com:80") I0720 21:30:02.339344 config_loader.go:73 Using explicit credential store: memory === RUN TestExternalUrlSignatureVerification/without_externalUrl,_internal_host_causes_mismatch I0720 21:30:02.339464 config_loader.go:73 Using explicit credential store: memory === RUN TestExternalUrlSignatureVerification/without_externalUrl,_matching_host_works I0720 21:30:02.339637 config_loader.go:73 Using explicit credential store: memory --- PASS: TestExternalUrlSignatureVerification (0.00s) --- PASS: TestExternalUrlSignatureVerification/non-standard_port_with_externalUrl (0.00s) --- PASS: TestExternalUrlSignatureVerification/HTTPS_default_port_443_with_externalUrl_(port_stripped_by_SDK_and_parseExternalUrlToHost) (0.00s) --- PASS: TestExternalUrlSignatureVerification/HTTPS_without_explicit_port_with_externalUrl (0.00s) --- PASS: TestExternalUrlSignatureVerification/HTTP_default_port_80_with_externalUrl (0.00s) --- PASS: TestExternalUrlSignatureVerification/without_externalUrl,_internal_host_causes_mismatch (0.00s) --- PASS: TestExternalUrlSignatureVerification/without_externalUrl,_matching_host_works (0.00s) === RUN TestRealSDKSignerWithForwardedHeaders === RUN TestRealSDKSignerWithForwardedHeaders/HTTPS_non-standard_port I0720 21:30:02.339769 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/HTTPS_standard_port_443_(SDK_strips,_we_must_too) I0720 21:30:02.339867 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/HTTP_standard_port_80_(SDK_strips,_we_must_too) I0720 21:30:02.339945 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/HTTP_non-standard_port_8080 I0720 21:30:02.340028 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/X-Forwarded-Host_with_port_(Traefik_style),_HTTPS_443 I0720 21:30:02.340099 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/X-Forwarded-Host_with_port_(Traefik_style),_HTTP_80 I0720 21:30:02.340162 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/no_forwarded_headers,_direct_access I0720 21:30:02.340231 config_loader.go:73 Using explicit credential store: memory === RUN TestRealSDKSignerWithForwardedHeaders/empty_proto_with_port_80_(defaults_to_http,_strip) I0720 21:30:02.340302 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRealSDKSignerWithForwardedHeaders (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/HTTPS_non-standard_port (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/HTTPS_standard_port_443_(SDK_strips,_we_must_too) (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/HTTP_standard_port_80_(SDK_strips,_we_must_too) (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/HTTP_non-standard_port_8080 (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/X-Forwarded-Host_with_port_(Traefik_style),_HTTPS_443 (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/X-Forwarded-Host_with_port_(Traefik_style),_HTTP_80 (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/no_forwarded_headers,_direct_access (0.00s) --- PASS: TestRealSDKSignerWithForwardedHeaders/empty_proto_with_port_80_(defaults_to_http,_strip) (0.00s) === RUN TestValidateV2AuthHeader === RUN TestValidateV2AuthHeader/valid_auth_header_with_space === RUN TestValidateV2AuthHeader/empty_auth_header === RUN TestValidateV2AuthHeader/wrong_algorithm_prefix === RUN TestValidateV2AuthHeader/algorithm_prefix_without_space === RUN TestValidateV2AuthHeader/missing_colon_separator === RUN TestValidateV2AuthHeader/empty_access_key === RUN TestValidateV2AuthHeader/empty_signature --- PASS: TestValidateV2AuthHeader (0.00s) --- PASS: TestValidateV2AuthHeader/valid_auth_header_with_space (0.00s) --- PASS: TestValidateV2AuthHeader/empty_auth_header (0.00s) --- PASS: TestValidateV2AuthHeader/wrong_algorithm_prefix (0.00s) --- PASS: TestValidateV2AuthHeader/algorithm_prefix_without_space (0.00s) --- PASS: TestValidateV2AuthHeader/missing_colon_separator (0.00s) --- PASS: TestValidateV2AuthHeader/empty_access_key (0.00s) --- PASS: TestValidateV2AuthHeader/empty_signature (0.00s) === RUN TestSignatureV2Format --- PASS: TestSignatureV2Format (0.00s) === RUN TestDoesSignV2Match === RUN TestDoesSignV2Match/valid_GET_request === RUN TestDoesSignV2Match/valid_PUT_request_with_content_headers === RUN TestDoesSignV2Match/request_with_query_parameters === RUN TestDoesSignV2Match/request_with_x-amz_headers === RUN TestDoesSignV2Match/invalid_signature === RUN TestDoesSignV2Match/non-existent_access_key W0720 21:30:02.340534 auth_signature_v2.go:137 InvalidAccessKeyId (V2 signed): attempted key 'NONEXISTENTKEY' not found. Available keys: 1, Auth enabled: false === RUN TestDoesSignV2Match/empty_authorization_header === RUN TestDoesSignV2Match/malformed_auth_-_missing_signature === RUN TestDoesSignV2Match/malformed_auth_-_wrong_prefix === RUN TestDoesSignV2Match/malformed_auth_-_no_space_after_AWS --- PASS: TestDoesSignV2Match (0.00s) --- PASS: TestDoesSignV2Match/valid_GET_request (0.00s) --- PASS: TestDoesSignV2Match/valid_PUT_request_with_content_headers (0.00s) --- PASS: TestDoesSignV2Match/request_with_query_parameters (0.00s) --- PASS: TestDoesSignV2Match/request_with_x-amz_headers (0.00s) --- PASS: TestDoesSignV2Match/invalid_signature (0.00s) --- PASS: TestDoesSignV2Match/non-existent_access_key (0.00s) --- PASS: TestDoesSignV2Match/empty_authorization_header (0.00s) --- PASS: TestDoesSignV2Match/malformed_auth_-_missing_signature (0.00s) --- PASS: TestDoesSignV2Match/malformed_auth_-_wrong_prefix (0.00s) --- PASS: TestDoesSignV2Match/malformed_auth_-_no_space_after_AWS (0.00s) === RUN TestDoesSignV2MatchUnicodePath === RUN TestDoesSignV2MatchUnicodePath/CJK_conversation_key_from_ai-dial-core === RUN TestDoesSignV2MatchUnicodePath/Japanese_filename_from_issue_8598 --- PASS: TestDoesSignV2MatchUnicodePath (0.00s) --- PASS: TestDoesSignV2MatchUnicodePath/CJK_conversation_key_from_ai-dial-core (0.00s) --- PASS: TestDoesSignV2MatchUnicodePath/Japanese_filename_from_issue_8598 (0.00s) === RUN TestDoesPresignV2MatchUnicodePath --- PASS: TestDoesPresignV2MatchUnicodePath (0.00s) === RUN TestValidateSTSSessionTokenAssignsDistinctAccount === RUN TestValidateSTSSessionTokenAssignsDistinctAccount/prefers_the_subject === RUN TestValidateSTSSessionTokenAssignsDistinctAccount/falls_back_to_the_assumed-role_user --- PASS: TestValidateSTSSessionTokenAssignsDistinctAccount (0.00s) --- PASS: TestValidateSTSSessionTokenAssignsDistinctAccount/prefers_the_subject (0.00s) --- PASS: TestValidateSTSSessionTokenAssignsDistinctAccount/falls_back_to_the_assumed-role_user (0.00s) === RUN TestVerifyV4SignatureWithSTSIdentity === RUN TestVerifyV4SignatureWithSTSIdentity/STS_identity_with_IAM_integration_-_should_authorize_via_IAM === RUN TestVerifyV4SignatureWithSTSIdentity/STS_identity_with_IAM_integration_-_IAM_denies === RUN TestVerifyV4SignatureWithSTSIdentity/STS_identity_without_IAM_integration_-_should_deny === RUN TestVerifyV4SignatureWithSTSIdentity/Traditional_identity_with_Actions_-_should_use_CanDo === RUN TestVerifyV4SignatureWithSTSIdentity/Traditional_identity_with_Actions_-_CanDo_denies === RUN TestVerifyV4SignatureWithSTSIdentity/shouldCheckPermissions_false_-_skip_authorization --- PASS: TestVerifyV4SignatureWithSTSIdentity (0.00s) --- PASS: TestVerifyV4SignatureWithSTSIdentity/STS_identity_with_IAM_integration_-_should_authorize_via_IAM (0.00s) --- PASS: TestVerifyV4SignatureWithSTSIdentity/STS_identity_with_IAM_integration_-_IAM_denies (0.00s) --- PASS: TestVerifyV4SignatureWithSTSIdentity/STS_identity_without_IAM_integration_-_should_deny (0.00s) --- PASS: TestVerifyV4SignatureWithSTSIdentity/Traditional_identity_with_Actions_-_should_use_CanDo (0.00s) --- PASS: TestVerifyV4SignatureWithSTSIdentity/Traditional_identity_with_Actions_-_CanDo_denies (0.00s) --- PASS: TestVerifyV4SignatureWithSTSIdentity/shouldCheckPermissions_false_-_skip_authorization (0.00s) === RUN TestVerifyV4SignatureSTSStreamingUpload --- PASS: TestVerifyV4SignatureSTSStreamingUpload (0.00s) === RUN TestVerifyV4SignatureActionDetermination === RUN TestVerifyV4SignatureActionDetermination/GET === RUN TestVerifyV4SignatureActionDetermination/HEAD === RUN TestVerifyV4SignatureActionDetermination/PUT === RUN TestVerifyV4SignatureActionDetermination/POST === RUN TestVerifyV4SignatureActionDetermination/DELETE --- PASS: TestVerifyV4SignatureActionDetermination (0.00s) --- PASS: TestVerifyV4SignatureActionDetermination/GET (0.00s) --- PASS: TestVerifyV4SignatureActionDetermination/HEAD (0.00s) --- PASS: TestVerifyV4SignatureActionDetermination/PUT (0.00s) --- PASS: TestVerifyV4SignatureActionDetermination/POST (0.00s) --- PASS: TestVerifyV4SignatureActionDetermination/DELETE (0.00s) === RUN TestExtractV4AuthInfoFromHeader_S3Tables === RUN TestExtractV4AuthInfoFromHeader_S3Tables/s3_service_should_not_auto-hash === RUN TestExtractV4AuthInfoFromHeader_S3Tables/s3tables_service_should_auto-hash --- PASS: TestExtractV4AuthInfoFromHeader_S3Tables (0.00s) --- PASS: TestExtractV4AuthInfoFromHeader_S3Tables/s3_service_should_not_auto-hash (0.00s) --- PASS: TestExtractV4AuthInfoFromHeader_S3Tables/s3tables_service_should_auto-hash (0.00s) === RUN TestParseSignedHeaderRejectsEmptyHeaderNames --- PASS: TestParseSignedHeaderRejectsEmptyHeaderNames (0.00s) === RUN TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames === RUN TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames/host; === RUN TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames/;host === RUN TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames/host;;x-amz-date --- PASS: TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames (0.00s) --- PASS: TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames/host; (0.00s) --- PASS: TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames/;host (0.00s) --- PASS: TestExtractV4AuthInfoFromQueryRejectsEmptySignedHeaderNames/host;;x-amz-date (0.00s) === RUN TestGetCanonicalQueryString === RUN TestGetCanonicalQueryString/sorts_repeated_values === RUN TestGetCanonicalQueryString/removes_presigned_signature === RUN TestGetCanonicalQueryString/sorts_mixed_single_and_repeated_parameters === RUN TestGetCanonicalQueryString/encodes_query_parameter_values === RUN TestGetCanonicalQueryString/empty_query_string --- PASS: TestGetCanonicalQueryString (0.00s) --- PASS: TestGetCanonicalQueryString/sorts_repeated_values (0.00s) --- PASS: TestGetCanonicalQueryString/removes_presigned_signature (0.00s) --- PASS: TestGetCanonicalQueryString/sorts_mixed_single_and_repeated_parameters (0.00s) --- PASS: TestGetCanonicalQueryString/encodes_query_parameter_values (0.00s) --- PASS: TestGetCanonicalQueryString/empty_query_string (0.00s) === RUN TestBuildPathWithForwardedPrefix === RUN TestBuildPathWithForwardedPrefix/empty_prefix_returns_urlPath === RUN TestBuildPathWithForwardedPrefix/prefix_without_trailing_slash === RUN TestBuildPathWithForwardedPrefix/prefix_with_trailing_slash === RUN TestBuildPathWithForwardedPrefix/prefix_without_leading_slash === RUN TestBuildPathWithForwardedPrefix/prefix_without_leading_slash_and_with_trailing_slash === RUN TestBuildPathWithForwardedPrefix/preserve_double_slashes_in_key === RUN TestBuildPathWithForwardedPrefix/preserve_trailing_slash_in_urlPath === RUN TestBuildPathWithForwardedPrefix/preserve_trailing_slash_with_prefix_having_trailing_slash === RUN TestBuildPathWithForwardedPrefix/root_path === RUN TestBuildPathWithForwardedPrefix/complex_key_with_multiple_slashes === RUN TestBuildPathWithForwardedPrefix/urlPath_without_leading_slash === RUN TestBuildPathWithForwardedPrefix/empty_urlPath_with_prefix --- PASS: TestBuildPathWithForwardedPrefix (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/empty_prefix_returns_urlPath (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/prefix_without_trailing_slash (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/prefix_with_trailing_slash (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/prefix_without_leading_slash (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/prefix_without_leading_slash_and_with_trailing_slash (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/preserve_double_slashes_in_key (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/preserve_trailing_slash_in_urlPath (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/preserve_trailing_slash_with_prefix_having_trailing_slash (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/root_path (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/complex_key_with_multiple_slashes (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/urlPath_without_leading_slash (0.00s) --- PASS: TestBuildPathWithForwardedPrefix/empty_urlPath_with_prefix (0.00s) === RUN TestExtractHostHeader === RUN TestExtractHostHeader/basic_host_without_forwarding === RUN TestExtractHostHeader/host_with_port_without_forwarding === RUN TestExtractHostHeader/X-Forwarded-Host_without_port === RUN TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTP_non-standard) === RUN TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTPS_non-standard) === RUN TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTP_standard_port_80) === RUN TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTPS_standard_port_443) === RUN TestExtractHostHeader/X-Forwarded-Host_with_port_already_included_(should_not_add_port_again) === RUN TestExtractHostHeader/X-Forwarded-Host_with_standard_port_already_included_(HTTPS_443) === RUN TestExtractHostHeader/X-Forwarded-Host_with_port,_no_X-Forwarded-Port_header === RUN TestExtractHostHeader/IPv6_address_with_brackets_and_port_in_X-Forwarded-Host === RUN TestExtractHostHeader/IPv6_address_without_brackets,_should_add_brackets_with_port === RUN TestExtractHostHeader/IPv6_address_without_brackets_and_standard_port,_should_strip_default_port === RUN TestExtractHostHeader/IPv6_address_without_brackets_and_standard_HTTPS_port,_should_strip_default_port === RUN TestExtractHostHeader/IPv6_address_with_brackets_but_no_port,_should_add_port === RUN TestExtractHostHeader/IPv6_full_address_with_brackets_and_default_port_(should_strip_default_port) === RUN TestExtractHostHeader/IPv4-mapped_IPv6_address_without_brackets,_should_add_brackets_with_port === RUN TestExtractHostHeader/Simple_port_442 === RUN TestExtractHostHeader/Port_442_with_X-Forwarded-Host === RUN TestExtractHostHeader/Port_442_with_X-Forwarded-Port === RUN TestExtractHostHeader/HTTPS_with_port_442_(should_NOT_strip) === RUN TestExtractHostHeader/X-Forwarded-Host_with_multiple_hosts_(including_port) === RUN TestExtractHostHeader/IPv6_with_port === RUN TestExtractHostHeader/X-Forwarded-Host_with_port_442,_but_X-Forwarded-Port_is_80_(should_PREFER_442) === RUN TestExtractHostHeader/externalHost_overrides_everything === RUN TestExtractHostHeader/externalHost_overrides_X-Forwarded-Host === RUN TestExtractHostHeader/externalHost_with_IPv6 === RUN TestExtractHostHeader/User_reported_case:_X-Forwarded-Port_misreports_443_but_Host_has_30007 === RUN TestExtractHostHeader/X-Forwarded-Host_already_contains_correct_port,_ignore_misaligned_X-Forwarded-Port === RUN TestExtractHostHeader/X-Forwarded-Host_has_no_port,_match_r.Host_hostname_and_take_its_port --- PASS: TestExtractHostHeader (0.00s) --- PASS: TestExtractHostHeader/basic_host_without_forwarding (0.00s) --- PASS: TestExtractHostHeader/host_with_port_without_forwarding (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_without_port (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTP_non-standard) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTPS_non-standard) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTP_standard_port_80) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_X-Forwarded-Port_(HTTPS_standard_port_443) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_port_already_included_(should_not_add_port_again) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_standard_port_already_included_(HTTPS_443) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_port,_no_X-Forwarded-Port_header (0.00s) --- PASS: TestExtractHostHeader/IPv6_address_with_brackets_and_port_in_X-Forwarded-Host (0.00s) --- PASS: TestExtractHostHeader/IPv6_address_without_brackets,_should_add_brackets_with_port (0.00s) --- PASS: TestExtractHostHeader/IPv6_address_without_brackets_and_standard_port,_should_strip_default_port (0.00s) --- PASS: TestExtractHostHeader/IPv6_address_without_brackets_and_standard_HTTPS_port,_should_strip_default_port (0.00s) --- PASS: TestExtractHostHeader/IPv6_address_with_brackets_but_no_port,_should_add_port (0.00s) --- PASS: TestExtractHostHeader/IPv6_full_address_with_brackets_and_default_port_(should_strip_default_port) (0.00s) --- PASS: TestExtractHostHeader/IPv4-mapped_IPv6_address_without_brackets,_should_add_brackets_with_port (0.00s) --- PASS: TestExtractHostHeader/Simple_port_442 (0.00s) --- PASS: TestExtractHostHeader/Port_442_with_X-Forwarded-Host (0.00s) --- PASS: TestExtractHostHeader/Port_442_with_X-Forwarded-Port (0.00s) --- PASS: TestExtractHostHeader/HTTPS_with_port_442_(should_NOT_strip) (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_multiple_hosts_(including_port) (0.00s) --- PASS: TestExtractHostHeader/IPv6_with_port (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_with_port_442,_but_X-Forwarded-Port_is_80_(should_PREFER_442) (0.00s) --- PASS: TestExtractHostHeader/externalHost_overrides_everything (0.00s) --- PASS: TestExtractHostHeader/externalHost_overrides_X-Forwarded-Host (0.00s) --- PASS: TestExtractHostHeader/externalHost_with_IPv6 (0.00s) --- PASS: TestExtractHostHeader/User_reported_case:_X-Forwarded-Port_misreports_443_but_Host_has_30007 (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_already_contains_correct_port,_ignore_misaligned_X-Forwarded-Port (0.00s) --- PASS: TestExtractHostHeader/X-Forwarded-Host_has_no_port,_match_r.Host_hostname_and_take_its_port (0.00s) === RUN TestExtractHostHeaderCandidates === RUN TestExtractHostHeaderCandidates/externalHost_is_the_only_candidate === RUN TestExtractHostHeaderCandidates/X-Forwarded-Host_with_port_is_trusted_as-is === RUN TestExtractHostHeaderCandidates/plain_Host_with_port_is_the_only_candidate === RUN TestExtractHostHeaderCandidates/portless_X-Forwarded-Host,_hostnames_match:_r.Host_port_first,_then_X-Forwarded-Port,_then_bare === RUN TestExtractHostHeaderCandidates/portless_X-Forwarded-Host,_hostnames_match,_no_X-Forwarded-Port:_bare_host_as_fallback === RUN TestExtractHostHeaderCandidates/portless_X-Forwarded-Host,_hostnames_differ:_X-Forwarded-Port,_then_bare === RUN TestExtractHostHeaderCandidates/default_X-Forwarded-Port_collapses_into_the_bare_candidate === RUN TestExtractHostHeaderCandidates/portless_Host_with_X-Forwarded-Port:_forwarded_port,_then_bare === RUN TestExtractHostHeaderCandidates/bracketed_portless_IPv6_X-Forwarded-Host_matches_the_request_host === RUN TestExtractHostHeaderCandidates/unbracketed_portless_IPv6_X-Forwarded-Host_with_X-Forwarded-Port --- PASS: TestExtractHostHeaderCandidates (0.00s) --- PASS: TestExtractHostHeaderCandidates/externalHost_is_the_only_candidate (0.00s) --- PASS: TestExtractHostHeaderCandidates/X-Forwarded-Host_with_port_is_trusted_as-is (0.00s) --- PASS: TestExtractHostHeaderCandidates/plain_Host_with_port_is_the_only_candidate (0.00s) --- PASS: TestExtractHostHeaderCandidates/portless_X-Forwarded-Host,_hostnames_match:_r.Host_port_first,_then_X-Forwarded-Port,_then_bare (0.00s) --- PASS: TestExtractHostHeaderCandidates/portless_X-Forwarded-Host,_hostnames_match,_no_X-Forwarded-Port:_bare_host_as_fallback (0.00s) --- PASS: TestExtractHostHeaderCandidates/portless_X-Forwarded-Host,_hostnames_differ:_X-Forwarded-Port,_then_bare (0.00s) --- PASS: TestExtractHostHeaderCandidates/default_X-Forwarded-Port_collapses_into_the_bare_candidate (0.00s) --- PASS: TestExtractHostHeaderCandidates/portless_Host_with_X-Forwarded-Port:_forwarded_port,_then_bare (0.00s) --- PASS: TestExtractHostHeaderCandidates/bracketed_portless_IPv6_X-Forwarded-Host_matches_the_request_host (0.00s) --- PASS: TestExtractHostHeaderCandidates/unbracketed_portless_IPv6_X-Forwarded-Host_with_X-Forwarded-Port (0.00s) === RUN TestExtractSignedHeadersCase === RUN TestExtractSignedHeadersCase/lowercase_host === RUN TestExtractSignedHeadersCase/uppercase_Host --- PASS: TestExtractSignedHeadersCase (0.00s) --- PASS: TestExtractSignedHeadersCase/lowercase_host (0.00s) --- PASS: TestExtractSignedHeadersCase/uppercase_Host (0.00s) === RUN TestVerifySignedHeadersCoverage_Unit === RUN TestVerifySignedHeadersCoverage_Unit/no_x-amz_headers_is_fine === RUN TestVerifySignedHeadersCoverage_Unit/signed_x-amz_header_is_accepted === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-tagging_is_rejected_(header-based) === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-meta-*_is_rejected === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-acl_is_rejected === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-storage-class_is_rejected === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-server-side-encryption_is_rejected === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-object-lock-retain-until-date_is_rejected === RUN TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-security-token_is_rejected_(even_presigned) === RUN TestVerifySignedHeadersCoverage_Unit/x-amz-content-sha256_is_always_exempt_(presigned) === RUN TestVerifySignedHeadersCoverage_Unit/x-amz-content-sha256_is_always_exempt_(header-based) === RUN TestVerifySignedHeadersCoverage_Unit/presigned_exempts_sigv4_query_params_echoed_as_headers === RUN TestVerifySignedHeadersCoverage_Unit/header-based_does_NOT_exempt_x-amz-date_when_unsigned === RUN TestVerifySignedHeadersCoverage_Unit/signed_headers_match_is_case-insensitive --- PASS: TestVerifySignedHeadersCoverage_Unit (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/no_x-amz_headers_is_fine (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/signed_x-amz_header_is_accepted (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-tagging_is_rejected_(header-based) (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-meta-*_is_rejected (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-acl_is_rejected (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-storage-class_is_rejected (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-server-side-encryption_is_rejected (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-object-lock-retain-until-date_is_rejected (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/unsigned_x-amz-security-token_is_rejected_(even_presigned) (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/x-amz-content-sha256_is_always_exempt_(presigned) (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/x-amz-content-sha256_is_always_exempt_(header-based) (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/presigned_exempts_sigv4_query_params_echoed_as_headers (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/header-based_does_NOT_exempt_x-amz-date_when_unsigned (0.00s) --- PASS: TestVerifySignedHeadersCoverage_Unit/signed_headers_match_is_case-insensitive (0.00s) === RUN TestPresignedPutRejectsUnsignedTagging --- PASS: TestPresignedPutRejectsUnsignedTagging (0.00s) === RUN TestPresignedPutAcceptsSignedTagging --- PASS: TestPresignedPutAcceptsSignedTagging (0.00s) === RUN TestPresignedPutRejectsUnsignedMetadataHeaders === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/acl === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/user-metadata === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/storage-class === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/server-side-encryption === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/sse-kms-key-id === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/object-lock-mode === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/object-lock-retain-until === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/website-redirect-location === RUN TestPresignedPutRejectsUnsignedMetadataHeaders/grant-full-control --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/acl (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/user-metadata (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/storage-class (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/server-side-encryption (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/sse-kms-key-id (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/object-lock-mode (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/object-lock-retain-until (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/website-redirect-location (0.00s) --- PASS: TestPresignedPutRejectsUnsignedMetadataHeaders/grant-full-control (0.00s) === RUN TestSTSIdentityPolicyNamesPopulation I0720 21:30:02.343690 config_loader.go:73 Using explicit credential store: memory auth_sts_identity_test.go:94: ✓ STS identity will use IAM authorization path (correct behavior) --- PASS: TestSTSIdentityPolicyNamesPopulation (0.00s) === RUN TestSTSIdentityAuthorizationFlow auth_sts_identity_test.go:154: ✓ Verified: STS identity correctly bypasses legacy CanDo() to use IAM authorization path --- PASS: TestSTSIdentityAuthorizationFlow (0.00s) === RUN TestSTSIdentityWithoutPolicyNames auth_sts_identity_test.go:174: Bug scenario: Actions=0, PolicyNames=0 → Would be denied auth_sts_identity_test.go:176: Fixed scenario: Actions=0, PolicyNames=1 → Can use IAM authorization --- PASS: TestSTSIdentityWithoutPolicyNames (0.00s) === RUN TestCanDoPathConstruction === RUN TestCanDoPathConstruction/Wildcard_match_for_write auth_sts_identity_test.go:249: Testing path: test-bucket/path/to/object.txt === RUN TestCanDoPathConstruction/Wildcard_match_for_read_in_docs_path auth_sts_identity_test.go:249: Testing path: test-bucket/docs/file.txt === RUN TestCanDoPathConstruction/No_match_-_wrong_action auth_sts_identity_test.go:249: Testing path: test-bucket/file.txt === RUN TestCanDoPathConstruction/No_match_-_read_outside_docs_path auth_sts_identity_test.go:249: Testing path: test-bucket/other/file.txt === RUN TestCanDoPathConstruction/ObjectKey_with_leading_slash auth_sts_identity_test.go:249: Testing path: test-bucket/docs/file.txt --- PASS: TestCanDoPathConstruction (0.00s) --- PASS: TestCanDoPathConstruction/Wildcard_match_for_write (0.00s) --- PASS: TestCanDoPathConstruction/Wildcard_match_for_read_in_docs_path (0.00s) --- PASS: TestCanDoPathConstruction/No_match_-_wrong_action (0.00s) --- PASS: TestCanDoPathConstruction/No_match_-_read_outside_docs_path (0.00s) --- PASS: TestCanDoPathConstruction/ObjectKey_with_leading_slash (0.00s) === RUN TestValidateSTSSessionTokenIntegration I0720 21:30:02.344093 config_loader.go:73 Using explicit credential store: memory auth_sts_identity_test.go:304: ✓ Integration test passed: STS identity properly configured for IAM authorization --- PASS: TestValidateSTSSessionTokenIntegration (0.00s) === RUN TestSTSIdentityClaimsPopulation I0720 21:30:02.344288 config_loader.go:73 Using explicit credential store: memory auth_sts_identity_test.go:369: ✓ Claims properly populated from RequestContext for policy variable substitution --- PASS: TestSTSIdentityClaimsPopulation (0.00s) === RUN TestAuthorizeWithIAMSessionTokenExtraction === RUN TestAuthorizeWithIAMSessionTokenExtraction/Extracts_X-SeaweedFS-Session-Token_from_JWT_auth === RUN TestAuthorizeWithIAMSessionTokenExtraction/Extracts_X-Amz-Security-Token_from_V4_STS_auth_header === RUN TestAuthorizeWithIAMSessionTokenExtraction/Extracts_X-Amz-Security-Token_from_query_parameter_(presigned_URL) === RUN TestAuthorizeWithIAMSessionTokenExtraction/JWT_token_takes_precedence_over_X-Amz-Security-Token --- PASS: TestAuthorizeWithIAMSessionTokenExtraction (0.00s) --- PASS: TestAuthorizeWithIAMSessionTokenExtraction/Extracts_X-SeaweedFS-Session-Token_from_JWT_auth (0.00s) --- PASS: TestAuthorizeWithIAMSessionTokenExtraction/Extracts_X-Amz-Security-Token_from_V4_STS_auth_header (0.00s) --- PASS: TestAuthorizeWithIAMSessionTokenExtraction/Extracts_X-Amz-Security-Token_from_query_parameter_(presigned_URL) (0.00s) --- PASS: TestAuthorizeWithIAMSessionTokenExtraction/JWT_token_takes_precedence_over_X-Amz-Security-Token (0.00s) === RUN TestSTSSessionTokenIntoCredentials --- PASS: TestSTSSessionTokenIntoCredentials (0.00s) === RUN TestActionConstantsForV4Auth --- PASS: TestActionConstantsForV4Auth (0.00s) === RUN TestIsRequestPresignedSignatureV4 --- PASS: TestIsRequestPresignedSignatureV4 (0.00s) === RUN TestIsReqAuthenticated --- PASS: TestIsReqAuthenticated (0.00s) === RUN TestCheckaAnonymousRequestAuthType --- PASS: TestCheckaAnonymousRequestAuthType (0.00s) === RUN TestAnonymousListBucketsWithPrefixAction --- PASS: TestAnonymousListBucketsWithPrefixAction (0.00s) === RUN TestCheckAdminRequestAuthType --- PASS: TestCheckAdminRequestAuthType (0.00s) === RUN TestSignatureV4WithForwardedPrefix === RUN TestSignatureV4WithForwardedPrefix/prefix_without_trailing_slash === RUN TestSignatureV4WithForwardedPrefix/prefix_with_trailing_slash --- PASS: TestSignatureV4WithForwardedPrefix (0.00s) --- PASS: TestSignatureV4WithForwardedPrefix/prefix_without_trailing_slash (0.00s) --- PASS: TestSignatureV4WithForwardedPrefix/prefix_with_trailing_slash (0.00s) === RUN TestSignatureV4WithForwardedPrefixTrailingSlash === RUN TestSignatureV4WithForwardedPrefixTrailingSlash/bucket_listObjects_with_trailing_slash === RUN TestSignatureV4WithForwardedPrefixTrailingSlash/prefix_path_with_trailing_slash === RUN TestSignatureV4WithForwardedPrefixTrailingSlash/root_bucket_with_trailing_slash === RUN TestSignatureV4WithForwardedPrefixTrailingSlash/nested_folder_with_trailing_slash --- PASS: TestSignatureV4WithForwardedPrefixTrailingSlash (0.00s) --- PASS: TestSignatureV4WithForwardedPrefixTrailingSlash/bucket_listObjects_with_trailing_slash (0.00s) --- PASS: TestSignatureV4WithForwardedPrefixTrailingSlash/prefix_path_with_trailing_slash (0.00s) --- PASS: TestSignatureV4WithForwardedPrefixTrailingSlash/root_bucket_with_trailing_slash (0.00s) --- PASS: TestSignatureV4WithForwardedPrefixTrailingSlash/nested_folder_with_trailing_slash (0.00s) === RUN TestSignatureV4WithoutProxy === RUN TestSignatureV4WithoutProxy/HTTP_with_non-standard_port === RUN TestSignatureV4WithoutProxy/HTTPS_with_non-standard_port === RUN TestSignatureV4WithoutProxy/HTTP_with_standard_port === RUN TestSignatureV4WithoutProxy/HTTPS_with_standard_port === RUN TestSignatureV4WithoutProxy/HTTP_without_port === RUN TestSignatureV4WithoutProxy/HTTPS_without_port === RUN TestSignatureV4WithoutProxy/IPv6_HTTP_with_non-standard_port === RUN TestSignatureV4WithoutProxy/IPv6_HTTPS_with_non-standard_port === RUN TestSignatureV4WithoutProxy/IPv6_HTTP_with_standard_port === RUN TestSignatureV4WithoutProxy/IPv6_HTTPS_with_standard_port === RUN TestSignatureV4WithoutProxy/IPv6_HTTP_without_port === RUN TestSignatureV4WithoutProxy/IPv6_HTTPS_without_port --- PASS: TestSignatureV4WithoutProxy (0.00s) --- PASS: TestSignatureV4WithoutProxy/HTTP_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/HTTPS_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/HTTP_with_standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/HTTPS_with_standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/HTTP_without_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/HTTPS_without_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/IPv6_HTTP_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/IPv6_HTTPS_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/IPv6_HTTP_with_standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/IPv6_HTTPS_with_standard_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/IPv6_HTTP_without_port (0.00s) --- PASS: TestSignatureV4WithoutProxy/IPv6_HTTPS_without_port (0.00s) === RUN TestSignatureV4WithForwardedPort === RUN TestSignatureV4WithForwardedPort/HTTP_with_non-standard_port === RUN TestSignatureV4WithForwardedPort/HTTPS_with_non-standard_port === RUN TestSignatureV4WithForwardedPort/HTTP_with_standard_port_(80) === RUN TestSignatureV4WithForwardedPort/HTTPS_with_standard_port_(443) === RUN TestSignatureV4WithForwardedPort/empty_proto_with_non-standard_port === RUN TestSignatureV4WithForwardedPort/empty_proto_with_port_80_(scheme_defaults_to_https_from_URL,_so_80_is_NOT_default) === RUN TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_port_already_included_(Traefik/HAProxy_style) === RUN TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_port,_no_X-Forwarded-Port_header === RUN TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_standard_https_port_already_included_(Traefik/HAProxy_style) === RUN TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_standard_http_port_already_included_(Traefik/HAProxy_style) === RUN TestSignatureV4WithForwardedPort/IPv6_X-Forwarded-Host_with_standard_https_port_already_included_(Traefik/HAProxy_style) === RUN TestSignatureV4WithForwardedPort/IPv6_X-Forwarded-Host_with_standard_http_port_already_included_(Traefik/HAProxy_style) === RUN TestSignatureV4WithForwardedPort/IPv6_with_port_in_brackets === RUN TestSignatureV4WithForwardedPort/IPv6_without_port_-_should_add_port_with_brackets === RUN TestSignatureV4WithForwardedPort/IPv6_in_brackets_without_port_-_should_add_port === RUN TestSignatureV4WithForwardedPort/IPv4-mapped_IPv6_without_port_-_should_add_port_with_brackets --- PASS: TestSignatureV4WithForwardedPort (0.00s) --- PASS: TestSignatureV4WithForwardedPort/HTTP_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithForwardedPort/HTTPS_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithForwardedPort/HTTP_with_standard_port_(80) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/HTTPS_with_standard_port_(443) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/empty_proto_with_non-standard_port (0.00s) --- PASS: TestSignatureV4WithForwardedPort/empty_proto_with_port_80_(scheme_defaults_to_https_from_URL,_so_80_is_NOT_default) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_port_already_included_(Traefik/HAProxy_style) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_port,_no_X-Forwarded-Port_header (0.00s) --- PASS: TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_standard_https_port_already_included_(Traefik/HAProxy_style) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/X-Forwarded-Host_with_standard_http_port_already_included_(Traefik/HAProxy_style) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/IPv6_X-Forwarded-Host_with_standard_https_port_already_included_(Traefik/HAProxy_style) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/IPv6_X-Forwarded-Host_with_standard_http_port_already_included_(Traefik/HAProxy_style) (0.00s) --- PASS: TestSignatureV4WithForwardedPort/IPv6_with_port_in_brackets (0.00s) --- PASS: TestSignatureV4WithForwardedPort/IPv6_without_port_-_should_add_port_with_brackets (0.00s) --- PASS: TestSignatureV4WithForwardedPort/IPv6_in_brackets_without_port_-_should_add_port (0.00s) --- PASS: TestSignatureV4WithForwardedPort/IPv4-mapped_IPv6_without_port_-_should_add_port_with_brackets (0.00s) === RUN TestPresignedSignatureV4Basic --- PASS: TestPresignedSignatureV4Basic (0.00s) === RUN TestPresignedSignatureV4MissingExpires --- PASS: TestPresignedSignatureV4MissingExpires (0.00s) === RUN TestPresignedSignatureV4WithForwardedPrefix === RUN TestPresignedSignatureV4WithForwardedPrefix/prefix_without_trailing_slash === RUN TestPresignedSignatureV4WithForwardedPrefix/prefix_with_trailing_slash --- PASS: TestPresignedSignatureV4WithForwardedPrefix (0.00s) --- PASS: TestPresignedSignatureV4WithForwardedPrefix/prefix_without_trailing_slash (0.00s) --- PASS: TestPresignedSignatureV4WithForwardedPrefix/prefix_with_trailing_slash (0.00s) === RUN TestPresignedSignatureV4WithForwardedPrefixTrailingSlash === RUN TestPresignedSignatureV4WithForwardedPrefixTrailingSlash/bucket_listObjects_with_trailing_slash === RUN TestPresignedSignatureV4WithForwardedPrefixTrailingSlash/prefix_path_with_trailing_slash === RUN TestPresignedSignatureV4WithForwardedPrefixTrailingSlash/api_path_with_trailing_slash --- PASS: TestPresignedSignatureV4WithForwardedPrefixTrailingSlash (0.00s) --- PASS: TestPresignedSignatureV4WithForwardedPrefixTrailingSlash/bucket_listObjects_with_trailing_slash (0.00s) --- PASS: TestPresignedSignatureV4WithForwardedPrefixTrailingSlash/prefix_path_with_trailing_slash (0.00s) --- PASS: TestPresignedSignatureV4WithForwardedPrefixTrailingSlash/api_path_with_trailing_slash (0.00s) === RUN TestGetStringToSignPUT --- PASS: TestGetStringToSignPUT (0.00s) === RUN TestGetStringToSignGETEmptyStringHash --- PASS: TestGetStringToSignGETEmptyStringHash (0.00s) === RUN TestIAMPayloadHashComputation --- PASS: TestIAMPayloadHashComputation (0.00s) === RUN TestS3PayloadHashNoRegression --- PASS: TestS3PayloadHashNoRegression (0.00s) === RUN TestIAMEmptyBodyPayloadHash --- PASS: TestIAMEmptyBodyPayloadHash (0.00s) === RUN TestSTSPayloadHashComputation --- PASS: TestSTSPayloadHashComputation (0.00s) === RUN TestGitHubIssue7080Scenario --- PASS: TestGitHubIssue7080Scenario (0.00s) === RUN TestIAMSignatureServiceMatching --- PASS: TestIAMSignatureServiceMatching (0.00s) === RUN TestStreamingSignatureServiceField --- PASS: TestStreamingSignatureServiceField (0.00s) === RUN TestIAMLargeBodySecurityLimit --- PASS: TestIAMLargeBodySecurityLimit (0.03s) === RUN TestStreamHashRequestBody === RUN TestStreamHashRequestBody/empty_body === RUN TestStreamHashRequestBody/small_payload === RUN TestStreamHashRequestBody/medium_payload === RUN TestStreamHashRequestBody/large_payload_within_limit --- PASS: TestStreamHashRequestBody (0.00s) --- PASS: TestStreamHashRequestBody/empty_body (0.00s) --- PASS: TestStreamHashRequestBody/small_payload (0.00s) --- PASS: TestStreamHashRequestBody/medium_payload (0.00s) --- PASS: TestStreamHashRequestBody/large_payload_within_limit (0.00s) === RUN TestStreamingVsNonStreamingConsistency === RUN TestStreamingVsNonStreamingConsistency/payload_0 === RUN TestStreamingVsNonStreamingConsistency/payload_1 === RUN TestStreamingVsNonStreamingConsistency/payload_2 === RUN TestStreamingVsNonStreamingConsistency/payload_3 === RUN TestStreamingVsNonStreamingConsistency/payload_4 === RUN TestStreamingVsNonStreamingConsistency/payload_5 --- PASS: TestStreamingVsNonStreamingConsistency (0.00s) --- PASS: TestStreamingVsNonStreamingConsistency/payload_0 (0.00s) --- PASS: TestStreamingVsNonStreamingConsistency/payload_1 (0.00s) --- PASS: TestStreamingVsNonStreamingConsistency/payload_2 (0.00s) --- PASS: TestStreamingVsNonStreamingConsistency/payload_3 (0.00s) --- PASS: TestStreamingVsNonStreamingConsistency/payload_4 (0.00s) --- PASS: TestStreamingVsNonStreamingConsistency/payload_5 (0.00s) === RUN TestStreamingWithSizeLimit --- PASS: TestStreamingWithSizeLimit (0.03s) === RUN TestBuildBucketMetadata W0720 21:30:02.416178 bucket_metadata.go:105 Invalid ownership: , bucket: ownershipEmptyStr W0720 21:30:02.416194 bucket_metadata.go:116 owner[id=xxxxx] is invalid, bucket: acpEmptyObject --- PASS: TestBuildBucketMetadata (0.00s) === RUN TestGetBucketMetadata --- PASS: TestGetBucketMetadata (2.01s) === RUN TestCollectCollectionInfoFromTopologyEC --- PASS: TestCollectCollectionInfoFromTopologyEC (0.00s) === RUN TestCollectCollectionInfoFromTopologyMixed --- PASS: TestCollectCollectionInfoFromTopologyMixed (0.00s) === RUN TestCollectCollectionInfoFromTopologyECFileCountMaxDedupe --- PASS: TestCollectCollectionInfoFromTopologyECFileCountMaxDedupe (0.00s) === RUN TestCollectionInfoLogicalSize === RUN TestCollectionInfoLogicalSize/no_garbage === RUN TestCollectionInfoLogicalSize/some_garbage === RUN TestCollectionInfoLogicalSize/all_garbage === RUN TestCollectionInfoLogicalSize/deleted_exceeds_size_clamps_to_zero --- PASS: TestCollectionInfoLogicalSize (0.00s) --- PASS: TestCollectionInfoLogicalSize/no_garbage (0.00s) --- PASS: TestCollectionInfoLogicalSize/some_garbage (0.00s) --- PASS: TestCollectionInfoLogicalSize/all_garbage (0.00s) --- PASS: TestCollectionInfoLogicalSize/deleted_exceeds_size_clamps_to_zero (0.00s) === RUN TestChunkedEncodingMixedFormat --- PASS: TestChunkedEncodingMixedFormat (0.00s) === RUN TestExtractChecksumAlgorithmIsCaseInsensitive --- PASS: TestExtractChecksumAlgorithmIsCaseInsensitive (0.00s) === RUN TestNewSignV4ChunkedReaderStreamingUnsignedPayloadTrailer --- PASS: TestNewSignV4ChunkedReaderStreamingUnsignedPayloadTrailer (0.00s) === RUN TestSignedStreamingUpload --- PASS: TestSignedStreamingUpload (0.00s) === RUN TestSignedStreamingUploadWithTrailer --- PASS: TestSignedStreamingUploadWithTrailer (0.00s) === RUN TestSignedStreamingUploadWithTrailerInvalidSignature --- PASS: TestSignedStreamingUploadWithTrailerInvalidSignature (0.00s) === RUN TestSignedStreamingUploadInvalidSignature --- PASS: TestSignedStreamingUploadInvalidSignature (0.00s) === RUN TestCopySourceWithExclamationMark === RUN TestCopySourceWithExclamationMark/encoded_exclamation_mark_-_different_dest === RUN TestCopySourceWithExclamationMark/unencoded_exclamation_mark_-_different_dest === RUN TestCopySourceWithExclamationMark/encoded_exclamation_mark_-_same_dest === RUN TestCopySourceWithExclamationMark/encoded_path_with_versionId === RUN TestCopySourceWithExclamationMark/unencoded_path_with_versionId === RUN TestCopySourceWithExclamationMark/plus_sign_in_key_with_versionId --- PASS: TestCopySourceWithExclamationMark (0.00s) --- PASS: TestCopySourceWithExclamationMark/encoded_exclamation_mark_-_different_dest (0.00s) --- PASS: TestCopySourceWithExclamationMark/unencoded_exclamation_mark_-_different_dest (0.00s) --- PASS: TestCopySourceWithExclamationMark/encoded_exclamation_mark_-_same_dest (0.00s) --- PASS: TestCopySourceWithExclamationMark/encoded_path_with_versionId (0.00s) --- PASS: TestCopySourceWithExclamationMark/unencoded_path_with_versionId (0.00s) --- PASS: TestCopySourceWithExclamationMark/plus_sign_in_key_with_versionId (0.00s) === RUN TestCopySourceDecodingPlusSign --- PASS: TestCopySourceDecodingPlusSign (0.00s) === RUN TestCopySourceRejectsTraversal === RUN TestCopySourceRejectsTraversal/clean_source_passes === RUN TestCopySourceRejectsTraversal/leading_slash_source_passes === RUN TestCopySourceRejectsTraversal/source_with_versionId_passes === RUN TestCopySourceRejectsTraversal/dotdot_escapes_bucket === RUN TestCopySourceRejectsTraversal/leading_slash_dotdot_escapes_bucket === RUN TestCopySourceRejectsTraversal/percent-encoded_dotdot_escapes_bucket === RUN TestCopySourceRejectsTraversal/encoded_slash_dotdot_escapes_bucket === RUN TestCopySourceRejectsTraversal/backslash_dotdot_escapes_bucket === RUN TestCopySourceRejectsTraversal/nested_dotdot_escapes_bucket === RUN TestCopySourceRejectsTraversal/bare_dot_segment_rejected === RUN TestCopySourceRejectsTraversal/dotdot_bucket_rejected === RUN TestCopySourceRejectsTraversal/dotdot_with_versionId_escapes_bucket === RUN TestCopySourceRejectsTraversal/versionId_encoded_slash_rejected === RUN TestCopySourceRejectsTraversal/versionId_backslash_rejected --- PASS: TestCopySourceRejectsTraversal (0.00s) --- PASS: TestCopySourceRejectsTraversal/clean_source_passes (0.00s) --- PASS: TestCopySourceRejectsTraversal/leading_slash_source_passes (0.00s) --- PASS: TestCopySourceRejectsTraversal/source_with_versionId_passes (0.00s) --- PASS: TestCopySourceRejectsTraversal/dotdot_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/leading_slash_dotdot_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/percent-encoded_dotdot_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/encoded_slash_dotdot_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/backslash_dotdot_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/nested_dotdot_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/bare_dot_segment_rejected (0.00s) --- PASS: TestCopySourceRejectsTraversal/dotdot_bucket_rejected (0.00s) --- PASS: TestCopySourceRejectsTraversal/dotdot_with_versionId_escapes_bucket (0.00s) --- PASS: TestCopySourceRejectsTraversal/versionId_encoded_slash_rejected (0.00s) --- PASS: TestCopySourceRejectsTraversal/versionId_backslash_rejected (0.00s) === RUN TestCopySourceRoutingWithSpecialChars === RUN TestCopySourceRoutingWithSpecialChars/different_keys,_source_has_encoded_! === RUN TestCopySourceRoutingWithSpecialChars/same_key_with_!,_source_encoded,_dest_unencoded === RUN TestCopySourceRoutingWithSpecialChars/same_key_with_!,_both_percent-encoded_differently === RUN TestCopySourceRoutingWithSpecialChars/key_with_+_sign,_source_has_literal_+ === RUN TestCopySourceRoutingWithSpecialChars/key_with_+_sign,_source_has_%2B === RUN TestCopySourceRoutingWithSpecialChars/lowercase_%2f_in_copy_source --- PASS: TestCopySourceRoutingWithSpecialChars (0.00s) --- PASS: TestCopySourceRoutingWithSpecialChars/different_keys,_source_has_encoded_! (0.00s) --- PASS: TestCopySourceRoutingWithSpecialChars/same_key_with_!,_source_encoded,_dest_unencoded (0.00s) --- PASS: TestCopySourceRoutingWithSpecialChars/same_key_with_!,_both_percent-encoded_differently (0.00s) --- PASS: TestCopySourceRoutingWithSpecialChars/key_with_+_sign,_source_has_literal_+ (0.00s) --- PASS: TestCopySourceRoutingWithSpecialChars/key_with_+_sign,_source_has_%2B (0.00s) --- PASS: TestCopySourceRoutingWithSpecialChars/lowercase_%2f_in_copy_source (0.00s) === RUN TestCombineCRC64NVME --- PASS: TestCombineCRC64NVME (0.00s) === RUN TestCombineCRC32 --- PASS: TestCombineCRC32 (0.00s) === RUN TestCombineCRC32C --- PASS: TestCombineCRC32C (0.00s) === RUN TestCombineCRC64NVMEEmptySecond --- PASS: TestCombineCRC64NVMEEmptySecond (0.00s) === RUN TestCombineCRC32EmptySecond --- PASS: TestCombineCRC32EmptySecond (0.00s) === RUN TestComputeFullObjectChecksumCRC64NVME --- PASS: TestComputeFullObjectChecksumCRC64NVME (0.01s) === RUN TestComputeFullObjectChecksumCRC32 --- PASS: TestComputeFullObjectChecksumCRC32 (0.01s) === RUN TestResolveMultipartChecksumType === RUN TestResolveMultipartChecksumType/crc64_default_full === RUN TestResolveMultipartChecksumType/crc64_explicit_full === RUN TestResolveMultipartChecksumType/crc64_composite_rejected === RUN TestResolveMultipartChecksumType/crc32_default_composite === RUN TestResolveMultipartChecksumType/crc32_explicit_composite === RUN TestResolveMultipartChecksumType/crc32_explicit_full === RUN TestResolveMultipartChecksumType/crc32c_explicit_full === RUN TestResolveMultipartChecksumType/sha256_default_composite === RUN TestResolveMultipartChecksumType/sha256_full_rejected === RUN TestResolveMultipartChecksumType/invalid_checksum_type --- PASS: TestResolveMultipartChecksumType (0.00s) --- PASS: TestResolveMultipartChecksumType/crc64_default_full (0.00s) --- PASS: TestResolveMultipartChecksumType/crc64_explicit_full (0.00s) --- PASS: TestResolveMultipartChecksumType/crc64_composite_rejected (0.00s) --- PASS: TestResolveMultipartChecksumType/crc32_default_composite (0.00s) --- PASS: TestResolveMultipartChecksumType/crc32_explicit_composite (0.00s) --- PASS: TestResolveMultipartChecksumType/crc32_explicit_full (0.00s) --- PASS: TestResolveMultipartChecksumType/crc32c_explicit_full (0.00s) --- PASS: TestResolveMultipartChecksumType/sha256_default_composite (0.00s) --- PASS: TestResolveMultipartChecksumType/sha256_full_rejected (0.00s) --- PASS: TestResolveMultipartChecksumType/invalid_checksum_type (0.00s) === RUN TestGetEtagFromEntryPrefersStoredExtendedETag --- PASS: TestGetEtagFromEntryPrefersStoredExtendedETag (0.00s) === RUN TestGetEtagFromEntryFallbacksToFilerETag --- PASS: TestGetEtagFromEntryFallbacksToFilerETag (0.00s) === RUN TestCalculateMultipartETagUsesStoredPartETags --- PASS: TestCalculateMultipartETagUsesStoredPartETags (0.00s) === RUN TestCompletedMultipartChunkBackfillsSSES3MetadataFromUploadEntry --- PASS: TestCompletedMultipartChunkBackfillsSSES3MetadataFromUploadEntry (0.00s) === RUN TestCompletedMultipartChunkPreservesExistingSSES3Metadata --- PASS: TestCompletedMultipartChunkPreservesExistingSSES3Metadata (0.00s) === RUN TestApplyMultipartSSES3HeadersFromUploadEntry === RUN TestApplyMultipartSSES3HeadersFromUploadEntry/backfills_missing_canonical_attributes === RUN TestApplyMultipartSSES3HeadersFromUploadEntry/does_not_clobber_existing_canonical_attributes === RUN TestApplyMultipartSSES3HeadersFromUploadEntry/nil_info_is_a_no-op --- PASS: TestApplyMultipartSSES3HeadersFromUploadEntry (0.00s) --- PASS: TestApplyMultipartSSES3HeadersFromUploadEntry/backfills_missing_canonical_attributes (0.00s) --- PASS: TestApplyMultipartSSES3HeadersFromUploadEntry/does_not_clobber_existing_canonical_attributes (0.00s) --- PASS: TestApplyMultipartSSES3HeadersFromUploadEntry/nil_info_is_a_no-op (0.00s) === RUN TestCompletedMultipartChunkBackfilledIVDecryptsActualCiphertext === RUN TestCompletedMultipartChunkBackfilledIVDecryptsActualCiphertext/part1 === RUN TestCompletedMultipartChunkBackfilledIVDecryptsActualCiphertext/part2 --- PASS: TestCompletedMultipartChunkBackfilledIVDecryptsActualCiphertext (0.09s) --- PASS: TestCompletedMultipartChunkBackfilledIVDecryptsActualCiphertext/part1 (0.02s) --- PASS: TestCompletedMultipartChunkBackfilledIVDecryptsActualCiphertext/part2 (0.02s) === RUN TestCompletedMultipartChunkRejectsPartNumberMultiplierFormula --- PASS: TestCompletedMultipartChunkRejectsPartNumberMultiplierFormula (0.00s) === RUN TestDeleteObjectEntryDemotesNonEmptyDirectoryMarker --- PASS: TestDeleteObjectEntryDemotesNonEmptyDirectoryMarker (0.00s) === RUN TestDeleteObjectEntryTreatsImplicitDirectoryAsSuccessfulNoop --- PASS: TestDeleteObjectEntryTreatsImplicitDirectoryAsSuccessfulNoop (0.00s) === RUN TestDeleteObjectEntryIgnoresConcurrentUpdateNotFound --- PASS: TestDeleteObjectEntryIgnoresConcurrentUpdateNotFound (0.00s) === RUN TestDeleteObjectEntryPropagatesNonDirectoryDeleteErrors --- PASS: TestDeleteObjectEntryPropagatesNonDirectoryDeleteErrors (0.00s) === RUN TestAuthorizeBatchDeleteKey_AwsCanonicalPolicy --- PASS: TestAuthorizeBatchDeleteKey_AwsCanonicalPolicy (0.00s) === RUN TestAuthorizeBatchDeleteKey_PrefixScopedPolicy --- PASS: TestAuthorizeBatchDeleteKey_PrefixScopedPolicy (0.00s) === RUN TestLoadIAMManagerFromConfig_Defaults --- PASS: TestLoadIAMManagerFromConfig_Defaults (0.00s) === RUN TestLoadIAMManagerFromConfig_Overrides --- PASS: TestLoadIAMManagerFromConfig_Overrides (0.00s) === RUN TestLoadIAMManagerFromConfig_PartialDefaults --- PASS: TestLoadIAMManagerFromConfig_PartialDefaults (0.00s) === RUN TestLoadIAMManagerFromConfig_ExplicitEmptyKey --- PASS: TestLoadIAMManagerFromConfig_ExplicitEmptyKey (0.00s) === RUN TestLoadIAMManagerFromConfig_MissingKeyError --- PASS: TestLoadIAMManagerFromConfig_MissingKeyError (0.00s) === RUN TestLoadIAMManagerFromConfig_ExplicitFileDefaultsToDeny --- PASS: TestLoadIAMManagerFromConfig_ExplicitFileDefaultsToDeny (0.00s) === RUN TestLoadIAMManagerFromConfig_NoFileDefaultsToAllow --- PASS: TestLoadIAMManagerFromConfig_NoFileDefaultsToAllow (0.00s) === RUN TestLoadIAMManagerFromConfig_ExplicitFileEnforcesUserScopedPolicy --- PASS: TestLoadIAMManagerFromConfig_ExplicitFileEnforcesUserScopedPolicy (0.00s) === RUN TestEvaluateIAMPolicies_ListBucketWithPrefix --- PASS: TestEvaluateIAMPolicies_ListBucketWithPrefix (0.00s) === RUN TestEvaluateIAMPolicies_ListBucketPrefixCondition --- PASS: TestEvaluateIAMPolicies_ListBucketPrefixCondition (0.00s) === RUN TestEvaluateIAMPolicies_ListBucketVersionsWithPrefix --- PASS: TestEvaluateIAMPolicies_ListBucketVersionsWithPrefix (0.00s) === RUN TestLoadIAMManagerWithNoConfig I0720 21:30:04.545283 config_loader.go:73 Using explicit credential store: memory --- PASS: TestLoadIAMManagerWithNoConfig (0.00s) === RUN TestLoadIAMManagerFromConfig_EmptyConfigWithFallbackKey I0720 21:30:04.545326 config_loader.go:73 Using explicit credential store: memory --- PASS: TestLoadIAMManagerFromConfig_EmptyConfigWithFallbackKey (0.00s) === RUN TestSetIAMIntegrationKeepsAuthDisabledWithoutConfig I0720 21:30:04.545351 config_loader.go:73 Using explicit credential store: memory --- PASS: TestSetIAMIntegrationKeepsAuthDisabledWithoutConfig (0.00s) === RUN TestIdentityFieldsForS3TablesReflection --- PASS: TestIdentityFieldsForS3TablesReflection (0.00s) === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/s3_prefix === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/iam_prefix === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/sts:AssumeRole === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/sts:GetFederationToken === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/sts:GetCallerIdentity === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/coarse_Read_maps_to_s3:GetObject === RUN TestMapBaseActionToS3Format_ServicePrefixPassthrough/coarse_Write_maps_to_s3:PutObject --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/s3_prefix (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/iam_prefix (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/sts:AssumeRole (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/sts:GetFederationToken (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/sts:GetCallerIdentity (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/coarse_Read_maps_to_s3:GetObject (0.00s) --- PASS: TestMapBaseActionToS3Format_ServicePrefixPassthrough/coarse_Write_maps_to_s3:PutObject (0.00s) === RUN TestResolveS3Action_STSActionsPassthrough === RUN TestResolveS3Action_STSActionsPassthrough/nil_request_sts:AssumeRole === RUN TestResolveS3Action_STSActionsPassthrough/with_request_sts:AssumeRole === RUN TestResolveS3Action_STSActionsPassthrough/nil_request_sts:GetFederationToken === RUN TestResolveS3Action_STSActionsPassthrough/with_request_sts:GetFederationToken === RUN TestResolveS3Action_STSActionsPassthrough/nil_request_sts:GetCallerIdentity === RUN TestResolveS3Action_STSActionsPassthrough/with_request_sts:GetCallerIdentity --- PASS: TestResolveS3Action_STSActionsPassthrough (0.00s) --- PASS: TestResolveS3Action_STSActionsPassthrough/nil_request_sts:AssumeRole (0.00s) --- PASS: TestResolveS3Action_STSActionsPassthrough/with_request_sts:AssumeRole (0.00s) --- PASS: TestResolveS3Action_STSActionsPassthrough/nil_request_sts:GetFederationToken (0.00s) --- PASS: TestResolveS3Action_STSActionsPassthrough/with_request_sts:GetFederationToken (0.00s) --- PASS: TestResolveS3Action_STSActionsPassthrough/nil_request_sts:GetCallerIdentity (0.00s) --- PASS: TestResolveS3Action_STSActionsPassthrough/with_request_sts:GetCallerIdentity (0.00s) === RUN TestResolveS3Action_AttributesBeforeVersionId === RUN TestResolveS3Action_AttributesBeforeVersionId/attributes_only === RUN TestResolveS3Action_AttributesBeforeVersionId/attributes_with_versionId === RUN TestResolveS3Action_AttributesBeforeVersionId/versionId_only_GET === RUN TestResolveS3Action_AttributesBeforeVersionId/versionId_only_DELETE --- PASS: TestResolveS3Action_AttributesBeforeVersionId (0.00s) --- PASS: TestResolveS3Action_AttributesBeforeVersionId/attributes_only (0.00s) --- PASS: TestResolveS3Action_AttributesBeforeVersionId/attributes_with_versionId (0.00s) --- PASS: TestResolveS3Action_AttributesBeforeVersionId/versionId_only_GET (0.00s) --- PASS: TestResolveS3Action_AttributesBeforeVersionId/versionId_only_DELETE (0.00s) === RUN TestContentEncodingPreservation === RUN TestContentEncodingPreservation/gzip_encoding === RUN TestContentEncodingPreservation/zstd_encoding === RUN TestContentEncodingPreservation/deflate_encoding === RUN TestContentEncodingPreservation/br_(Brotli)_encoding === RUN TestContentEncodingPreservation/multiple_encodings === RUN TestContentEncodingPreservation/encoding_with_language === RUN TestContentEncodingPreservation/language_only --- PASS: TestContentEncodingPreservation (0.00s) --- PASS: TestContentEncodingPreservation/gzip_encoding (0.00s) --- PASS: TestContentEncodingPreservation/zstd_encoding (0.00s) --- PASS: TestContentEncodingPreservation/deflate_encoding (0.00s) --- PASS: TestContentEncodingPreservation/br_(Brotli)_encoding (0.00s) --- PASS: TestContentEncodingPreservation/multiple_encodings (0.00s) --- PASS: TestContentEncodingPreservation/encoding_with_language (0.00s) --- PASS: TestContentEncodingPreservation/language_only (0.00s) === RUN TestContentEncodingWithOtherHeaders --- PASS: TestContentEncodingWithOtherHeaders (0.00s) === RUN TestEncodePath --- PASS: TestEncodePath (0.00s) === RUN TestEncodePathEqual --- PASS: TestEncodePathEqual (0.00s) === RUN TestS3EndToEndWithJWT === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_CreateBucket s3_end_to_end_test.go:793: S3 Operation: PUT /test-bucket -> 403 (DENIED) === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_ListBucket s3_end_to_end_test.go:793: S3 Operation: GET /test-bucket -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_PutObject s3_end_to_end_test.go:793: S3 Operation: PUT /test-bucket/test-file.txt -> 403 (DENIED) === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_GetObject s3_end_to_end_test.go:793: S3 Operation: GET /test-bucket/test-file.txt -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_HeadObject s3_end_to_end_test.go:793: S3 Operation: HEAD /test-bucket/test-file.txt -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_DeleteObject s3_end_to_end_test.go:793: S3 Operation: DELETE /test-bucket/test-file.txt -> 403 (DENIED) === RUN TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow === RUN TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_CreateBucket s3_end_to_end_test.go:793: S3 Operation: PUT /admin-bucket -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_PutObject s3_end_to_end_test.go:793: S3 Operation: PUT /admin-bucket/admin-file.txt -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_GetObject s3_end_to_end_test.go:793: S3 Operation: GET /admin-bucket/admin-file.txt -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_DeleteObject s3_end_to_end_test.go:793: S3 Operation: DELETE /admin-bucket/admin-file.txt -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_DeleteBucket s3_end_to_end_test.go:793: S3 Operation: DELETE /admin-bucket -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_IP-Restricted_Role === RUN TestS3EndToEndWithJWT/S3_IP-Restricted_Role/S3_IP-Restricted_Role_GetObject s3_end_to_end_test.go:793: S3 Operation: GET /restricted-bucket/file.txt -> 200 (ALLOWED) === RUN TestS3EndToEndWithJWT/S3_IP-Restricted_Role/S3_IP-Restricted_Role_GetObject#01 s3_end_to_end_test.go:793: S3 Operation: GET /restricted-bucket/file.txt -> 403 (DENIED) --- PASS: TestS3EndToEndWithJWT (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_CreateBucket (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_ListBucket (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_PutObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_GetObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_HeadObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Read-Only_Role_Complete_Workflow/S3_Read-Only_Role_Complete_Workflow_DeleteObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_CreateBucket (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_PutObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_GetObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_DeleteObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_Admin_Role_Complete_Workflow/S3_Admin_Role_Complete_Workflow_DeleteBucket (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_IP-Restricted_Role (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_IP-Restricted_Role/S3_IP-Restricted_Role_GetObject (0.00s) --- PASS: TestS3EndToEndWithJWT/S3_IP-Restricted_Role/S3_IP-Restricted_Role_GetObject#01 (0.00s) === RUN TestS3MultipartUploadWithJWT === RUN TestS3MultipartUploadWithJWT/Initialize_Multipart_Upload s3_end_to_end_test.go:793: S3 Operation: POST /multipart-bucket/large-file.txt?uploads -> 404 (DENIED) s3_end_to_end_test.go:799: Non-auth error: Not found === RUN TestS3MultipartUploadWithJWT/Upload_Part s3_end_to_end_test.go:793: S3 Operation: PUT /multipart-bucket/large-file.txt?partNumber=1&uploadId=test-upload-id -> 200 (ALLOWED) === RUN TestS3MultipartUploadWithJWT/List_Parts s3_end_to_end_test.go:793: S3 Operation: GET /multipart-bucket/large-file.txt?uploadId=test-upload-id -> 200 (ALLOWED) === RUN TestS3MultipartUploadWithJWT/Complete_Multipart_Upload s3_end_to_end_test.go:793: S3 Operation: POST /multipart-bucket/large-file.txt?uploadId=test-upload-id -> 404 (DENIED) s3_end_to_end_test.go:799: Non-auth error: Not found --- PASS: TestS3MultipartUploadWithJWT (0.00s) --- PASS: TestS3MultipartUploadWithJWT/Initialize_Multipart_Upload (0.00s) --- PASS: TestS3MultipartUploadWithJWT/Upload_Part (0.00s) --- PASS: TestS3MultipartUploadWithJWT/List_Parts (0.00s) --- PASS: TestS3MultipartUploadWithJWT/Complete_Multipart_Upload (0.00s) === RUN TestS3ListObjectsV2PrefixCondition === RUN TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_matching_prefix_query_param_and_empty_objectKey === RUN TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_matching_prefix_propagated_as_objectKey === RUN TestS3ListObjectsV2PrefixCondition/ListObjectsV1_with_matching_prefix_query_param === RUN TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_non-matching_prefix_should_be_denied === RUN TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_no_prefix_should_be_denied --- PASS: TestS3ListObjectsV2PrefixCondition (0.00s) --- PASS: TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_matching_prefix_query_param_and_empty_objectKey (0.00s) --- PASS: TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_matching_prefix_propagated_as_objectKey (0.00s) --- PASS: TestS3ListObjectsV2PrefixCondition/ListObjectsV1_with_matching_prefix_query_param (0.00s) --- PASS: TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_non-matching_prefix_should_be_denied (0.00s) --- PASS: TestS3ListObjectsV2PrefixCondition/ListObjectsV2_with_no_prefix_should_be_denied (0.00s) === RUN TestS3CORSWithJWT --- PASS: TestS3CORSWithJWT (0.00s) === RUN TestS3PerformanceWithIAM s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-0.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-1.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-2.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-3.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-4.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-5.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-6.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-7.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-8.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-9.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-10.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-11.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-12.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-13.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-14.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-15.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-16.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-17.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-18.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-19.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-20.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-21.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-22.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-23.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-24.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-25.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-26.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-27.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-28.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-29.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-30.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-31.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-32.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-33.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-34.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-35.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-36.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-37.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-38.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-39.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-40.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-41.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-42.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-43.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-44.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-45.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-46.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-47.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-48.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-49.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-50.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-51.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-52.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-53.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-54.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-55.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-56.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-57.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-58.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-59.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-60.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-61.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-62.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-63.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-64.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-65.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-66.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-67.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-68.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-69.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-70.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-71.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-72.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-73.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-74.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-75.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-76.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-77.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-78.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-79.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-80.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-81.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-82.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-83.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-84.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-85.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-86.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-87.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-88.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-89.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-90.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-91.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-92.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-93.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-94.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-95.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-96.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-97.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-98.txt -> 200 (ALLOWED) s3_end_to_end_test.go:793: S3 Operation: GET /perf-bucket/file-99.txt -> 200 (ALLOWED) s3_end_to_end_test.go:432: Performance Results: s3_end_to_end_test.go:433: - Total requests: 100 s3_end_to_end_test.go:434: - Total time: 14.871093ms s3_end_to_end_test.go:435: - Average latency: 148.71µs s3_end_to_end_test.go:436: - Requests per second: 6724.46 --- PASS: TestS3PerformanceWithIAM (0.02s) === RUN TestS3AuthenticationDenied === RUN TestS3AuthenticationDenied/no_authorization_header === RUN TestS3AuthenticationDenied/empty_bearer_token === RUN TestS3AuthenticationDenied/invalid_jwt_token === RUN TestS3AuthenticationDenied/malformed_authorization_header === RUN TestS3AuthenticationDenied/expired_jwt_token --- PASS: TestS3AuthenticationDenied (0.00s) --- PASS: TestS3AuthenticationDenied/no_authorization_header (0.00s) --- PASS: TestS3AuthenticationDenied/empty_bearer_token (0.00s) --- PASS: TestS3AuthenticationDenied/invalid_jwt_token (0.00s) --- PASS: TestS3AuthenticationDenied/malformed_authorization_header (0.00s) --- PASS: TestS3AuthenticationDenied/expired_jwt_token (0.00s) === RUN TestS3IAMOnlyModeRejectsAnonymous --- PASS: TestS3IAMOnlyModeRejectsAnonymous (0.00s) === RUN TestCheckPolicyWithEntry === RUN TestCheckPolicyWithEntry/object_with_public_tag_-_allow === RUN TestCheckPolicyWithEntry/object_with_private_tag_-_indeterminate_(no_matching_statement) === RUN TestCheckPolicyWithEntry/object_with_no_tags_-_indeterminate_(no_matching_statement) === RUN TestCheckPolicyWithEntry/nil_entry_-_indeterminate_(condition_cannot_be_evaluated) === RUN TestCheckPolicyWithEntry/object_with_multiple_tags_including_public_-_allow --- PASS: TestCheckPolicyWithEntry (0.00s) --- PASS: TestCheckPolicyWithEntry/object_with_public_tag_-_allow (0.00s) --- PASS: TestCheckPolicyWithEntry/object_with_private_tag_-_indeterminate_(no_matching_statement) (0.00s) --- PASS: TestCheckPolicyWithEntry/object_with_no_tags_-_indeterminate_(no_matching_statement) (0.00s) --- PASS: TestCheckPolicyWithEntry/nil_entry_-_indeterminate_(condition_cannot_be_evaluated) (0.00s) --- PASS: TestCheckPolicyWithEntry/object_with_multiple_tags_including_public_-_allow (0.00s) === RUN TestCheckPolicyWithEntryNoPolicyForBucket --- PASS: TestCheckPolicyWithEntryNoPolicyForBucket (0.00s) === RUN TestCheckPolicyWithEntryNilPolicyEngine --- PASS: TestCheckPolicyWithEntryNilPolicyEngine (0.00s) === RUN TestCheckPolicyWithEntryDenyPolicy === RUN TestCheckPolicyWithEntryDenyPolicy/public_object_-_allow === RUN TestCheckPolicyWithEntryDenyPolicy/confidential_object_-_deny === RUN TestCheckPolicyWithEntryDenyPolicy/no_classification_tag_-_allow --- PASS: TestCheckPolicyWithEntryDenyPolicy (0.00s) --- PASS: TestCheckPolicyWithEntryDenyPolicy/public_object_-_allow (0.00s) --- PASS: TestCheckPolicyWithEntryDenyPolicy/confidential_object_-_deny (0.00s) --- PASS: TestCheckPolicyWithEntryDenyPolicy/no_classification_tag_-_allow (0.00s) === RUN TestHasPolicyForBucket --- PASS: TestHasPolicyForBucket (0.00s) === RUN TestGranularActionMappingSecurity === RUN TestGranularActionMappingSecurity/delete_object_security_fix s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: DELETE object operations should map to s3:DeleteObject, not s3:PutObject s3_granular_action_security_test.go:168: Problem Fixed: Old mapping incorrectly mapped DELETE object to s3:PutObject, allowing users with only PUT permissions to delete objects - a critical security flaw s3_granular_action_security_test.go:169: Granular Action: s3:DeleteObject === RUN TestGranularActionMappingSecurity/get_object_acl_precision s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: GET object ACL should map to s3:GetObjectAcl, not generic s3:GetObject s3_granular_action_security_test.go:168: Problem Fixed: Old mapping would allow users with s3:GetObject permission to read ACLs, potentially exposing sensitive permission information s3_granular_action_security_test.go:169: Granular Action: s3:GetObjectAcl === RUN TestGranularActionMappingSecurity/put_object_tagging_precision s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: PUT object tagging should map to s3:PutObjectTagging, not generic s3:PutObject s3_granular_action_security_test.go:168: Problem Fixed: Old mapping couldn't distinguish between actual object uploads and metadata operations like tagging, making fine-grained permissions impossible s3_granular_action_security_test.go:169: Granular Action: s3:PutObjectTagging === RUN TestGranularActionMappingSecurity/multipart_upload_precision s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: Multipart upload initiation should map to s3:CreateMultipartUpload s3_granular_action_security_test.go:168: Problem Fixed: Old mapping would treat multipart operations as generic s3:PutObject, preventing policies that allow regular uploads but restrict large multipart operations s3_granular_action_security_test.go:169: Granular Action: s3:CreateMultipartUpload === RUN TestGranularActionMappingSecurity/bucket_policy_vs_bucket_creation s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: Bucket policy modifications should map to s3:PutBucketPolicy, not s3:CreateBucket s3_granular_action_security_test.go:168: Problem Fixed: Old mapping couldn't distinguish between creating buckets and modifying bucket policies, potentially allowing unauthorized policy changes s3_granular_action_security_test.go:169: Granular Action: s3:PutBucketPolicy === RUN TestGranularActionMappingSecurity/list_vs_read_distinction s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: Listing multipart uploads should map to s3:ListBucketMultipartUploads s3_granular_action_security_test.go:168: Problem Fixed: Old mapping would use generic s3:ListBucket for all bucket operations, preventing fine-grained control over who can see ongoing multipart operations s3_granular_action_security_test.go:169: Granular Action: s3:ListBucketMultipartUploads === RUN TestGranularActionMappingSecurity/delete_object_tagging_precision s3_granular_action_security_test.go:167: SECURITY IMPROVEMENT: Delete object tagging should map to s3:DeleteObjectTagging, not s3:DeleteObject s3_granular_action_security_test.go:168: Problem Fixed: Old mapping couldn't distinguish between deleting objects and deleting tags, preventing policies that allow tag management but not object deletion s3_granular_action_security_test.go:169: Granular Action: s3:DeleteObjectTagging --- PASS: TestGranularActionMappingSecurity (0.00s) --- PASS: TestGranularActionMappingSecurity/delete_object_security_fix (0.00s) --- PASS: TestGranularActionMappingSecurity/get_object_acl_precision (0.00s) --- PASS: TestGranularActionMappingSecurity/put_object_tagging_precision (0.00s) --- PASS: TestGranularActionMappingSecurity/multipart_upload_precision (0.00s) --- PASS: TestGranularActionMappingSecurity/bucket_policy_vs_bucket_creation (0.00s) --- PASS: TestGranularActionMappingSecurity/list_vs_read_distinction (0.00s) --- PASS: TestGranularActionMappingSecurity/delete_object_tagging_precision (0.00s) === RUN TestBackwardCompatibilityFallback === RUN TestBackwardCompatibilityFallback/generic_read_fallback s3_granular_action_security_test.go:237: COMPATIBILITY: Generic read operations should fall back to s3:GetObject for compatibility - s3:GetObject === RUN TestBackwardCompatibilityFallback/generic_write_fallback s3_granular_action_security_test.go:237: COMPATIBILITY: Generic write operations should fall back to s3:PutObject for compatibility - s3:PutObject === RUN TestBackwardCompatibilityFallback/already_granular_passthrough s3_granular_action_security_test.go:237: COMPATIBILITY: Already granular actions should pass through unchanged - s3:GetBucketLocation === RUN TestBackwardCompatibilityFallback/unknown_action_conversion s3_granular_action_security_test.go:237: COMPATIBILITY: Unknown actions should be converted to S3 format for consistency - s3:CustomAction --- PASS: TestBackwardCompatibilityFallback (0.00s) --- PASS: TestBackwardCompatibilityFallback/generic_read_fallback (0.00s) --- PASS: TestBackwardCompatibilityFallback/generic_write_fallback (0.00s) --- PASS: TestBackwardCompatibilityFallback/already_granular_passthrough (0.00s) --- PASS: TestBackwardCompatibilityFallback/unknown_action_conversion (0.00s) === RUN TestPolicyEnforcementScenarios === RUN TestPolicyEnforcementScenarios/allow_read_deny_acl_access s3_granular_action_security_test.go:338: SECURITY SCENARIO: allow_read_deny_acl_access s3_granular_action_security_test.go:339: Expected Action: s3:GetObjectAcl s3_granular_action_security_test.go:340: Security Benefit: Policy allows reading objects but denies ACL access - granular actions enable this distinction s3_granular_action_security_test.go:341: Policy Example: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::sensitive-bucket/*" } ] } === RUN TestPolicyEnforcementScenarios/allow_tagging_deny_object_modification s3_granular_action_security_test.go:338: SECURITY SCENARIO: allow_tagging_deny_object_modification s3_granular_action_security_test.go:339: Expected Action: s3:PutObjectTagging s3_granular_action_security_test.go:340: Security Benefit: Policy allows tag management but prevents actual object uploads - critical for metadata-only roles s3_granular_action_security_test.go:341: Policy Example: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:PutObjectTagging", "s3:DeleteObjectTagging"], "Resource": "arn:aws:s3:::data-bucket/*" } ] } === RUN TestPolicyEnforcementScenarios/restrict_multipart_uploads s3_granular_action_security_test.go:338: SECURITY SCENARIO: restrict_multipart_uploads s3_granular_action_security_test.go:339: Expected Action: s3:CreateMultipartUpload s3_granular_action_security_test.go:340: Security Benefit: Policy allows regular uploads but blocks large multipart uploads - prevents resource abuse s3_granular_action_security_test.go:341: Policy Example: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::uploads/*" }, { "Effect": "Deny", "Action": ["s3:CreateMultipartUpload", "s3:UploadPart"], "Resource": "arn:aws:s3:::uploads/*" } ] } --- PASS: TestPolicyEnforcementScenarios (0.00s) --- PASS: TestPolicyEnforcementScenarios/allow_read_deny_acl_access (0.00s) --- PASS: TestPolicyEnforcementScenarios/allow_tagging_deny_object_modification (0.00s) --- PASS: TestPolicyEnforcementScenarios/restrict_multipart_uploads (0.00s) === RUN TestDeleteObjectPolicyEnforcement === RUN TestDeleteObjectPolicyEnforcement/delete_object_maps_to_correct_action s3_granular_action_security_test.go:414: ARCHITECTURAL FIX VERIFIED: delete_object_maps_to_correct_action s3_granular_action_security_test.go:415: Method: DELETE -> S3 Action: s3:DeleteObject s3_granular_action_security_test.go:416: Policy Scenario: Policy that denies s3:DeleteObject but allows s3:PutObject should now work correctly === RUN TestDeleteObjectPolicyEnforcement/put_object_maps_to_correct_action s3_granular_action_security_test.go:414: ARCHITECTURAL FIX VERIFIED: put_object_maps_to_correct_action s3_granular_action_security_test.go:415: Method: PUT -> S3 Action: s3:PutObject s3_granular_action_security_test.go:416: Policy Scenario: Policy that allows s3:PutObject but denies s3:DeleteObject should allow uploads === RUN TestDeleteObjectPolicyEnforcement/batch_delete_maps_to_delete_action s3_granular_action_security_test.go:414: ARCHITECTURAL FIX VERIFIED: batch_delete_maps_to_delete_action s3_granular_action_security_test.go:415: Method: POST -> S3 Action: s3:DeleteObject s3_granular_action_security_test.go:416: Policy Scenario: Batch delete operations should also map to s3:DeleteObject --- PASS: TestDeleteObjectPolicyEnforcement (0.00s) --- PASS: TestDeleteObjectPolicyEnforcement/delete_object_maps_to_correct_action (0.00s) --- PASS: TestDeleteObjectPolicyEnforcement/put_object_maps_to_correct_action (0.00s) --- PASS: TestDeleteObjectPolicyEnforcement/batch_delete_maps_to_delete_action (0.00s) === RUN TestFineGrainedPolicyExample s3_granular_action_security_test.go:486: Testing Fine-Grained Policy: s3_granular_action_security_test.go:487: { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowObjectUploads", "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::test-bucket/*" }, { "Sid": "DenyObjectDeletion", "Effect": "Deny", "Action": "s3:DeleteObject", "Resource": "arn:aws:s3:::test-bucket/*" } ] } === RUN TestFineGrainedPolicyExample/PUT_object s3_granular_action_security_test.go:519: [ALLOWED] PUT object -> s3:PutObject s3_granular_action_security_test.go:520: Rationale: Policy explicitly allows s3:PutObject - upload should succeed === RUN TestFineGrainedPolicyExample/DELETE_object s3_granular_action_security_test.go:519: [DENIED] DELETE object -> s3:DeleteObject s3_granular_action_security_test.go:520: Rationale: Policy explicitly denies s3:DeleteObject - deletion should be blocked === RUN TestFineGrainedPolicyExample/Batch_DELETE s3_granular_action_security_test.go:519: [DENIED] Batch DELETE -> s3:DeleteObject s3_granular_action_security_test.go:520: Rationale: Policy explicitly denies s3:DeleteObject - batch deletion should be blocked === NAME TestFineGrainedPolicyExample s3_granular_action_security_test.go:524: ARCHITECTURAL LIMITATION RESOLVED! s3_granular_action_security_test.go:525: Fine-grained policies like 'allow PUT but deny DELETE' now work correctly s3_granular_action_security_test.go:526: The policy engine can distinguish between s3:PutObject and s3:DeleteObject --- PASS: TestFineGrainedPolicyExample (0.00s) --- PASS: TestFineGrainedPolicyExample/PUT_object (0.00s) --- PASS: TestFineGrainedPolicyExample/DELETE_object (0.00s) --- PASS: TestFineGrainedPolicyExample/Batch_DELETE (0.00s) === RUN TestCoarseActionResolution === RUN TestCoarseActionResolution/PUT_with_ACTION_WRITE_resolves_to_PutObject s3_granular_action_security_test.go:595: SUCCESS: PUT_with_ACTION_WRITE_resolves_to_PutObject s3_granular_action_security_test.go:596: Input: PUT /test-bucket/test-file.txt + ACTION_WRITE s3_granular_action_security_test.go:597: Output: s3:PutObject s3_granular_action_security_test.go:598: Policy impact: Policy allowing s3:PutObject should match PUT requests === RUN TestCoarseActionResolution/DELETE_with_ACTION_WRITE_resolves_to_DeleteObject s3_granular_action_security_test.go:595: SUCCESS: DELETE_with_ACTION_WRITE_resolves_to_DeleteObject s3_granular_action_security_test.go:596: Input: DELETE /test-bucket/test-file.txt + ACTION_WRITE s3_granular_action_security_test.go:597: Output: s3:DeleteObject s3_granular_action_security_test.go:598: Policy impact: Policy denying s3:DeleteObject should block DELETE requests === RUN TestCoarseActionResolution/batch_DELETE_with_ACTION_WRITE_resolves_to_DeleteObject s3_granular_action_security_test.go:595: SUCCESS: batch_DELETE_with_ACTION_WRITE_resolves_to_DeleteObject s3_granular_action_security_test.go:596: Input: POST /test-bucket + ACTION_WRITE s3_granular_action_security_test.go:597: Output: s3:DeleteObject s3_granular_action_security_test.go:598: Policy impact: Policy denying s3:DeleteObject should block batch DELETE === RUN TestCoarseActionResolution/POST_multipart_with_ACTION_WRITE_resolves_to_CreateMultipartUpload s3_granular_action_security_test.go:595: SUCCESS: POST_multipart_with_ACTION_WRITE_resolves_to_CreateMultipartUpload s3_granular_action_security_test.go:596: Input: POST /test-bucket/large-file.mp4 + ACTION_WRITE s3_granular_action_security_test.go:597: Output: s3:CreateMultipartUpload s3_granular_action_security_test.go:598: Policy impact: Policy allowing s3:PutObject but denying s3:CreateMultipartUpload can now work === NAME TestCoarseActionResolution s3_granular_action_security_test.go:602: === ARCHITECTURAL LIMITATION RESOLVED === s3_granular_action_security_test.go:603: Handlers can use coarse ACTION_WRITE constant, and the context-aware s3_granular_action_security_test.go:604: resolver will map it to the correct specific S3 action (PutObject, s3_granular_action_security_test.go:605: DeleteObject, CreateMultipartUpload, etc.) based on HTTP method and s3_granular_action_security_test.go:606: query parameters. This enables fine-grained bucket policies like: s3_granular_action_security_test.go:607: - Allow s3:PutObject but Deny s3:DeleteObject (append-only) s3_granular_action_security_test.go:608: - Allow regular uploads but Deny multipart (size limits) --- PASS: TestCoarseActionResolution (0.00s) --- PASS: TestCoarseActionResolution/PUT_with_ACTION_WRITE_resolves_to_PutObject (0.00s) --- PASS: TestCoarseActionResolution/DELETE_with_ACTION_WRITE_resolves_to_DeleteObject (0.00s) --- PASS: TestCoarseActionResolution/batch_DELETE_with_ACTION_WRITE_resolves_to_DeleteObject (0.00s) --- PASS: TestCoarseActionResolution/POST_multipart_with_ACTION_WRITE_resolves_to_CreateMultipartUpload (0.00s) === RUN TestOIDCBearerEnforcesRoleTrustPolicy --- PASS: TestOIDCBearerEnforcesRoleTrustPolicy (0.05s) === RUN TestGroupsRolesPolicyEnforcement === RUN TestGroupsRolesPolicyEnforcement/group_member_gets_team-a_bucket_only === RUN TestGroupsRolesPolicyEnforcement/role_holder_gets_team-b_bucket_only === RUN TestGroupsRolesPolicyEnforcement/member_of_both_gets_both_buckets_in_one_session === RUN TestGroupsRolesPolicyEnforcement/user_without_matching_claims_is_denied_everywhere --- PASS: TestGroupsRolesPolicyEnforcement (0.00s) --- PASS: TestGroupsRolesPolicyEnforcement/group_member_gets_team-a_bucket_only (0.00s) --- PASS: TestGroupsRolesPolicyEnforcement/role_holder_gets_team-b_bucket_only (0.00s) --- PASS: TestGroupsRolesPolicyEnforcement/member_of_both_gets_both_buckets_in_one_session (0.00s) --- PASS: TestGroupsRolesPolicyEnforcement/user_without_matching_claims_is_denied_everywhere (0.00s) === RUN TestGroupsRolesSessionRequestContext --- PASS: TestGroupsRolesSessionRequestContext (0.00s) === RUN TestSelectPrimaryRole === RUN TestSelectPrimaryRole/empty_roles_returns_empty === RUN TestSelectPrimaryRole/single_role_returns_that_role === RUN TestSelectPrimaryRole/multiple_roles_returns_first === RUN TestSelectPrimaryRole/order_matters === RUN TestSelectPrimaryRole/complex_enterprise_roles --- PASS: TestSelectPrimaryRole (0.00s) --- PASS: TestSelectPrimaryRole/empty_roles_returns_empty (0.00s) --- PASS: TestSelectPrimaryRole/single_role_returns_that_role (0.00s) --- PASS: TestSelectPrimaryRole/multiple_roles_returns_first (0.00s) --- PASS: TestSelectPrimaryRole/order_matters (0.00s) --- PASS: TestSelectPrimaryRole/complex_enterprise_roles (0.00s) === RUN TestJWTAuthenticationFlow === RUN TestJWTAuthenticationFlow/Read-Only_JWT_Authentication === RUN TestJWTAuthenticationFlow/Read-Only_JWT_Authentication/Read === RUN TestJWTAuthenticationFlow/Read-Only_JWT_Authentication/Write === RUN TestJWTAuthenticationFlow/Read-Only_JWT_Authentication/List === RUN TestJWTAuthenticationFlow/Admin_JWT_Authentication === RUN TestJWTAuthenticationFlow/Admin_JWT_Authentication/Read === RUN TestJWTAuthenticationFlow/Admin_JWT_Authentication/Write === RUN TestJWTAuthenticationFlow/Admin_JWT_Authentication/DeleteBucket --- PASS: TestJWTAuthenticationFlow (0.00s) --- PASS: TestJWTAuthenticationFlow/Read-Only_JWT_Authentication (0.00s) --- PASS: TestJWTAuthenticationFlow/Read-Only_JWT_Authentication/Read (0.00s) --- PASS: TestJWTAuthenticationFlow/Read-Only_JWT_Authentication/Write (0.00s) --- PASS: TestJWTAuthenticationFlow/Read-Only_JWT_Authentication/List (0.00s) --- PASS: TestJWTAuthenticationFlow/Admin_JWT_Authentication (0.00s) --- PASS: TestJWTAuthenticationFlow/Admin_JWT_Authentication/Read (0.00s) --- PASS: TestJWTAuthenticationFlow/Admin_JWT_Authentication/Write (0.00s) --- PASS: TestJWTAuthenticationFlow/Admin_JWT_Authentication/DeleteBucket (0.00s) === RUN TestJWTTokenValidation === RUN TestJWTTokenValidation/Empty_token === RUN TestJWTTokenValidation/Invalid_token_format === RUN TestJWTTokenValidation/Expired_token --- PASS: TestJWTTokenValidation (0.00s) --- PASS: TestJWTTokenValidation/Empty_token (0.00s) --- PASS: TestJWTTokenValidation/Invalid_token_format (0.00s) --- PASS: TestJWTTokenValidation/Expired_token (0.00s) === RUN TestRequestContextExtraction === RUN TestRequestContextExtraction/Standard_request_with_IP === RUN TestRequestContextExtraction/Request_with_X-Real-IP --- PASS: TestRequestContextExtraction (0.00s) --- PASS: TestRequestContextExtraction/Standard_request_with_IP (0.00s) --- PASS: TestRequestContextExtraction/Request_with_X-Real-IP (0.00s) === RUN TestIPBasedPolicyEnforcement === RUN TestIPBasedPolicyEnforcement/Allow_from_office_IP === RUN TestIPBasedPolicyEnforcement/Block_from_external_IP === RUN TestIPBasedPolicyEnforcement/Allow_from_internal_range --- PASS: TestIPBasedPolicyEnforcement (0.00s) --- PASS: TestIPBasedPolicyEnforcement/Allow_from_office_IP (0.00s) --- PASS: TestIPBasedPolicyEnforcement/Block_from_external_IP (0.00s) --- PASS: TestIPBasedPolicyEnforcement/Allow_from_internal_range (0.00s) === RUN TestListPartsActionMapping === RUN TestListPartsActionMapping/get_object_without_uploadId === RUN TestListPartsActionMapping/get_object_with_uploadId === RUN TestListPartsActionMapping/get_object_with_uploadId_and_other_params === RUN TestListPartsActionMapping/get_object_with_versionId === RUN TestListPartsActionMapping/get_object_acl_without_uploadId === RUN TestListPartsActionMapping/post_multipart_upload_without_uploadId --- PASS: TestListPartsActionMapping (0.00s) --- PASS: TestListPartsActionMapping/get_object_without_uploadId (0.00s) --- PASS: TestListPartsActionMapping/get_object_with_uploadId (0.00s) --- PASS: TestListPartsActionMapping/get_object_with_uploadId_and_other_params (0.00s) --- PASS: TestListPartsActionMapping/get_object_with_versionId (0.00s) --- PASS: TestListPartsActionMapping/get_object_acl_without_uploadId (0.00s) --- PASS: TestListPartsActionMapping/post_multipart_upload_without_uploadId (0.00s) === RUN TestListPartsActionMappingSecurityScenarios === RUN TestListPartsActionMappingSecurityScenarios/privilege_separation_listparts_vs_getobject === RUN TestListPartsActionMappingSecurityScenarios/policy_enforcement_precision --- PASS: TestListPartsActionMappingSecurityScenarios (0.00s) --- PASS: TestListPartsActionMappingSecurityScenarios/privilege_separation_listparts_vs_getobject (0.00s) --- PASS: TestListPartsActionMappingSecurityScenarios/policy_enforcement_precision (0.00s) === RUN TestListPartsActionRealWorldScenarios === RUN TestListPartsActionRealWorldScenarios/large_file_upload_workflow === RUN TestListPartsActionRealWorldScenarios/edge_case_upload_ids --- PASS: TestListPartsActionRealWorldScenarios (0.00s) --- PASS: TestListPartsActionRealWorldScenarios/large_file_upload_workflow (0.00s) --- PASS: TestListPartsActionRealWorldScenarios/edge_case_upload_ids (0.00s) === RUN TestCalculateIVWithOffset === RUN TestCalculateIVWithOffset/BlockAligned_0 === RUN TestCalculateIVWithOffset/BlockAligned_16 === RUN TestCalculateIVWithOffset/BlockAligned_32 === RUN TestCalculateIVWithOffset/BlockAligned_48 === RUN TestCalculateIVWithOffset/NonAligned_1 === RUN TestCalculateIVWithOffset/NonAligned_5 === RUN TestCalculateIVWithOffset/NonAligned_10 === RUN TestCalculateIVWithOffset/NonAligned_15 === RUN TestCalculateIVWithOffset/NonAligned_17 === RUN TestCalculateIVWithOffset/NonAligned_21 === RUN TestCalculateIVWithOffset/NonAligned_33 === RUN TestCalculateIVWithOffset/NonAligned_47 === RUN TestCalculateIVWithOffset/LargeOffset --- PASS: TestCalculateIVWithOffset (0.00s) --- PASS: TestCalculateIVWithOffset/BlockAligned_0 (0.00s) --- PASS: TestCalculateIVWithOffset/BlockAligned_16 (0.00s) --- PASS: TestCalculateIVWithOffset/BlockAligned_32 (0.00s) --- PASS: TestCalculateIVWithOffset/BlockAligned_48 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_1 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_5 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_10 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_15 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_17 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_21 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_33 (0.00s) --- PASS: TestCalculateIVWithOffset/NonAligned_47 (0.00s) --- PASS: TestCalculateIVWithOffset/LargeOffset (0.00s) === RUN TestCTRDecryptionWithNonBlockAlignedOffset === RUN TestCTRDecryptionWithNonBlockAlignedOffset/A === RUN TestCTRDecryptionWithNonBlockAlignedOffset/B === RUN TestCTRDecryptionWithNonBlockAlignedOffset/F === RUN TestCTRDecryptionWithNonBlockAlignedOffset/K === RUN TestCTRDecryptionWithNonBlockAlignedOffset/P === RUN TestCTRDecryptionWithNonBlockAlignedOffset/Q === RUN TestCTRDecryptionWithNonBlockAlignedOffset/R === RUN TestCTRDecryptionWithNonBlockAlignedOffset/V === RUN TestCTRDecryptionWithNonBlockAlignedOffset/a === RUN TestCTRDecryptionWithNonBlockAlignedOffset/b === RUN TestCTRDecryptionWithNonBlockAlignedOffset/p === RUN TestCTRDecryptionWithNonBlockAlignedOffset/q === RUN TestCTRDecryptionWithNonBlockAlignedOffset/¥ === RUN TestCTRDecryptionWithNonBlockAlignedOffset/ȵ --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/A (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/B (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/F (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/K (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/P (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/Q (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/R (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/V (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/a (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/b (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/p (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/q (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/¥ (0.00s) --- PASS: TestCTRDecryptionWithNonBlockAlignedOffset/ȵ (0.00s) === RUN TestCTRRangeRequestSimulation === RUN TestCTRRangeRequestSimulation/First_byte === RUN TestCTRRangeRequestSimulation/First_100_bytes === RUN TestCTRRangeRequestSimulation/Mid-block_range === RUN TestCTRRangeRequestSimulation/Single_mid-block_byte === RUN TestCTRRangeRequestSimulation/Cross-block_range === RUN TestCTRRangeRequestSimulation/Large_range === RUN TestCTRRangeRequestSimulation/Tail_range --- PASS: TestCTRRangeRequestSimulation (0.01s) --- PASS: TestCTRRangeRequestSimulation/First_byte (0.00s) --- PASS: TestCTRRangeRequestSimulation/First_100_bytes (0.00s) --- PASS: TestCTRRangeRequestSimulation/Mid-block_range (0.00s) --- PASS: TestCTRRangeRequestSimulation/Single_mid-block_byte (0.00s) --- PASS: TestCTRRangeRequestSimulation/Cross-block_range (0.00s) --- PASS: TestCTRRangeRequestSimulation/Large_range (0.00s) --- PASS: TestCTRRangeRequestSimulation/Tail_range (0.00s) === RUN TestCTRDecryptionWithIOReader === RUN TestCTRDecryptionWithIOReader/A === RUN TestCTRDecryptionWithIOReader/F === RUN TestCTRDecryptionWithIOReader/K === RUN TestCTRDecryptionWithIOReader/Q === RUN TestCTRDecryptionWithIOReader/R === RUN TestCTRDecryptionWithIOReader/_ --- PASS: TestCTRDecryptionWithIOReader (0.00s) --- PASS: TestCTRDecryptionWithIOReader/A (0.00s) --- PASS: TestCTRDecryptionWithIOReader/F (0.00s) --- PASS: TestCTRDecryptionWithIOReader/K (0.00s) --- PASS: TestCTRDecryptionWithIOReader/Q (0.00s) --- PASS: TestCTRDecryptionWithIOReader/R (0.00s) --- PASS: TestCTRDecryptionWithIOReader/_ (0.00s) === RUN TestSSES3EndToEndSmallFile === RUN TestSSES3EndToEndSmallFile/tiny_file_(10_bytes) === RUN TestSSES3EndToEndSmallFile/small_file_(50_bytes) === RUN TestSSES3EndToEndSmallFile/medium_file_(256_bytes) --- PASS: TestSSES3EndToEndSmallFile (0.00s) --- PASS: TestSSES3EndToEndSmallFile/tiny_file_(10_bytes) (0.00s) --- PASS: TestSSES3EndToEndSmallFile/small_file_(50_bytes) (0.00s) --- PASS: TestSSES3EndToEndSmallFile/medium_file_(256_bytes) (0.00s) === RUN TestSSES3EndToEndChunkedFile --- PASS: TestSSES3EndToEndChunkedFile (0.00s) === RUN TestSSES3EndToEndWithDetectPrimaryType === RUN TestSSES3EndToEndWithDetectPrimaryType/Inline_SSE-S3_file_(no_chunks) === RUN TestSSES3EndToEndWithDetectPrimaryType/Single_chunk_SSE-S3_file === RUN TestSSES3EndToEndWithDetectPrimaryType/SSE-KMS_file_(has_KMS_key_ID) --- PASS: TestSSES3EndToEndWithDetectPrimaryType (0.00s) --- PASS: TestSSES3EndToEndWithDetectPrimaryType/Inline_SSE-S3_file_(no_chunks) (0.00s) --- PASS: TestSSES3EndToEndWithDetectPrimaryType/Single_chunk_SSE-S3_file (0.00s) --- PASS: TestSSES3EndToEndWithDetectPrimaryType/SSE-KMS_file_(has_KMS_key_ID) (0.00s) === RUN TestBuildMultipartSSES3Reader_PerChunkKeys --- PASS: TestBuildMultipartSSES3Reader_PerChunkKeys (0.00s) === RUN TestBuildMultipartSSES3Reader_InvalidIVLength === RUN TestBuildMultipartSSES3Reader_InvalidIVLength/missing_IV === RUN TestBuildMultipartSSES3Reader_InvalidIVLength/short_IV --- PASS: TestBuildMultipartSSES3Reader_InvalidIVLength (0.00s) --- PASS: TestBuildMultipartSSES3Reader_InvalidIVLength/missing_IV (0.00s) --- PASS: TestBuildMultipartSSES3Reader_InvalidIVLength/short_IV (0.00s) === RUN TestBuildMultipartSSES3Reader_RejectsBadChunkBeforeAnyFetch --- PASS: TestBuildMultipartSSES3Reader_RejectsBadChunkBeforeAnyFetch (0.00s) === RUN TestSetKEKPassphraseEnablesEncryptedRoundTrip --- PASS: TestSetKEKPassphraseEnablesEncryptedRoundTrip (0.00s) === RUN TestSetKEKPassphraseDifferentInstancesNoCollision --- PASS: TestSetKEKPassphraseDifferentInstancesNoCollision (0.00s) === RUN TestNoPassphraseKeepsLegacyHexDecodePath --- PASS: TestNoPassphraseKeepsLegacyHexDecodePath (0.00s) === RUN TestSSES3MultipartChunkViewDecryption === RUN TestSSES3MultipartChunkViewDecryption/Part_1 === RUN TestSSES3MultipartChunkViewDecryption/Part_2 --- PASS: TestSSES3MultipartChunkViewDecryption (0.00s) --- PASS: TestSSES3MultipartChunkViewDecryption/Part_1 (0.00s) --- PASS: TestSSES3MultipartChunkViewDecryption/Part_2 (0.00s) === RUN TestSSES3SinglePartChunkViewDecryption --- PASS: TestSSES3SinglePartChunkViewDecryption (0.00s) === RUN TestSSES3IVOffsetCalculation === RUN TestSSES3IVOffsetCalculation/Part_1 === RUN TestSSES3IVOffsetCalculation/Part_2 === RUN TestSSES3IVOffsetCalculation/Part_3 === RUN TestSSES3IVOffsetCalculation/Part_10 --- PASS: TestSSES3IVOffsetCalculation (0.00s) --- PASS: TestSSES3IVOffsetCalculation/Part_1 (0.00s) --- PASS: TestSSES3IVOffsetCalculation/Part_2 (0.00s) --- PASS: TestSSES3IVOffsetCalculation/Part_3 (0.00s) --- PASS: TestSSES3IVOffsetCalculation/Part_10 (0.00s) === RUN TestSSES3ChunkMetadataDetection === RUN TestSSES3ChunkMetadataDetection/Multipart_chunk_with_metadata === RUN TestSSES3ChunkMetadataDetection/Single-part_chunk_without_metadata === RUN TestSSES3ChunkMetadataDetection/Non-SSE-S3_chunk --- PASS: TestSSES3ChunkMetadataDetection (0.00s) --- PASS: TestSSES3ChunkMetadataDetection/Multipart_chunk_with_metadata (0.00s) --- PASS: TestSSES3ChunkMetadataDetection/Single-part_chunk_without_metadata (0.00s) --- PASS: TestSSES3ChunkMetadataDetection/Non-SSE-S3_chunk (0.00s) === RUN TestSSES3EncryptionConsistency --- PASS: TestSSES3EncryptionConsistency (0.00s) === RUN TestS3IAMIntegration_isSTSIssuer === RUN TestS3IAMIntegration_isSTSIssuer/exact_match_with_configured_issuer === RUN TestS3IAMIntegration_isSTSIssuer/similar_but_not_exact_issuer === RUN TestS3IAMIntegration_isSTSIssuer/substring_of_configured_issuer === RUN TestS3IAMIntegration_isSTSIssuer/contains_configured_issuer_as_substring === RUN TestS3IAMIntegration_isSTSIssuer/case_sensitive_-_different_case === RUN TestS3IAMIntegration_isSTSIssuer/Google_OIDC === RUN TestS3IAMIntegration_isSTSIssuer/Azure_AD === RUN TestS3IAMIntegration_isSTSIssuer/Auth0 === RUN TestS3IAMIntegration_isSTSIssuer/Keycloak === RUN TestS3IAMIntegration_isSTSIssuer/Empty_string --- PASS: TestS3IAMIntegration_isSTSIssuer (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/exact_match_with_configured_issuer (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/similar_but_not_exact_issuer (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/substring_of_configured_issuer (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/contains_configured_issuer_as_substring (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/case_sensitive_-_different_case (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/Google_OIDC (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/Azure_AD (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/Auth0 (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/Keycloak (0.00s) --- PASS: TestS3IAMIntegration_isSTSIssuer/Empty_string (0.00s) === RUN TestS3IAMIntegration_isSTSIssuer_NoSTSService --- PASS: TestS3IAMIntegration_isSTSIssuer_NoSTSService (0.00s) === RUN TestVerifyKeyCommitment_DefaultAcceptsMissing --- PASS: TestVerifyKeyCommitment_DefaultAcceptsMissing (0.00s) === RUN TestVerifyKeyCommitment_StrictRejectsMissing --- PASS: TestVerifyKeyCommitment_StrictRejectsMissing (0.00s) === RUN TestVerifyKeyCommitment_StrictAcceptsValidCommitment --- PASS: TestVerifyKeyCommitment_StrictAcceptsValidCommitment (0.00s) === RUN TestVerifyKeyCommitment_RejectsTamperedKey === RUN TestVerifyKeyCommitment_RejectsTamperedKey/tampered_key === RUN TestVerifyKeyCommitment_RejectsTamperedKey/tampered_IV === RUN TestVerifyKeyCommitment_RejectsTamperedKey/tampered_algorithm === RUN TestVerifyKeyCommitment_RejectsTamperedKey/tampered_commitment --- PASS: TestVerifyKeyCommitment_RejectsTamperedKey (0.00s) --- PASS: TestVerifyKeyCommitment_RejectsTamperedKey/tampered_key (0.00s) --- PASS: TestVerifyKeyCommitment_RejectsTamperedKey/tampered_IV (0.00s) --- PASS: TestVerifyKeyCommitment_RejectsTamperedKey/tampered_algorithm (0.00s) --- PASS: TestVerifyKeyCommitment_RejectsTamperedKey/tampered_commitment (0.00s) === RUN TestSetRequireKeyCommitment --- PASS: TestSetRequireKeyCommitment (0.00s) === RUN TestBucketConfigStubs === RUN TestBucketConfigStubs/AnalyticsList === RUN TestBucketConfigStubs/InventoryList === RUN TestBucketConfigStubs/IntelligentTieringList === RUN TestBucketConfigStubs/MetricsList === RUN TestBucketConfigStubs/AnalyticsGet === RUN TestBucketConfigStubs/InventoryGet === RUN TestBucketConfigStubs/IntelligentTieringGet === RUN TestBucketConfigStubs/MetricsGet --- PASS: TestBucketConfigStubs (0.00s) --- PASS: TestBucketConfigStubs/AnalyticsList (0.00s) --- PASS: TestBucketConfigStubs/InventoryList (0.00s) --- PASS: TestBucketConfigStubs/IntelligentTieringList (0.00s) --- PASS: TestBucketConfigStubs/MetricsList (0.00s) --- PASS: TestBucketConfigStubs/AnalyticsGet (0.00s) --- PASS: TestBucketConfigStubs/InventoryGet (0.00s) --- PASS: TestBucketConfigStubs/IntelligentTieringGet (0.00s) --- PASS: TestBucketConfigStubs/MetricsGet (0.00s) === RUN TestUpdateBucketConfigDoesNotMutateCacheOnPersistFailure W0720 21:30:04.624754 s3api_server.go:489 getFilerAddress: no filer addresses available E0720 21:30:04.624792 s3api_bucket_config.go:491 updateBucketConfig: failed to get bucket entry for cleanup-test-net: getOrCreateConnection : fail to dial : grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) --- PASS: TestUpdateBucketConfigDoesNotMutateCacheOnPersistFailure (0.00s) === RUN TestGetListBucketsArgs === RUN TestGetListBucketsArgs/defaults === RUN TestGetListBucketsArgs/max-buckets === RUN TestGetListBucketsArgs/max-buckets_upper_bound === RUN TestGetListBucketsArgs/max-buckets_zero === RUN TestGetListBucketsArgs/max-buckets_negative === RUN TestGetListBucketsArgs/max-buckets_too_large === RUN TestGetListBucketsArgs/max-buckets_not_a_number === RUN TestGetListBucketsArgs/prefix === RUN TestGetListBucketsArgs/continuation_token === RUN TestGetListBucketsArgs/bad_continuation_token --- PASS: TestGetListBucketsArgs (0.00s) --- PASS: TestGetListBucketsArgs/defaults (0.00s) --- PASS: TestGetListBucketsArgs/max-buckets (0.00s) --- PASS: TestGetListBucketsArgs/max-buckets_upper_bound (0.00s) --- PASS: TestGetListBucketsArgs/max-buckets_zero (0.00s) --- PASS: TestGetListBucketsArgs/max-buckets_negative (0.00s) --- PASS: TestGetListBucketsArgs/max-buckets_too_large (0.00s) --- PASS: TestGetListBucketsArgs/max-buckets_not_a_number (0.00s) --- PASS: TestGetListBucketsArgs/prefix (0.00s) --- PASS: TestGetListBucketsArgs/continuation_token (0.00s) --- PASS: TestGetListBucketsArgs/bad_continuation_token (0.00s) === RUN TestCanListBucketsFromOwnerIndex === RUN TestCanListBucketsFromOwnerIndex/nil_identity_scans === RUN TestCanListBucketsFromOwnerIndex/admin_scans === RUN TestCanListBucketsFromOwnerIndex/bare_List_scans === RUN TestCanListBucketsFromOwnerIndex/wildcard_scans === RUN TestCanListBucketsFromOwnerIndex/no_actions_is_owned-only === RUN TestCanListBucketsFromOwnerIndex/attached_policies_is_owned-only === RUN TestCanListBucketsFromOwnerIndex/named_grants_enumerate === RUN TestCanListBucketsFromOwnerIndex/unsorted_grants_keep_action_order === RUN TestCanListBucketsFromOwnerIndex/session_token_routes_to_policies_when_integrated --- PASS: TestCanListBucketsFromOwnerIndex (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/nil_identity_scans (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/admin_scans (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/bare_List_scans (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/wildcard_scans (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/no_actions_is_owned-only (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/attached_policies_is_owned-only (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/named_grants_enumerate (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/unsorted_grants_keep_action_order (0.00s) --- PASS: TestCanListBucketsFromOwnerIndex/session_token_routes_to_policies_when_integrated (0.00s) === RUN TestContinuationTokenRoundTrip --- PASS: TestContinuationTokenRoundTrip (0.00s) === RUN TestHasExplicitBucketACL === RUN TestHasExplicitBucketACL/none === RUN TestHasExplicitBucketACL/private_is_default === RUN TestHasExplicitBucketACL/canned_public-read === RUN TestHasExplicitBucketACL/canned_case-insensitive_private === RUN TestHasExplicitBucketACL/grant_read === RUN TestHasExplicitBucketACL/grant_full_control --- PASS: TestHasExplicitBucketACL (0.00s) --- PASS: TestHasExplicitBucketACL/none (0.00s) --- PASS: TestHasExplicitBucketACL/private_is_default (0.00s) --- PASS: TestHasExplicitBucketACL/canned_public-read (0.00s) --- PASS: TestHasExplicitBucketACL/canned_case-insensitive_private (0.00s) --- PASS: TestHasExplicitBucketACL/grant_read (0.00s) --- PASS: TestHasExplicitBucketACL/grant_full_control (0.00s) === RUN TestGetBucketPolicyStatusIsPublic === RUN TestGetBucketPolicyStatusIsPublic/public_allow_star === RUN TestGetBucketPolicyStatusIsPublic/deny_is_not_public === RUN TestGetBucketPolicyStatusIsPublic/condition_makes_it_non-public === RUN TestGetBucketPolicyStatusIsPublic/specific_principal_is_not_public --- PASS: TestGetBucketPolicyStatusIsPublic (0.00s) --- PASS: TestGetBucketPolicyStatusIsPublic/public_allow_star (0.00s) --- PASS: TestGetBucketPolicyStatusIsPublic/deny_is_not_public (0.00s) --- PASS: TestGetBucketPolicyStatusIsPublic/condition_makes_it_non-public (0.00s) --- PASS: TestGetBucketPolicyStatusIsPublic/specific_principal_is_not_public (0.00s) === RUN TestPutBucketRequestPaymentBucketOwner --- PASS: TestPutBucketRequestPaymentBucketOwner (0.00s) === RUN TestPutBucketRequestPaymentRequesterRejected --- PASS: TestPutBucketRequestPaymentRequesterRejected (0.00s) === RUN TestPutBucketOwnershipControlsRejectsRuleWithoutObjectOwnership --- PASS: TestPutBucketOwnershipControlsRejectsRuleWithoutObjectOwnership (0.00s) === RUN TestGetBucketAccelerateConfiguration --- PASS: TestGetBucketAccelerateConfiguration (0.00s) === RUN TestGetBucketLogging --- PASS: TestGetBucketLogging (0.00s) === RUN TestGetBucketLifecycleConfigurationHandlerUsesStoredLifecycleConfig --- PASS: TestGetBucketLifecycleConfigurationHandlerUsesStoredLifecycleConfig (0.00s) === RUN TestGetBucketLifecycleConfigurationHandlerDefaultsTransitionMinimumObjectSize --- PASS: TestGetBucketLifecycleConfigurationHandlerDefaultsTransitionMinimumObjectSize (0.00s) === RUN TestPutBucketLifecycleConfigurationHandlerRejectsOversizedBody W0720 21:30:04.626107 s3api_bucket_handlers.go:1089 PutBucketLifecycleConfigurationHandler read body: http: request body too large --- PASS: TestPutBucketLifecycleConfigurationHandlerRejectsOversizedBody (0.00s) === RUN TestPutBucketLifecycleConfigurationHandlerMapsReadErrorsToInvalidRequest W0720 21:30:04.626163 s3api_bucket_handlers.go:1089 PutBucketLifecycleConfigurationHandler read body: read failed --- PASS: TestPutBucketLifecycleConfigurationHandlerMapsReadErrorsToInvalidRequest (0.00s) === RUN TestBucketMetadataStruct --- PASS: TestBucketMetadataStruct (0.00s) === RUN TestBucketMetadataUpdatePattern --- PASS: TestBucketMetadataUpdatePattern (0.00s) === RUN TestBucketMetadataHelperFunctions --- PASS: TestBucketMetadataHelperFunctions (0.00s) === RUN TestEscapeOwnerName --- PASS: TestEscapeOwnerName (0.00s) === RUN TestListMergedBuckets === RUN TestListMergedBuckets/interleave_and_dedup === RUN TestListMergedBuckets/truncation_and_resume === RUN TestListMergedBuckets/no_owned_stream === RUN TestListMergedBuckets/owned_only --- PASS: TestListMergedBuckets (0.00s) --- PASS: TestListMergedBuckets/interleave_and_dedup (0.00s) --- PASS: TestListMergedBuckets/truncation_and_resume (0.00s) --- PASS: TestListMergedBuckets/no_owned_stream (0.00s) --- PASS: TestListMergedBuckets/owned_only (0.00s) === RUN TestBucketEntryOwner --- PASS: TestBucketEntryOwner (0.00s) === RUN TestBuildResourceARN === RUN TestBuildResourceARN/bucket_only === RUN TestBuildResourceARN/bucket_with_slash === RUN TestBuildResourceARN/bucket_and_object === RUN TestBuildResourceARN/bucket_and_object_with_leading_slash --- PASS: TestBuildResourceARN (0.00s) --- PASS: TestBuildResourceARN/bucket_only (0.00s) --- PASS: TestBuildResourceARN/bucket_with_slash (0.00s) --- PASS: TestBuildResourceARN/bucket_and_object (0.00s) --- PASS: TestBuildResourceARN/bucket_and_object_with_leading_slash (0.00s) === RUN TestBuildPrincipalARN === RUN TestBuildPrincipalARN/nil_identity_(anonymous) === RUN TestBuildPrincipalARN/explicit_principal_ARN === RUN TestBuildPrincipalARN/anonymous_user_by_name === RUN TestBuildPrincipalARN/anonymous_user_by_account_ID === RUN TestBuildPrincipalARN/identity_with_account_and_name === RUN TestBuildPrincipalARN/identity_without_account_ID === RUN TestBuildPrincipalARN/identity_without_name --- PASS: TestBuildPrincipalARN (0.00s) --- PASS: TestBuildPrincipalARN/nil_identity_(anonymous) (0.00s) --- PASS: TestBuildPrincipalARN/explicit_principal_ARN (0.00s) --- PASS: TestBuildPrincipalARN/anonymous_user_by_name (0.00s) --- PASS: TestBuildPrincipalARN/anonymous_user_by_account_ID (0.00s) --- PASS: TestBuildPrincipalARN/identity_with_account_and_name (0.00s) --- PASS: TestBuildPrincipalARN/identity_without_account_ID (0.00s) --- PASS: TestBuildPrincipalARN/identity_without_name (0.00s) === RUN TestLimit --- PASS: TestLimit (0.00s) === RUN TestLimitInterceptor === RUN TestLimitInterceptor/nil_interceptor_is_a_no-op === RUN TestLimitInterceptor/interceptor_can_reject === RUN TestLimitInterceptor/interceptor_can_pass_through === RUN TestLimitInterceptor/interceptor_installed_after_registration_takes_effect --- PASS: TestLimitInterceptor (0.00s) --- PASS: TestLimitInterceptor/nil_interceptor_is_a_no-op (0.00s) --- PASS: TestLimitInterceptor/interceptor_can_reject (0.00s) --- PASS: TestLimitInterceptor/interceptor_can_pass_through (0.00s) --- PASS: TestLimitInterceptor/interceptor_installed_after_registration_takes_effect (0.00s) === RUN TestClassifyCopySourceError === RUN TestClassifyCopySourceError/regular_entry === RUN TestClassifyCopySourceError/nil_entry === RUN TestClassifyCopySourceError/directory_entry === RUN TestClassifyCopySourceError/delete_marker_entry === RUN TestClassifyCopySourceError/not_found_sentinel === RUN TestClassifyCopySourceError/wrapped_not_found === RUN TestClassifyCopySourceError/grpc_not_found === RUN TestClassifyCopySourceError/invalid_version_id === RUN TestClassifyCopySourceError/delete_marker_error === RUN TestClassifyCopySourceError/transient_unavailable === RUN TestClassifyCopySourceError/store_error === RUN TestClassifyCopySourceError/sentinel_text_in_message --- PASS: TestClassifyCopySourceError (0.00s) --- PASS: TestClassifyCopySourceError/regular_entry (0.00s) --- PASS: TestClassifyCopySourceError/nil_entry (0.00s) --- PASS: TestClassifyCopySourceError/directory_entry (0.00s) --- PASS: TestClassifyCopySourceError/delete_marker_entry (0.00s) --- PASS: TestClassifyCopySourceError/not_found_sentinel (0.00s) --- PASS: TestClassifyCopySourceError/wrapped_not_found (0.00s) --- PASS: TestClassifyCopySourceError/grpc_not_found (0.00s) --- PASS: TestClassifyCopySourceError/invalid_version_id (0.00s) --- PASS: TestClassifyCopySourceError/delete_marker_error (0.00s) --- PASS: TestClassifyCopySourceError/transient_unavailable (0.00s) --- PASS: TestClassifyCopySourceError/store_error (0.00s) --- PASS: TestClassifyCopySourceError/sentinel_text_in_message (0.00s) === RUN TestClassifyDomainNames === RUN TestClassifyDomainNames/Mixed_path-style_and_virtual-host_with_single_parent === RUN TestClassifyDomainNames/Multiple_subdomains_with_same_parent === RUN TestClassifyDomainNames/Subdomain_without_parent_in_list === RUN TestClassifyDomainNames/Only_top-level_domain === RUN TestClassifyDomainNames/Multiple_independent_domains === RUN TestClassifyDomainNames/Mixed_with_nested_levels === RUN TestClassifyDomainNames/Domain_without_dot === RUN TestClassifyDomainNames/Empty_list === RUN TestClassifyDomainNames/Mixed_localhost_and_domain === RUN TestClassifyDomainNames/Three-level_subdomain_hierarchy === RUN TestClassifyDomainNames/Real-world_example_from_issue_#7356 --- PASS: TestClassifyDomainNames (0.00s) --- PASS: TestClassifyDomainNames/Mixed_path-style_and_virtual-host_with_single_parent (0.00s) --- PASS: TestClassifyDomainNames/Multiple_subdomains_with_same_parent (0.00s) --- PASS: TestClassifyDomainNames/Subdomain_without_parent_in_list (0.00s) --- PASS: TestClassifyDomainNames/Only_top-level_domain (0.00s) --- PASS: TestClassifyDomainNames/Multiple_independent_domains (0.00s) --- PASS: TestClassifyDomainNames/Mixed_with_nested_levels (0.00s) --- PASS: TestClassifyDomainNames/Domain_without_dot (0.00s) --- PASS: TestClassifyDomainNames/Empty_list (0.00s) --- PASS: TestClassifyDomainNames/Mixed_localhost_and_domain (0.00s) --- PASS: TestClassifyDomainNames/Three-level_subdomain_hierarchy (0.00s) --- PASS: TestClassifyDomainNames/Real-world_example_from_issue_#7356 (0.00s) === RUN TestClassifyDomainNamesOrder === RUN TestClassifyDomainNamesOrder/Parent_before_child === RUN TestClassifyDomainNamesOrder/Child_before_parent === RUN TestClassifyDomainNamesOrder/Mixed_order_with_multiple_children --- PASS: TestClassifyDomainNamesOrder (0.00s) --- PASS: TestClassifyDomainNamesOrder/Parent_before_child (0.00s) --- PASS: TestClassifyDomainNamesOrder/Child_before_parent (0.00s) --- PASS: TestClassifyDomainNamesOrder/Mixed_order_with_multiple_children (0.00s) === RUN TestClassifyDomainNamesEdgeCases === RUN TestClassifyDomainNamesEdgeCases/Duplicate_domains === RUN TestClassifyDomainNamesEdgeCases/Very_long_domain_name === RUN TestClassifyDomainNamesEdgeCases/Similar_but_different_domains === RUN TestClassifyDomainNamesEdgeCases/IP_address_as_domain --- PASS: TestClassifyDomainNamesEdgeCases (0.00s) --- PASS: TestClassifyDomainNamesEdgeCases/Duplicate_domains (0.00s) --- PASS: TestClassifyDomainNamesEdgeCases/Very_long_domain_name (0.00s) --- PASS: TestClassifyDomainNamesEdgeCases/Similar_but_different_domains (0.00s) --- PASS: TestClassifyDomainNamesEdgeCases/IP_address_as_domain (0.00s) === RUN TestClassifyDomainNamesUseCases === RUN TestClassifyDomainNamesUseCases/Issue_#7356_-_Prometheus_blackbox_exporter_scenario === RUN TestClassifyDomainNamesUseCases/Multi-environment_setup === RUN TestClassifyDomainNamesUseCases/Mixed_production_setup --- PASS: TestClassifyDomainNamesUseCases (0.00s) --- PASS: TestClassifyDomainNamesUseCases/Issue_#7356_-_Prometheus_blackbox_exporter_scenario (0.00s) --- PASS: TestClassifyDomainNamesUseCases/Multi-environment_setup (0.00s) --- PASS: TestClassifyDomainNamesUseCases/Mixed_production_setup (0.00s) === RUN TestEmbeddedIamCreateUser --- PASS: TestEmbeddedIamCreateUser (0.00s) === RUN TestEmbeddedIamCreateUserDoesNotSaveAllUsers --- PASS: TestEmbeddedIamCreateUserDoesNotSaveAllUsers (0.00s) === RUN TestEmbeddedIamCreateUserSkipPersist --- PASS: TestEmbeddedIamCreateUserSkipPersist (0.00s) === RUN TestEmbeddedIamListUsers --- PASS: TestEmbeddedIamListUsers (0.00s) === RUN TestEmbeddedIamListAccessKeys --- PASS: TestEmbeddedIamListAccessKeys (0.00s) === RUN TestEmbeddedIamGetUser --- PASS: TestEmbeddedIamGetUser (0.00s) === RUN TestEmbeddedIamGetUserImplicitUsername --- PASS: TestEmbeddedIamGetUserImplicitUsername (0.00s) === RUN TestEmbeddedIamCreatePolicy --- PASS: TestEmbeddedIamCreatePolicy (0.00s) === RUN TestEmbeddedIamCreatePolicyVersion --- PASS: TestEmbeddedIamCreatePolicyVersion (0.00s) === RUN TestEmbeddedIamCreatePolicyVersionMissingPolicy E0720 21:30:04.628800 s3api_embedded_iam.go:2767 CreatePolicyVersion: policy does-not-exist not found E0720 21:30:04.628806 s3api_embedded_iam.go:209 IAM Response policy does-not-exist not found --- PASS: TestEmbeddedIamCreatePolicyVersionMissingPolicy (0.00s) === RUN TestEmbeddedIamCreatePolicyVersionRequiresSetAsDefault E0720 21:30:04.628986 s3api_embedded_iam.go:2767 CreatePolicyVersion: SetAsDefault must be true: SeaweedFS stores a single managed policy version E0720 21:30:04.628991 s3api_embedded_iam.go:209 IAM Response SetAsDefault must be true: SeaweedFS stores a single managed policy version --- PASS: TestEmbeddedIamCreatePolicyVersionRequiresSetAsDefault (0.00s) === RUN TestEmbeddedIamPutUserPolicy --- PASS: TestEmbeddedIamPutUserPolicy (0.00s) === RUN TestEmbeddedIamPutUserPolicyError E0720 21:30:04.629274 s3api_embedded_iam.go:2672 PutUserPolicy: the user with name InvalidUser cannot be found E0720 21:30:04.629278 s3api_embedded_iam.go:209 IAM Response the user with name InvalidUser cannot be found --- PASS: TestEmbeddedIamPutUserPolicyError (0.00s) === RUN TestEmbeddedIamGetUserPolicy --- PASS: TestEmbeddedIamGetUserPolicy (0.00s) === RUN TestEmbeddedIamDeleteUserPolicy --- PASS: TestEmbeddedIamDeleteUserPolicy (0.00s) === RUN TestEmbeddedIamDeleteUserPolicyUserNotFound E0720 21:30:04.629531 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. --- PASS: TestEmbeddedIamDeleteUserPolicyUserNotFound (0.00s) === RUN TestEmbeddedIamListUserPolicies E0720 21:30:04.629645 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. --- PASS: TestEmbeddedIamListUserPolicies (0.00s) === RUN TestEmbeddedIamGroupInlinePolicies_CRUD --- PASS: TestEmbeddedIamGroupInlinePolicies_CRUD (0.00s) === RUN TestEmbeddedIamAttachUserPolicy W0720 21:30:04.629824 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "TestManagedPolicy": invalid policy: invalid policy: policy must contain at least one statement W0720 21:30:04.629841 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "TestManagedPolicy": invalid policy: invalid policy: policy must contain at least one statement --- PASS: TestEmbeddedIamAttachUserPolicy (0.00s) === RUN TestEmbeddedIamAttachUserPolicyRefreshesIAM --- PASS: TestEmbeddedIamAttachUserPolicyRefreshesIAM (0.00s) === RUN TestEmbeddedIamAttachUserPolicyNoSuchPolicy E0720 21:30:04.630050 s3api_embedded_iam.go:209 IAM Response policy DoesNotExist not found --- PASS: TestEmbeddedIamAttachUserPolicyNoSuchPolicy (0.00s) === RUN TestEmbeddedIamDetachUserPolicy W0720 21:30:04.630162 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "TestManagedPolicy": invalid policy: invalid policy: policy must contain at least one statement W0720 21:30:04.630170 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "KeepPolicy": invalid policy: invalid policy: policy must contain at least one statement W0720 21:30:04.630189 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "TestManagedPolicy": invalid policy: invalid policy: policy must contain at least one statement W0720 21:30:04.630194 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "KeepPolicy": invalid policy: invalid policy: policy must contain at least one statement --- PASS: TestEmbeddedIamDetachUserPolicy (0.00s) === RUN TestEmbeddedIamDeletePolicyInUse E0720 21:30:04.630290 s3api_embedded_iam.go:2664 DeletePolicy: policy TestPolicy is attached to user TestUser E0720 21:30:04.630294 s3api_embedded_iam.go:209 IAM Response policy TestPolicy is attached to user TestUser --- PASS: TestEmbeddedIamDeletePolicyInUse (0.00s) === RUN TestEmbeddedIamAttachAlreadyAttachedPolicy W0720 21:30:04.630397 auth_credentials.go:2819 IAM policy cache rebuild: skipping invalid policy "TestManagedPolicy": invalid policy: invalid policy: policy must contain at least one statement --- PASS: TestEmbeddedIamAttachAlreadyAttachedPolicy (0.00s) === RUN TestEmbeddedIamDetachNotAttachedPolicy E0720 21:30:04.630484 s3api_embedded_iam.go:209 IAM Response policy MissingPolicy not attached to user TestUser --- PASS: TestEmbeddedIamDetachNotAttachedPolicy (0.00s) === RUN TestEmbeddedIamAttachPolicyLimitExceeded E0720 21:30:04.630677 s3api_embedded_iam.go:209 IAM Response cannot attach more than 10 managed policies to user TestUser --- PASS: TestEmbeddedIamAttachPolicyLimitExceeded (0.00s) === RUN TestEmbeddedIamListAttachedUserPolicies --- PASS: TestEmbeddedIamListAttachedUserPolicies (0.00s) === RUN TestEmbeddedIamUpdateUser --- PASS: TestEmbeddedIamUpdateUser (0.00s) === RUN TestEmbeddedIamUpdateUserPreservesInlinePolicies --- PASS: TestEmbeddedIamUpdateUserPreservesInlinePolicies (0.00s) === RUN TestEmbeddedIamDeleteUser --- PASS: TestEmbeddedIamDeleteUser (0.00s) === RUN TestEmbeddedIamCreateAccessKey --- PASS: TestEmbeddedIamCreateAccessKey (0.00s) === RUN TestEmbeddedIamCreateAccessKeyRejectsMissingUser E0720 21:30:04.631335 s3api_embedded_iam.go:2647 CreateAccessKey: the user with name GhostUser cannot be found. E0720 21:30:04.631340 s3api_embedded_iam.go:209 IAM Response the user with name GhostUser cannot be found. --- PASS: TestEmbeddedIamCreateAccessKeyRejectsMissingUser (0.00s) === RUN TestEmbeddedIamCreateAccessKeyWithCallerSuppliedKeys --- PASS: TestEmbeddedIamCreateAccessKeyWithCallerSuppliedKeys (0.00s) === RUN TestEmbeddedIamCreateAccessKeyRejectsWeakKeys E0720 21:30:04.631484 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631487 s3api_embedded_iam.go:209 IAM Response AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631507 s3api_embedded_iam.go:2647 CreateAccessKey: SecretAccessKey must be between 8 and 128 characters E0720 21:30:04.631510 s3api_embedded_iam.go:209 IAM Response SecretAccessKey must be between 8 and 128 characters E0720 21:30:04.631529 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631532 s3api_embedded_iam.go:209 IAM Response AccessKeyId must be 4 to 128 alphanumeric characters --- PASS: TestEmbeddedIamCreateAccessKeyRejectsWeakKeys (0.00s) === RUN TestEmbeddedIamCreateAccessKeyRejectsCollision E0720 21:30:04.631588 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId is already in use E0720 21:30:04.631591 s3api_embedded_iam.go:209 IAM Response AccessKeyId is already in use --- PASS: TestEmbeddedIamCreateAccessKeyRejectsCollision (0.00s) === RUN TestEmbeddedIamCreateAccessKeyRejectsPartialSupply E0720 21:30:04.631634 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId and SecretAccessKey must be supplied together E0720 21:30:04.631637 s3api_embedded_iam.go:209 IAM Response AccessKeyId and SecretAccessKey must be supplied together E0720 21:30:04.631658 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId and SecretAccessKey must be supplied together E0720 21:30:04.631661 s3api_embedded_iam.go:209 IAM Response AccessKeyId and SecretAccessKey must be supplied together --- PASS: TestEmbeddedIamCreateAccessKeyRejectsPartialSupply (0.00s) === RUN TestEmbeddedIamCreateAccessKeyBoundary E0720 21:30:04.631734 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631737 s3api_embedded_iam.go:209 IAM Response AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631817 s3api_embedded_iam.go:2647 CreateAccessKey: AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631820 s3api_embedded_iam.go:209 IAM Response AccessKeyId must be 4 to 128 alphanumeric characters E0720 21:30:04.631845 s3api_embedded_iam.go:2647 CreateAccessKey: SecretAccessKey must be between 8 and 128 characters E0720 21:30:04.631847 s3api_embedded_iam.go:209 IAM Response SecretAccessKey must be between 8 and 128 characters E0720 21:30:04.631921 s3api_embedded_iam.go:2647 CreateAccessKey: SecretAccessKey must be between 8 and 128 characters E0720 21:30:04.631924 s3api_embedded_iam.go:209 IAM Response SecretAccessKey must be between 8 and 128 characters --- PASS: TestEmbeddedIamCreateAccessKeyBoundary (0.00s) === RUN TestEmbeddedIamDeleteAccessKey --- PASS: TestEmbeddedIamDeleteAccessKey (0.00s) === RUN TestEmbeddedIamHandleImplicitUsername --- PASS: TestEmbeddedIamHandleImplicitUsername (0.00s) === RUN TestEmbeddedIamFullWorkflow === RUN TestEmbeddedIamFullWorkflow/CreateUser === RUN TestEmbeddedIamFullWorkflow/CreateAccessKey === RUN TestEmbeddedIamFullWorkflow/PutUserPolicy === RUN TestEmbeddedIamFullWorkflow/ListUsers === RUN TestEmbeddedIamFullWorkflow/DeleteUser --- PASS: TestEmbeddedIamFullWorkflow (0.00s) --- PASS: TestEmbeddedIamFullWorkflow/CreateUser (0.00s) --- PASS: TestEmbeddedIamFullWorkflow/CreateAccessKey (0.00s) --- PASS: TestEmbeddedIamFullWorkflow/PutUserPolicy (0.00s) --- PASS: TestEmbeddedIamFullWorkflow/ListUsers (0.00s) --- PASS: TestEmbeddedIamFullWorkflow/DeleteUser (0.00s) === RUN TestIamStringSlicesEqual === RUN TestIamStringSlicesEqual/both_empty === RUN TestIamStringSlicesEqual/both_nil === RUN TestIamStringSlicesEqual/same_elements_same_order === RUN TestIamStringSlicesEqual/same_elements_different_order === RUN TestIamStringSlicesEqual/different_lengths === RUN TestIamStringSlicesEqual/different_elements === RUN TestIamStringSlicesEqual/one_empty_one_not === RUN TestIamStringSlicesEqual/duplicates_same --- PASS: TestIamStringSlicesEqual (0.00s) --- PASS: TestIamStringSlicesEqual/both_empty (0.00s) --- PASS: TestIamStringSlicesEqual/both_nil (0.00s) --- PASS: TestIamStringSlicesEqual/same_elements_same_order (0.00s) --- PASS: TestIamStringSlicesEqual/same_elements_different_order (0.00s) --- PASS: TestIamStringSlicesEqual/different_lengths (0.00s) --- PASS: TestIamStringSlicesEqual/different_elements (0.00s) --- PASS: TestIamStringSlicesEqual/one_empty_one_not (0.00s) --- PASS: TestIamStringSlicesEqual/duplicates_same (0.00s) === RUN TestIamHash --- PASS: TestIamHash (0.00s) === RUN TestIamStringWithCharset --- PASS: TestIamStringWithCharset (0.00s) === RUN TestIamMapToStatementAction === RUN TestIamMapToStatementAction/* === RUN TestIamMapToStatementAction/Get* === RUN TestIamMapToStatementAction/Put* === RUN TestIamMapToStatementAction/List* === RUN TestIamMapToStatementAction/Tagging* === RUN TestIamMapToStatementAction/DeleteBucket* === RUN TestIamMapToStatementAction/PutBucketAcl === RUN TestIamMapToStatementAction/GetBucketAcl === RUN TestIamMapToStatementAction/InvalidAction --- PASS: TestIamMapToStatementAction (0.00s) --- PASS: TestIamMapToStatementAction/* (0.00s) --- PASS: TestIamMapToStatementAction/Get* (0.00s) --- PASS: TestIamMapToStatementAction/Put* (0.00s) --- PASS: TestIamMapToStatementAction/List* (0.00s) --- PASS: TestIamMapToStatementAction/Tagging* (0.00s) --- PASS: TestIamMapToStatementAction/DeleteBucket* (0.00s) --- PASS: TestIamMapToStatementAction/PutBucketAcl (0.00s) --- PASS: TestIamMapToStatementAction/GetBucketAcl (0.00s) --- PASS: TestIamMapToStatementAction/InvalidAction (0.00s) === RUN TestIamMapToIdentitiesAction === RUN TestIamMapToIdentitiesAction/Admin === RUN TestIamMapToIdentitiesAction/Read === RUN TestIamMapToIdentitiesAction/Write === RUN TestIamMapToIdentitiesAction/List === RUN TestIamMapToIdentitiesAction/Tagging === RUN TestIamMapToIdentitiesAction/Unknown --- PASS: TestIamMapToIdentitiesAction (0.00s) --- PASS: TestIamMapToIdentitiesAction/Admin (0.00s) --- PASS: TestIamMapToIdentitiesAction/Read (0.00s) --- PASS: TestIamMapToIdentitiesAction/Write (0.00s) --- PASS: TestIamMapToIdentitiesAction/List (0.00s) --- PASS: TestIamMapToIdentitiesAction/Tagging (0.00s) --- PASS: TestIamMapToIdentitiesAction/Unknown (0.00s) === RUN TestEmbeddedIamGetUserNotFound E0720 21:30:04.632851 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. --- PASS: TestEmbeddedIamGetUserNotFound (0.00s) === RUN TestEmbeddedIamDeleteUserNotFound E0720 21:30:04.632912 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. --- PASS: TestEmbeddedIamDeleteUserNotFound (0.00s) === RUN TestEmbeddedIamUpdateUserNotFound E0720 21:30:04.632977 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. --- PASS: TestEmbeddedIamUpdateUserNotFound (0.00s) === RUN TestEmbeddedIamCreateAccessKeyForExistingUser --- PASS: TestEmbeddedIamCreateAccessKeyForExistingUser (0.00s) === RUN TestEmbeddedIamPutGetUserPolicyRoundTrip --- PASS: TestEmbeddedIamPutGetUserPolicyRoundTrip (0.00s) === RUN TestEmbeddedIamGetUserPolicyFallback --- PASS: TestEmbeddedIamGetUserPolicyFallback (0.00s) === RUN TestEmbeddedIamGetUserPolicyUserNotFound E0720 21:30:04.633213 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. --- PASS: TestEmbeddedIamGetUserPolicyUserNotFound (0.00s) === RUN TestEmbeddedIamCreatePolicyMalformed E0720 21:30:04.633273 s3api_embedded_iam.go:2656 CreatePolicy: invalid character 'i' looking for beginning of value E0720 21:30:04.633277 s3api_embedded_iam.go:209 IAM Response invalid character 'i' looking for beginning of value --- PASS: TestEmbeddedIamCreatePolicyMalformed (0.00s) === RUN TestEmbeddedIamListAccessKeysForUser --- PASS: TestEmbeddedIamListAccessKeysForUser (0.00s) === RUN TestEmbeddedIamNotImplementedAction E0720 21:30:04.633423 s3api_embedded_iam.go:209 IAM Response A header you provided implies functionality that is not implemented --- PASS: TestEmbeddedIamNotImplementedAction (0.00s) === RUN TestGetPolicyDocument --- PASS: TestGetPolicyDocument (0.00s) === RUN TestEmbeddedIamGetActionsFromPolicy --- PASS: TestEmbeddedIamGetActionsFromPolicy (0.00s) === RUN TestEmbeddedIamGetActionsPrefixScopedResource --- PASS: TestEmbeddedIamGetActionsPrefixScopedResource (0.00s) === RUN TestEmbeddedIamPutUserPolicyAllResourceWildcard --- PASS: TestEmbeddedIamPutUserPolicyAllResourceWildcard (0.00s) === RUN TestEmbeddedIamSetUserStatus === RUN TestEmbeddedIamSetUserStatus/DisableUser === RUN TestEmbeddedIamSetUserStatus/EnableUser --- PASS: TestEmbeddedIamSetUserStatus (0.00s) --- PASS: TestEmbeddedIamSetUserStatus/DisableUser (0.00s) --- PASS: TestEmbeddedIamSetUserStatus/EnableUser (0.00s) === RUN TestEmbeddedIamSetUserStatusErrors === RUN TestEmbeddedIamSetUserStatusErrors/UserNotFound E0720 21:30:04.633701 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. === RUN TestEmbeddedIamSetUserStatusErrors/InvalidStatus E0720 21:30:04.633732 s3api_embedded_iam.go:209 IAM Response Status must be 'Active' or 'Inactive' === RUN TestEmbeddedIamSetUserStatusErrors/MissingUserName E0720 21:30:04.633764 s3api_embedded_iam.go:209 IAM Response UserName is required === RUN TestEmbeddedIamSetUserStatusErrors/MissingStatus E0720 21:30:04.633797 s3api_embedded_iam.go:209 IAM Response Status parameter is required --- PASS: TestEmbeddedIamSetUserStatusErrors (0.00s) --- PASS: TestEmbeddedIamSetUserStatusErrors/UserNotFound (0.00s) --- PASS: TestEmbeddedIamSetUserStatusErrors/InvalidStatus (0.00s) --- PASS: TestEmbeddedIamSetUserStatusErrors/MissingUserName (0.00s) --- PASS: TestEmbeddedIamSetUserStatusErrors/MissingStatus (0.00s) === RUN TestEmbeddedIamUpdateAccessKey === RUN TestEmbeddedIamUpdateAccessKey/DeactivateAccessKey === RUN TestEmbeddedIamUpdateAccessKey/ActivateAccessKey --- PASS: TestEmbeddedIamUpdateAccessKey (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKey/DeactivateAccessKey (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKey/ActivateAccessKey (0.00s) === RUN TestEmbeddedIamUpdateAccessKeyErrors === RUN TestEmbeddedIamUpdateAccessKeyErrors/AccessKeyNotFound E0720 21:30:04.634061 s3api_embedded_iam.go:209 IAM Response the access key with id NONEXISTENT123 for user TestUser cannot be found === RUN TestEmbeddedIamUpdateAccessKeyErrors/InvalidStatus E0720 21:30:04.634098 s3api_embedded_iam.go:209 IAM Response Status must be 'Active' or 'Inactive' === RUN TestEmbeddedIamUpdateAccessKeyErrors/MissingUserName E0720 21:30:04.634133 s3api_embedded_iam.go:209 IAM Response UserName is required === RUN TestEmbeddedIamUpdateAccessKeyErrors/MissingAccessKeyId E0720 21:30:04.634173 s3api_embedded_iam.go:209 IAM Response AccessKeyId is required === RUN TestEmbeddedIamUpdateAccessKeyErrors/UserNotFound E0720 21:30:04.634223 s3api_embedded_iam.go:209 IAM Response the user with name NonExistentUser cannot be found. === RUN TestEmbeddedIamUpdateAccessKeyErrors/MissingStatus E0720 21:30:04.634259 s3api_embedded_iam.go:209 IAM Response Status parameter is required --- PASS: TestEmbeddedIamUpdateAccessKeyErrors (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKeyErrors/AccessKeyNotFound (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKeyErrors/InvalidStatus (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKeyErrors/MissingUserName (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKeyErrors/MissingAccessKeyId (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKeyErrors/UserNotFound (0.00s) --- PASS: TestEmbeddedIamUpdateAccessKeyErrors/MissingStatus (0.00s) === RUN TestEmbeddedIamListAccessKeysShowsStatus --- PASS: TestEmbeddedIamListAccessKeysShowsStatus (0.00s) === RUN TestDisabledUserLookupFails --- PASS: TestDisabledUserLookupFails (0.00s) === RUN TestInactiveAccessKeyLookupFails --- PASS: TestInactiveAccessKeyLookupFails (0.00s) === RUN TestAuthIamAuthenticatesBeforeParseForm --- PASS: TestAuthIamAuthenticatesBeforeParseForm (0.00s) === RUN TestOldCodeOrderWouldFail s3api_embedded_iam_test.go:2776: This demonstrates the bug: ParseForm before auth causes SignatureDoesNotMatch --- PASS: TestOldCodeOrderWouldFail (0.00s) === RUN TestEmbeddedIamExecuteAction --- PASS: TestEmbeddedIamExecuteAction (0.00s) === RUN TestEmbeddedIamReadOnly E0720 21:30:04.635023 s3api_embedded_iam.go:209 IAM Response IAM write operations are disabled on this server --- PASS: TestEmbeddedIamReadOnly (0.00s) === RUN TestEmbeddedIamTagUser --- PASS: TestEmbeddedIamTagUser (0.00s) === RUN TestEmbeddedIamListUserTags --- PASS: TestEmbeddedIamListUserTags (0.00s) === RUN TestEmbeddedIamUntagUser --- PASS: TestEmbeddedIamUntagUser (0.00s) === RUN TestEmbeddedIamTagUserInvalidKeyLength E0720 21:30:04.635723 s3api_embedded_iam.go:209 IAM Response tag key "kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk" exceeds 128 characters --- PASS: TestEmbeddedIamTagUserInvalidKeyLength (0.00s) === RUN TestEmbeddedIamTagUserReplacesDuplicate --- PASS: TestEmbeddedIamTagUserReplacesDuplicate (0.00s) === RUN TestEmbeddedIamTagUserLimitExceeded E0720 21:30:04.636001 s3api_embedded_iam.go:209 IAM Response cannot exceed 50 tags per user --- PASS: TestEmbeddedIamTagUserLimitExceeded (0.00s) === RUN TestEmbeddedIamUntagUserNoOpForMissingKey --- PASS: TestEmbeddedIamUntagUserNoOpForMissingKey (0.00s) === RUN TestEmbeddedIamTagUserRejectsDuplicateKeyInRequest E0720 21:30:04.636133 s3api_embedded_iam.go:209 IAM Response duplicate tag key "env" --- PASS: TestEmbeddedIamTagUserRejectsDuplicateKeyInRequest (0.00s) === RUN TestEmbeddedIamUntagUserRejectsInvalidKeys === RUN TestEmbeddedIamUntagUserRejectsInvalidKeys/no_TagKeys_entries E0720 21:30:04.636238 s3api_embedded_iam.go:209 IAM Response at least one TagKeys entry is required === RUN TestEmbeddedIamUntagUserRejectsInvalidKeys/empty_TagKeys_entry E0720 21:30:04.636282 s3api_embedded_iam.go:209 IAM Response tag key cannot be empty === RUN TestEmbeddedIamUntagUserRejectsInvalidKeys/TagKeys_entry_over_MaxUserTagKeyLength E0720 21:30:04.636328 s3api_embedded_iam.go:209 IAM Response tag key "kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk" exceeds 128 characters --- PASS: TestEmbeddedIamUntagUserRejectsInvalidKeys (0.00s) --- PASS: TestEmbeddedIamUntagUserRejectsInvalidKeys/no_TagKeys_entries (0.00s) --- PASS: TestEmbeddedIamUntagUserRejectsInvalidKeys/empty_TagKeys_entry (0.00s) --- PASS: TestEmbeddedIamUntagUserRejectsInvalidKeys/TagKeys_entry_over_MaxUserTagKeyLength (0.00s) === RUN TestEmbeddedIamTagUserNotFound E0720 21:30:04.636406 s3api_embedded_iam.go:209 IAM Response the user with name ghost cannot be found. --- PASS: TestEmbeddedIamTagUserNotFound (0.00s) === RUN TestCreateDestinationChunkPreservesEncryption === RUN TestCreateDestinationChunkPreservesEncryption/Encrypted_and_compressed_chunk === RUN TestCreateDestinationChunkPreservesEncryption/Only_encrypted_chunk === RUN TestCreateDestinationChunkPreservesEncryption/Only_compressed_chunk === RUN TestCreateDestinationChunkPreservesEncryption/Unencrypted_and_uncompressed_chunk --- PASS: TestCreateDestinationChunkPreservesEncryption (0.00s) --- PASS: TestCreateDestinationChunkPreservesEncryption/Encrypted_and_compressed_chunk (0.00s) --- PASS: TestCreateDestinationChunkPreservesEncryption/Only_encrypted_chunk (0.00s) --- PASS: TestCreateDestinationChunkPreservesEncryption/Only_compressed_chunk (0.00s) --- PASS: TestCreateDestinationChunkPreservesEncryption/Unencrypted_and_uncompressed_chunk (0.00s) === RUN TestEncryptedVolumeCopyScenario === RUN TestEncryptedVolumeCopyScenario/Scenario:_Copy_file_on_encrypted_volume_with_multiple_chunks s3api_encrypted_volume_copy_test.go:173: ✓ All chunk metadata properly preserved for encrypted volume copy scenario --- PASS: TestEncryptedVolumeCopyScenario (0.00s) --- PASS: TestEncryptedVolumeCopyScenario/Scenario:_Copy_file_on_encrypted_volume_with_multiple_chunks (0.00s) === RUN TestCheckGovernanceBypassPermissionResourceGeneration === RUN TestCheckGovernanceBypassPermissionResourceGeneration/simple_object === RUN TestCheckGovernanceBypassPermissionResourceGeneration/object_with_leading_slash === RUN TestCheckGovernanceBypassPermissionResourceGeneration/nested_object === RUN TestCheckGovernanceBypassPermissionResourceGeneration/empty_object === RUN TestCheckGovernanceBypassPermissionResourceGeneration/root_object --- PASS: TestCheckGovernanceBypassPermissionResourceGeneration (0.00s) --- PASS: TestCheckGovernanceBypassPermissionResourceGeneration/simple_object (0.00s) --- PASS: TestCheckGovernanceBypassPermissionResourceGeneration/object_with_leading_slash (0.00s) --- PASS: TestCheckGovernanceBypassPermissionResourceGeneration/nested_object (0.00s) --- PASS: TestCheckGovernanceBypassPermissionResourceGeneration/empty_object (0.00s) --- PASS: TestCheckGovernanceBypassPermissionResourceGeneration/root_object (0.00s) === RUN TestCheckGovernanceBypassPermissionActionGeneration === RUN TestCheckGovernanceBypassPermissionActionGeneration/bypass_action_generation === RUN TestCheckGovernanceBypassPermissionActionGeneration/leading_slash_handling --- PASS: TestCheckGovernanceBypassPermissionActionGeneration (0.00s) --- PASS: TestCheckGovernanceBypassPermissionActionGeneration/bypass_action_generation (0.00s) --- PASS: TestCheckGovernanceBypassPermissionActionGeneration/leading_slash_handling (0.00s) === RUN TestCheckGovernanceBypassPermissionErrorHandling === RUN TestCheckGovernanceBypassPermissionErrorHandling/empty_bucket s3api_governance_permissions_test.go:169: Generated resource path for Empty bucket should be handled gracefully: /test-object.txt === RUN TestCheckGovernanceBypassPermissionErrorHandling/special_characters s3api_governance_permissions_test.go:169: Generated resource path for Objects with special characters should be handled: test-bucket/test object with spaces.txt === RUN TestCheckGovernanceBypassPermissionErrorHandling/unicode_characters s3api_governance_permissions_test.go:169: Generated resource path for Objects with unicode characters should be handled: test-bucket/测试文件.txt --- PASS: TestCheckGovernanceBypassPermissionErrorHandling (0.00s) --- PASS: TestCheckGovernanceBypassPermissionErrorHandling/empty_bucket (0.00s) --- PASS: TestCheckGovernanceBypassPermissionErrorHandling/special_characters (0.00s) --- PASS: TestCheckGovernanceBypassPermissionErrorHandling/unicode_characters (0.00s) === RUN TestCheckGovernanceBypassPermissionIntegrationBehavior s3api_governance_permissions_test.go:177: Documentation test - describes expected behavior with full IAM integration --- SKIP: TestCheckGovernanceBypassPermissionIntegrationBehavior (0.00s) === RUN TestGovernanceBypassWithIAMPermission s3api_governance_permissions_test.go:206: Integration test requires full IAM setup - demonstrates expected behavior --- SKIP: TestGovernanceBypassWithIAMPermission (0.00s) === RUN TestGovernancePermissionIntegration s3api_governance_permissions_test.go:224: Integration test requires full IAM setup - demonstrates expected behavior --- SKIP: TestGovernancePermissionIntegration (0.00s) === RUN TestGovernanceBypassHeader === RUN TestGovernanceBypassHeader/bypass_header_true === RUN TestGovernanceBypassHeader/bypass_header_false === RUN TestGovernanceBypassHeader/bypass_header_empty === RUN TestGovernanceBypassHeader/bypass_header_invalid --- PASS: TestGovernanceBypassHeader (0.00s) --- PASS: TestGovernanceBypassHeader/bypass_header_true (0.00s) --- PASS: TestGovernanceBypassHeader/bypass_header_false (0.00s) --- PASS: TestGovernanceBypassHeader/bypass_header_empty (0.00s) --- PASS: TestGovernanceBypassHeader/bypass_header_invalid (0.00s) === RUN TestGovernanceRetentionModeChecking === RUN TestGovernanceRetentionModeChecking/compliance_mode_cannot_bypass === RUN TestGovernanceRetentionModeChecking/governance_mode_without_bypass === RUN TestGovernanceRetentionModeChecking/governance_mode_with_bypass_no_permission === RUN TestGovernanceRetentionModeChecking/governance_mode_with_bypass_and_permission --- PASS: TestGovernanceRetentionModeChecking (0.00s) --- PASS: TestGovernanceRetentionModeChecking/compliance_mode_cannot_bypass (0.00s) --- PASS: TestGovernanceRetentionModeChecking/governance_mode_without_bypass (0.00s) --- PASS: TestGovernanceRetentionModeChecking/governance_mode_with_bypass_no_permission (0.00s) --- PASS: TestGovernanceRetentionModeChecking/governance_mode_with_bypass_and_permission (0.00s) === RUN TestGovernancePermissionActionGeneration === RUN TestGovernancePermissionActionGeneration/bucket_and_object_action === RUN TestGovernancePermissionActionGeneration/bucket_only_action === RUN TestGovernancePermissionActionGeneration/nested_object_action --- PASS: TestGovernancePermissionActionGeneration (0.00s) --- PASS: TestGovernancePermissionActionGeneration/bucket_and_object_action (0.00s) --- PASS: TestGovernancePermissionActionGeneration/bucket_only_action (0.00s) --- PASS: TestGovernancePermissionActionGeneration/nested_object_action (0.00s) === RUN TestGovernancePermissionEndToEnd s3api_governance_permissions_test.go:406: End-to-end testing requires full S3 API server setup - demonstrates expected behavior --- SKIP: TestGovernancePermissionEndToEnd (0.00s) === RUN TestGovernancePermissionHTTPFlow === RUN TestGovernancePermissionHTTPFlow/bypass_header_true === RUN TestGovernancePermissionHTTPFlow/bypass_header_false === RUN TestGovernancePermissionHTTPFlow/bypass_header_missing --- PASS: TestGovernancePermissionHTTPFlow (0.00s) --- PASS: TestGovernancePermissionHTTPFlow/bypass_header_true (0.00s) --- PASS: TestGovernancePermissionHTTPFlow/bypass_header_false (0.00s) --- PASS: TestGovernancePermissionHTTPFlow/bypass_header_missing (0.00s) === RUN TestGovernancePermissionMethodCalls === RUN TestGovernancePermissionMethodCalls/delete_object_handler_pattern s3api_governance_permissions_test.go:482: Extracted bucket: , object: === RUN TestGovernancePermissionMethodCalls/put_object_handler_pattern s3api_governance_permissions_test.go:503: Extracted bucket: , object: --- PASS: TestGovernancePermissionMethodCalls (0.00s) --- PASS: TestGovernancePermissionMethodCalls/delete_object_handler_pattern (0.00s) --- PASS: TestGovernancePermissionMethodCalls/put_object_handler_pattern (0.00s) === RUN TestGovernanceBypassNotPermittedError === RUN TestGovernanceBypassNotPermittedError/governance_bypass_without_permission === RUN TestGovernanceBypassNotPermittedError/governance_bypass_with_permission === RUN TestGovernanceBypassNotPermittedError/governance_no_bypass --- PASS: TestGovernanceBypassNotPermittedError (0.00s) --- PASS: TestGovernanceBypassNotPermittedError/governance_bypass_without_permission (0.00s) --- PASS: TestGovernanceBypassNotPermittedError/governance_bypass_with_permission (0.00s) --- PASS: TestGovernanceBypassNotPermittedError/governance_no_bypass (0.00s) === RUN TestListOpenIDConnectProviders --- PASS: TestListOpenIDConnectProviders (0.00s) === RUN TestGetOpenIDConnectProvider --- PASS: TestGetOpenIDConnectProvider (0.00s) === RUN TestGetOpenIDConnectProviderMissing --- PASS: TestGetOpenIDConnectProviderMissing (0.00s) === RUN TestGetOpenIDConnectProviderRequiresArn --- PASS: TestGetOpenIDConnectProviderRequiresArn (0.00s) === RUN TestReadOnlyAllowsOIDCList --- PASS: TestReadOnlyAllowsOIDCList (0.00s) === RUN TestReadOnlyDeniesOIDCMutations === RUN TestReadOnlyDeniesOIDCMutations/CreateOpenIDConnectProvider === RUN TestReadOnlyDeniesOIDCMutations/DeleteOpenIDConnectProvider === RUN TestReadOnlyDeniesOIDCMutations/AddClientIDToOpenIDConnectProvider === RUN TestReadOnlyDeniesOIDCMutations/RemoveClientIDFromOpenIDConnectProvider === RUN TestReadOnlyDeniesOIDCMutations/UpdateOpenIDConnectProviderThumbprint === RUN TestReadOnlyDeniesOIDCMutations/TagOpenIDConnectProvider === RUN TestReadOnlyDeniesOIDCMutations/UntagOpenIDConnectProvider --- PASS: TestReadOnlyDeniesOIDCMutations (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/CreateOpenIDConnectProvider (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/DeleteOpenIDConnectProvider (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/AddClientIDToOpenIDConnectProvider (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/RemoveClientIDFromOpenIDConnectProvider (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/UpdateOpenIDConnectProviderThumbprint (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/TagOpenIDConnectProvider (0.00s) --- PASS: TestReadOnlyDeniesOIDCMutations/UntagOpenIDConnectProvider (0.00s) === RUN TestCreateOpenIDConnectProvider I0720 21:30:04.639332 oidc_provider_audit.go:66 oidc-audit: {"type":"Create","arn":"arn:aws:iam::111122223333:oidc-provider/auth.example.com","url":"https://auth.example.com","occurredAt":"2026-07-20T21:30:04.63927651Z"} --- PASS: TestCreateOpenIDConnectProvider (0.00s) === RUN TestCreateOpenIDConnectProviderRejectsBadInput === RUN TestCreateOpenIDConnectProviderRejectsBadInput/missing_Url === RUN TestCreateOpenIDConnectProviderRejectsBadInput/missing_ClientIDList === RUN TestCreateOpenIDConnectProviderRejectsBadInput/invalid_thumbprint --- PASS: TestCreateOpenIDConnectProviderRejectsBadInput (0.00s) --- PASS: TestCreateOpenIDConnectProviderRejectsBadInput/missing_Url (0.00s) --- PASS: TestCreateOpenIDConnectProviderRejectsBadInput/missing_ClientIDList (0.00s) --- PASS: TestCreateOpenIDConnectProviderRejectsBadInput/invalid_thumbprint (0.00s) === RUN TestCreateOpenIDConnectProviderConflict --- PASS: TestCreateOpenIDConnectProviderConflict (0.00s) === RUN TestDeleteOpenIDConnectProvider I0720 21:30:04.639474 oidc_provider_audit.go:66 oidc-audit: {"type":"Delete","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","occurredAt":"2026-07-20T21:30:04.639470189Z"} I0720 21:30:04.639482 oidc_provider_audit.go:66 oidc-audit: {"type":"Delete","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","occurredAt":"2026-07-20T21:30:04.639479818Z"} --- PASS: TestDeleteOpenIDConnectProvider (0.00s) === RUN TestAddRemoveClientID I0720 21:30:04.639515 oidc_provider_audit.go:66 oidc-audit: {"type":"AddClientID","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","url":"https://accounts.google.com","detail":{"clientId":"extra-client"},"occurredAt":"2026-07-20T21:30:04.639510075Z"} I0720 21:30:04.639527 oidc_provider_audit.go:66 oidc-audit: {"type":"RemoveClientID","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","url":"https://accounts.google.com","detail":{"clientId":"extra-client"},"occurredAt":"2026-07-20T21:30:04.639524353Z"} --- PASS: TestAddRemoveClientID (0.00s) === RUN TestUpdateThumbprintAndTags I0720 21:30:04.639567 oidc_provider_audit.go:66 oidc-audit: {"type":"UpdateThumbprints","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","url":"https://accounts.google.com","detail":{"count":"2"},"occurredAt":"2026-07-20T21:30:04.639563988Z"} I0720 21:30:04.639575 oidc_provider_audit.go:66 oidc-audit: {"type":"Tag","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","url":"https://accounts.google.com","detail":{"count":"1"},"occurredAt":"2026-07-20T21:30:04.639572995Z"} I0720 21:30:04.639586 oidc_provider_audit.go:66 oidc-audit: {"type":"Untag","arn":"arn:aws:iam::111122223333:oidc-provider/accounts.google.com","url":"https://accounts.google.com","detail":{"count":"1"},"occurredAt":"2026-07-20T21:30:04.639583766Z"} --- PASS: TestUpdateThumbprintAndTags (0.00s) === RUN TestImplicitDirectoryBehaviorLogic === RUN TestImplicitDirectoryBehaviorLogic/Implicit_directory:_0-byte_file_with_children,_no_trailing_slash s3api_implicit_directory_test.go:143: ✓ Implicit directory: 0-byte file with children, no trailing slash: correctly returns 404 === RUN TestImplicitDirectoryBehaviorLogic/Implicit_directory:_actual_directory_with_children,_no_trailing_slash s3api_implicit_directory_test.go:143: ✓ Implicit directory: actual directory with children, no trailing slash: correctly returns 404 === RUN TestImplicitDirectoryBehaviorLogic/Explicit_directory_request:_trailing_slash s3api_implicit_directory_test.go:143: ✓ Explicit directory request: trailing slash: correctly returns 200 === RUN TestImplicitDirectoryBehaviorLogic/Empty_file:_0-byte_file_without_children s3api_implicit_directory_test.go:143: ✓ Empty file: 0-byte file without children: correctly returns 200 === RUN TestImplicitDirectoryBehaviorLogic/Empty_directory:_0-byte_directory_without_children s3api_implicit_directory_test.go:143: ✓ Empty directory: 0-byte directory without children: correctly returns 404 === RUN TestImplicitDirectoryBehaviorLogic/Regular_file:_non-zero_size s3api_implicit_directory_test.go:143: ✓ Regular file: non-zero size: correctly returns 200 === RUN TestImplicitDirectoryBehaviorLogic/Versioned_bucket:_implicit_directory_should_return_200 s3api_implicit_directory_test.go:143: ✓ Versioned bucket: implicit directory should return 200: correctly returns 200 === RUN TestImplicitDirectoryBehaviorLogic/PyArrow_directory_marker:_0-byte_with_children s3api_implicit_directory_test.go:143: ✓ PyArrow directory marker: 0-byte with children: correctly returns 404 --- PASS: TestImplicitDirectoryBehaviorLogic (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Implicit_directory:_0-byte_file_with_children,_no_trailing_slash (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Implicit_directory:_actual_directory_with_children,_no_trailing_slash (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Explicit_directory_request:_trailing_slash (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Empty_file:_0-byte_file_without_children (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Empty_directory:_0-byte_directory_without_children (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Regular_file:_non-zero_size (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/Versioned_bucket:_implicit_directory_should_return_200 (0.00s) --- PASS: TestImplicitDirectoryBehaviorLogic/PyArrow_directory_marker:_0-byte_with_children (0.00s) === RUN TestHasChildrenLogic === RUN TestHasChildrenLogic/Directory_with_children s3api_implicit_directory_test.go:218: ✓ Directory with children: correctly returns true === RUN TestHasChildrenLogic/Empty_directory s3api_implicit_directory_test.go:218: ✓ Empty directory: correctly returns false === RUN TestHasChildrenLogic/Directory_with_leading_slash_in_prefix s3api_implicit_directory_test.go:218: ✓ Directory with leading slash in prefix: correctly returns true --- PASS: TestHasChildrenLogic (0.00s) --- PASS: TestHasChildrenLogic/Directory_with_children (0.00s) --- PASS: TestHasChildrenLogic/Empty_directory (0.00s) --- PASS: TestHasChildrenLogic/Directory_with_leading_slash_in_prefix (0.00s) === RUN TestImplicitDirectoryEdgeCases === RUN TestImplicitDirectoryEdgeCases/PyArrow_write_dataset_creates_0-byte_files s3api_implicit_directory_test.go:265: Scenario: PyArrow creates 'dataset' as 0-byte file, then writes 'dataset/file.parquet' s3api_implicit_directory_test.go:266: Expected: HEAD dataset → 404 (has children), s3fs uses LIST → correctly identifies as directory === RUN TestImplicitDirectoryEdgeCases/Filer_creates_actual_directories s3api_implicit_directory_test.go:265: Scenario: Filer creates 'dataset' as actual directory with IsDirectory=true s3api_implicit_directory_test.go:266: Expected: HEAD dataset → 404 (has children), s3fs uses LIST → correctly identifies as directory === RUN TestImplicitDirectoryEdgeCases/Empty_file_edge_case s3api_implicit_directory_test.go:265: Scenario: User creates 'empty.txt' as 0-byte file with no children s3api_implicit_directory_test.go:266: Expected: HEAD empty.txt → 200 (no children), s3fs correctly reports as file === RUN TestImplicitDirectoryEdgeCases/Explicit_directory_request s3api_implicit_directory_test.go:265: Scenario: User requests 'dataset/' with trailing slash s3api_implicit_directory_test.go:266: Expected: HEAD dataset/ → 200 (explicit directory request), normal directory behavior === RUN TestImplicitDirectoryEdgeCases/Versioned_bucket s3api_implicit_directory_test.go:265: Scenario: Bucket has versioning enabled s3api_implicit_directory_test.go:266: Expected: HEAD dataset → 200 (skip implicit dir check), versioned semantics apply === RUN TestImplicitDirectoryEdgeCases/AWS_S3_compatibility s3api_implicit_directory_test.go:265: Scenario: Only 'dataset/file.txt' exists, no marker at 'dataset' s3api_implicit_directory_test.go:266: Expected: HEAD dataset → 404 (object doesn't exist), matches AWS S3 behavior --- PASS: TestImplicitDirectoryEdgeCases (0.00s) --- PASS: TestImplicitDirectoryEdgeCases/PyArrow_write_dataset_creates_0-byte_files (0.00s) --- PASS: TestImplicitDirectoryEdgeCases/Filer_creates_actual_directories (0.00s) --- PASS: TestImplicitDirectoryEdgeCases/Empty_file_edge_case (0.00s) --- PASS: TestImplicitDirectoryEdgeCases/Explicit_directory_request (0.00s) --- PASS: TestImplicitDirectoryEdgeCases/Versioned_bucket (0.00s) --- PASS: TestImplicitDirectoryEdgeCases/AWS_S3_compatibility (0.00s) === RUN TestImplicitDirectoryIntegration s3api_implicit_directory_test.go:278: Integration test - run manually with: cd test/s3/parquet && make test-implicit-dir-with-server --- SKIP: TestImplicitDirectoryIntegration (0.00s) === RUN TestUserInlinePolicySourceIpCondition_Denies --- PASS: TestUserInlinePolicySourceIpCondition_Denies (0.00s) === RUN TestUserInlinePolicySourceIpCondition_Allows --- PASS: TestUserInlinePolicySourceIpCondition_Allows (0.00s) === RUN TestGroupInlinePolicy_PutAndEnforce --- PASS: TestGroupInlinePolicy_PutAndEnforce (0.00s) === RUN TestUserInlinePolicy_ConditionDiscriminatesAllowVsDeny --- PASS: TestUserInlinePolicy_ConditionDiscriminatesAllowVsDeny (0.00s) === RUN TestUserInlinePolicy_ReloadFromStore --- PASS: TestUserInlinePolicy_ReloadFromStore (0.00s) === RUN TestComputeEntryIdentity_BasicFields --- PASS: TestComputeEntryIdentity_BasicFields (0.00s) === RUN TestComputeEntryIdentity_NilSafeMissingChunks --- PASS: TestComputeEntryIdentity_NilSafeMissingChunks (0.00s) === RUN TestHashExtended_OrderStable --- PASS: TestHashExtended_OrderStable (0.00s) === RUN TestHashExtended_DelimiterCollisionResistant --- PASS: TestHashExtended_DelimiterCollisionResistant (0.00s) === RUN TestHashExtended_NilEqualsEmpty --- PASS: TestHashExtended_NilEqualsEmpty (0.00s) === RUN TestIdentityMatches_NilWantTreatedAsMatch --- PASS: TestIdentityMatches_NilWantTreatedAsMatch (0.00s) === RUN TestIdentityMatches_NilLiveDoesNotMatch --- PASS: TestIdentityMatches_NilLiveDoesNotMatch (0.00s) === RUN TestIdentityMatches_AllFieldsCompared === RUN TestIdentityMatches_AllFieldsCompared/identical === RUN TestIdentityMatches_AllFieldsCompared/mtime-drift === RUN TestIdentityMatches_AllFieldsCompared/size-drift === RUN TestIdentityMatches_AllFieldsCompared/fid-drift === RUN TestIdentityMatches_AllFieldsCompared/extended-drift --- PASS: TestIdentityMatches_AllFieldsCompared (0.00s) --- PASS: TestIdentityMatches_AllFieldsCompared/identical (0.00s) --- PASS: TestIdentityMatches_AllFieldsCompared/mtime-drift (0.00s) --- PASS: TestIdentityMatches_AllFieldsCompared/size-drift (0.00s) --- PASS: TestIdentityMatches_AllFieldsCompared/fid-drift (0.00s) --- PASS: TestIdentityMatches_AllFieldsCompared/extended-drift (0.00s) === RUN TestLifecycleDelete_RejectsEmptyRequest --- PASS: TestLifecycleDelete_RejectsEmptyRequest (0.00s) === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs/#00 === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/ === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/. === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/.. === RUN TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/u1/extra --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs (0.00s) --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs/#00 (0.00s) --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads (0.00s) --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/ (0.00s) --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/. (0.00s) --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/.. (0.00s) --- PASS: TestLifecycleAbortMPU_RejectsTraversalUploadIDs/.uploads/u1/extra (0.00s) === RUN TestLifecycleDispatch_AbortMPUAfterFetchIsBlocked --- PASS: TestLifecycleDispatch_AbortMPUAfterFetchIsBlocked (0.00s) === RUN TestLifecycleDispatch_UnknownActionKindIsBlocked --- PASS: TestLifecycleDispatch_UnknownActionKindIsBlocked (0.00s) === RUN TestLifecycleDispatch_NoncurrentRequiresVersionID === RUN TestLifecycleDispatch_NoncurrentRequiresVersionID/NONCURRENT_DAYS === RUN TestLifecycleDispatch_NoncurrentRequiresVersionID/NEWER_NONCURRENT === RUN TestLifecycleDispatch_NoncurrentRequiresVersionID/EXPIRED_DELETE_MARKER --- PASS: TestLifecycleDispatch_NoncurrentRequiresVersionID (0.00s) --- PASS: TestLifecycleDispatch_NoncurrentRequiresVersionID/NONCURRENT_DAYS (0.00s) --- PASS: TestLifecycleDispatch_NoncurrentRequiresVersionID/NEWER_NONCURRENT (0.00s) --- PASS: TestLifecycleDispatch_NoncurrentRequiresVersionID/EXPIRED_DELETE_MARKER (0.00s) === RUN TestEntryUsesMetadataOnlyDelete === RUN TestEntryUsesMetadataOnlyDelete/nil_entry === RUN TestEntryUsesMetadataOnlyDelete/nil_attributes === RUN TestEntryUsesMetadataOnlyDelete/TtlSec=0_(no_per-write_stamp) === RUN TestEntryUsesMetadataOnlyDelete/TtlSec>0_(PR_9377_stamped_a_fast-path_TTL) === RUN TestEntryUsesMetadataOnlyDelete/TtlSec<0_should_not_happen_but_must_not_flip_the_path_on --- PASS: TestEntryUsesMetadataOnlyDelete (0.00s) --- PASS: TestEntryUsesMetadataOnlyDelete/nil_entry (0.00s) --- PASS: TestEntryUsesMetadataOnlyDelete/nil_attributes (0.00s) --- PASS: TestEntryUsesMetadataOnlyDelete/TtlSec=0_(no_per-write_stamp) (0.00s) --- PASS: TestEntryUsesMetadataOnlyDelete/TtlSec>0_(PR_9377_stamped_a_fast-path_TTL) (0.00s) --- PASS: TestEntryUsesMetadataOnlyDelete/TtlSec<0_should_not_happen_but_must_not_flip_the_path_on (0.00s) === RUN TestRecordMetadataOnlyIf_OnlyFiresWhenOn --- PASS: TestRecordMetadataOnlyIf_OnlyFiresWhenOn (0.00s) === RUN TestRecordMetadataOnlyIf_NilRequestSafe --- PASS: TestRecordMetadataOnlyIf_NilRequestSafe (0.00s) === RUN TestRecordMetadataOnlyIf_EmptyRuleHashCollapsesToEmptyLabel --- PASS: TestRecordMetadataOnlyIf_EmptyRuleHashCollapsesToEmptyLabel (0.00s) === RUN TestPrefixNormalizationInList === RUN TestPrefixNormalizationInList/simple_prefix === RUN TestPrefixNormalizationInList/leading_slash === RUN TestPrefixNormalizationInList/duplicate_slashes === RUN TestPrefixNormalizationInList/backslashes --- PASS: TestPrefixNormalizationInList (0.00s) --- PASS: TestPrefixNormalizationInList/simple_prefix (0.00s) --- PASS: TestPrefixNormalizationInList/leading_slash (0.00s) --- PASS: TestPrefixNormalizationInList/duplicate_slashes (0.00s) --- PASS: TestPrefixNormalizationInList/backslashes (0.00s) === RUN TestListPrefixConsistency --- PASS: TestListPrefixConsistency (0.00s) === RUN TestCheckUploadIDRequiresGeneratedFormat === RUN TestCheckUploadIDRequiresGeneratedFormat/legacy_hash === RUN TestCheckUploadIDRequiresGeneratedFormat/current_format === RUN TestCheckUploadIDRequiresGeneratedFormat/wrong_object_hash E0720 21:30:04.641335 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0000000000000000000000000000000000000000_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/arbitrary_suffix E0720 21:30:04.641346 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331_suffix are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/uppercase_suffix E0720 21:30:04.641354 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/short_suffix E0720 21:30:04.641364 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/long_suffix E0720 21:30:04.641372 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/slash_traversal E0720 21:30:04.641385 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331/../../../victim-bucket are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/suffix_traversal E0720 21:30:04.641395 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/../../../victim-bucket are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/backslash_traversal E0720 21:30:04.641403 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331\..\..\victim-bucket are not matched === RUN TestCheckUploadIDRequiresGeneratedFormat/nul_suffix E0720 21:30:04.641409 s3api_object_handlers_multipart.go:534 object dir/object and uploadID 0fb17d4690a62965c850aca2ac98bd432bdd2331_abcdef are not matched --- PASS: TestCheckUploadIDRequiresGeneratedFormat (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/legacy_hash (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/current_format (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/wrong_object_hash (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/arbitrary_suffix (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/uppercase_suffix (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/short_suffix (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/long_suffix (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/slash_traversal (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/suffix_traversal (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/backslash_traversal (0.00s) --- PASS: TestCheckUploadIDRequiresGeneratedFormat/nul_suffix (0.00s) === RUN TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch === RUN TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/missing_IV === RUN TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/empty_IV === RUN TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/short_IV === RUN TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/long_IV --- PASS: TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch (0.00s) --- PASS: TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/missing_IV (0.00s) --- PASS: TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/empty_IV (0.00s) --- PASS: TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/short_IV (0.00s) --- PASS: TestBuildMultipartSSEKMSReader_RejectsBadIVBeforeAnyFetch/long_IV (0.00s) === RUN TestBuildMultipartSSEKMSReader_RejectsMissingMetadataBeforeAnyFetch --- PASS: TestBuildMultipartSSEKMSReader_RejectsMissingMetadataBeforeAnyFetch (0.00s) === RUN TestBuildMultipartSSEKMSReader_RejectsUnparseableMetadataBeforeAnyFetch --- PASS: TestBuildMultipartSSEKMSReader_RejectsUnparseableMetadataBeforeAnyFetch (0.00s) === RUN TestBuildMultipartSSEKMSReader_SortsByOffset --- PASS: TestBuildMultipartSSEKMSReader_SortsByOffset (0.00s) === RUN TestNextMarkerWithNestedPrefix === RUN TestNextMarkerWithNestedPrefix/nested_prefix_with_both_requestDir_and_prefix === RUN TestNextMarkerWithNestedPrefix/only_requestDir,_no_prefix === RUN TestNextMarkerWithNestedPrefix/no_requestDir,_only_prefix === RUN TestNextMarkerWithNestedPrefix/deeply_nested_prefix --- PASS: TestNextMarkerWithNestedPrefix (0.00s) --- PASS: TestNextMarkerWithNestedPrefix/nested_prefix_with_both_requestDir_and_prefix (0.00s) --- PASS: TestNextMarkerWithNestedPrefix/only_requestDir,_no_prefix (0.00s) --- PASS: TestNextMarkerWithNestedPrefix/no_requestDir,_only_prefix (0.00s) --- PASS: TestNextMarkerWithNestedPrefix/deeply_nested_prefix (0.00s) === RUN TestNextMarkerWithCommonPrefix === RUN TestNextMarkerWithCommonPrefix/nested_prefix_with_CommonPrefix === RUN TestNextMarkerWithCommonPrefix/only_requestDir_with_CommonPrefix === RUN TestNextMarkerWithCommonPrefix/no_requestDir_with_CommonPrefix --- PASS: TestNextMarkerWithCommonPrefix (0.00s) --- PASS: TestNextMarkerWithCommonPrefix/nested_prefix_with_CommonPrefix (0.00s) --- PASS: TestNextMarkerWithCommonPrefix/only_requestDir_with_CommonPrefix (0.00s) --- PASS: TestNextMarkerWithCommonPrefix/no_requestDir_with_CommonPrefix (0.00s) === RUN TestObjectMetadataUpdateDirectory === RUN TestObjectMetadataUpdateDirectory/non-versioned_nested_key === RUN TestObjectMetadataUpdateDirectory/non-versioned_root_key === RUN TestObjectMetadataUpdateDirectory/non-versioned_deep_key === RUN TestObjectMetadataUpdateDirectory/null_version_nested_key === RUN TestObjectMetadataUpdateDirectory/explicit_null_version_nested_key === RUN TestObjectMetadataUpdateDirectory/empty_version_nested_key === RUN TestObjectMetadataUpdateDirectory/specific_version_nested_key === RUN TestObjectMetadataUpdateDirectory/latest_versioned_nested_key --- PASS: TestObjectMetadataUpdateDirectory (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/non-versioned_nested_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/non-versioned_root_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/non-versioned_deep_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/null_version_nested_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/explicit_null_version_nested_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/empty_version_nested_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/specific_version_nested_key (0.00s) --- PASS: TestObjectMetadataUpdateDirectory/latest_versioned_nested_key (0.00s) === RUN TestDownloadChunkData_AllocationBound s3api_object_handlers_copy_alloc_test.go:105: allocated=16805448 bytes, bound=25165824, output=16777216 --- PASS: TestDownloadChunkData_AllocationBound (0.04s) === RUN TestApplyDestChecksumHeaderToCopyRequest --- PASS: TestApplyDestChecksumHeaderToCopyRequest (0.00s) === RUN TestUploadEntryHasChecksum --- PASS: TestUploadEntryHasChecksum (0.00s) === RUN TestBuildCopyPartResult === RUN TestBuildCopyPartResult/CRC32 === RUN TestBuildCopyPartResult/CRC32C === RUN TestBuildCopyPartResult/CRC64NVME === RUN TestBuildCopyPartResult/SHA1 === RUN TestBuildCopyPartResult/SHA256 === RUN TestBuildCopyPartResult/Unknown --- PASS: TestBuildCopyPartResult (0.00s) --- PASS: TestBuildCopyPartResult/CRC32 (0.00s) --- PASS: TestBuildCopyPartResult/CRC32C (0.00s) --- PASS: TestBuildCopyPartResult/CRC64NVME (0.00s) --- PASS: TestBuildCopyPartResult/SHA1 (0.00s) --- PASS: TestBuildCopyPartResult/SHA256 (0.00s) --- PASS: TestBuildCopyPartResult/Unknown (0.00s) === RUN TestNewChunkUploadOption_AvoidsBytePool === RUN TestNewChunkUploadOption_AvoidsBytePool/empty_chunk === RUN TestNewChunkUploadOption_AvoidsBytePool/small_chunk === RUN TestNewChunkUploadOption_AvoidsBytePool/typical_8_MiB_chunk === RUN TestNewChunkUploadOption_AvoidsBytePool/large_64_MiB_chunk --- PASS: TestNewChunkUploadOption_AvoidsBytePool (0.01s) --- PASS: TestNewChunkUploadOption_AvoidsBytePool/empty_chunk (0.00s) --- PASS: TestNewChunkUploadOption_AvoidsBytePool/small_chunk (0.00s) --- PASS: TestNewChunkUploadOption_AvoidsBytePool/typical_8_MiB_chunk (0.00s) --- PASS: TestNewChunkUploadOption_AvoidsBytePool/large_64_MiB_chunk (0.01s) === RUN TestNewChunkUploadOption_PerCallIsolation --- PASS: TestNewChunkUploadOption_PerCallIsolation (0.00s) === RUN TestCopyDestinationPathResolvesBucketCollection --- PASS: TestCopyDestinationPathResolvesBucketCollection (0.00s) === RUN TestCopyEntryETagPrefersStoredExtendedETag --- PASS: TestCopyEntryETagPrefersStoredExtendedETag (0.00s) === RUN TestCopyEntryETagFallsBackToFilerETag --- PASS: TestCopyEntryETagFallsBackToFilerETag (0.00s) === RUN TestValidateConditionalCopyHeadersUsesStoredExtendedETag --- PASS: TestValidateConditionalCopyHeadersUsesStoredExtendedETag (0.00s) === RUN TestResolveDestinationMime === RUN TestResolveDestinationMime/COPY_keeps_source_mime === RUN TestResolveDestinationMime/COPY_without_request_CT === RUN TestResolveDestinationMime/COPY_with_empty_source_mime === RUN TestResolveDestinationMime/REPLACE_with_request_CT_wins === RUN TestResolveDestinationMime/REPLACE_without_request_CT_uses_default === RUN TestResolveDestinationMime/REPLACE_with_request_CT_and_empty_source === RUN TestResolveDestinationMime/REPLACE_without_request_CT_and_empty_source --- PASS: TestResolveDestinationMime (0.00s) --- PASS: TestResolveDestinationMime/COPY_keeps_source_mime (0.00s) --- PASS: TestResolveDestinationMime/COPY_without_request_CT (0.00s) --- PASS: TestResolveDestinationMime/COPY_with_empty_source_mime (0.00s) --- PASS: TestResolveDestinationMime/REPLACE_with_request_CT_wins (0.00s) --- PASS: TestResolveDestinationMime/REPLACE_without_request_CT_uses_default (0.00s) --- PASS: TestResolveDestinationMime/REPLACE_with_request_CT_and_empty_source (0.00s) --- PASS: TestResolveDestinationMime/REPLACE_without_request_CT_and_empty_source (0.00s) === RUN TestIsValidDirective === RUN TestIsValidDirective/#00 === RUN TestIsValidDirective/COPY === RUN TestIsValidDirective/REPLACE === RUN TestIsValidDirective/copy === RUN TestIsValidDirective/replace === RUN TestIsValidDirective/FOO === RUN TestIsValidDirective/REPLACE_ --- PASS: TestIsValidDirective (0.00s) --- PASS: TestIsValidDirective/#00 (0.00s) --- PASS: TestIsValidDirective/COPY (0.00s) --- PASS: TestIsValidDirective/REPLACE (0.00s) --- PASS: TestIsValidDirective/copy (0.00s) --- PASS: TestIsValidDirective/replace (0.00s) --- PASS: TestIsValidDirective/FOO (0.00s) --- PASS: TestIsValidDirective/REPLACE_ (0.00s) === RUN TestProcessMetadataBytes_ReplaceSystemHeaders --- PASS: TestProcessMetadataBytes_ReplaceSystemHeaders (0.00s) === RUN TestIsManagedCopyMetadataKey_CoversSystemHeaders --- PASS: TestIsManagedCopyMetadataKey_CoversSystemHeaders (0.00s) === RUN TestIsManagedCopyMetadataKey_CaseInsensitive === RUN TestIsManagedCopyMetadataKey_CaseInsensitive/cache-control === RUN TestIsManagedCopyMetadataKey_CaseInsensitive/CACHE-CONTROL === RUN TestIsManagedCopyMetadataKey_CaseInsensitive/content-encoding === RUN TestIsManagedCopyMetadataKey_CaseInsensitive/x-amz-meta-owner === RUN TestIsManagedCopyMetadataKey_CaseInsensitive/X-AMZ-META-OWNER === RUN TestIsManagedCopyMetadataKey_CaseInsensitive/x-amz-tagging-env --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive (0.00s) --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive/cache-control (0.00s) --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive/CACHE-CONTROL (0.00s) --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive/content-encoding (0.00s) --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive/x-amz-meta-owner (0.00s) --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive/X-AMZ-META-OWNER (0.00s) --- PASS: TestIsManagedCopyMetadataKey_CaseInsensitive/x-amz-tagging-env (0.00s) === RUN TestProcessMetadataBytes_CopyAcceptsLowercaseSourceKeys --- PASS: TestProcessMetadataBytes_CopyAcceptsLowercaseSourceKeys (0.00s) === RUN TestProcessMetadataBytes_CopyCanonicalSystemHeaderWinsOverLegacy --- PASS: TestProcessMetadataBytes_CopyCanonicalSystemHeaderWinsOverLegacy (0.00s) === RUN TestProcessMetadataBytes_CopyCanonicalTagWinsOverLegacy --- PASS: TestProcessMetadataBytes_CopyCanonicalTagWinsOverLegacy (0.00s) === RUN TestProcessMetadataBytes_CopyAcceptsLowercaseTagKeys --- PASS: TestProcessMetadataBytes_CopyAcceptsLowercaseTagKeys (0.00s) === RUN TestMergeCopyMetadata_ReplaceDropsStaleSystemHeader --- PASS: TestMergeCopyMetadata_ReplaceDropsStaleSystemHeader (0.00s) === RUN TestMergeCopyMetadata_ReplaceTaggingDropsStaleTags --- PASS: TestMergeCopyMetadata_ReplaceTaggingDropsStaleTags (0.00s) === RUN TestProcessMetadataBytes_CopyInheritsSystemHeaders --- PASS: TestProcessMetadataBytes_CopyInheritsSystemHeaders (0.00s) === RUN TestProcessMetadataBytes_CopyAppliesRequestedSSES3Header --- PASS: TestProcessMetadataBytes_CopyAppliesRequestedSSES3Header (0.00s) === RUN TestProcessMetadataBytes_CopyPreservesSourceSSEWhenRequestOmitsHeader --- PASS: TestProcessMetadataBytes_CopyPreservesSourceSSEWhenRequestOmitsHeader (0.00s) === RUN TestValidateDeleteObjectIdentifier === RUN TestValidateDeleteObjectIdentifier/clean_key === RUN TestValidateDeleteObjectIdentifier/clean_version === RUN TestValidateDeleteObjectIdentifier/key_traversal === RUN TestValidateDeleteObjectIdentifier/encoded_traversal_already_decoded === RUN TestValidateDeleteObjectIdentifier/backslash_traversal === RUN TestValidateDeleteObjectIdentifier/version_traversal === RUN TestValidateDeleteObjectIdentifier/version_backslash --- PASS: TestValidateDeleteObjectIdentifier (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/clean_key (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/clean_version (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/key_traversal (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/encoded_traversal_already_decoded (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/backslash_traversal (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/version_traversal (0.00s) --- PASS: TestValidateDeleteObjectIdentifier/version_backslash (0.00s) === RUN TestGetSpecificObjectVersionRejectsUnsafeVersionID --- PASS: TestGetSpecificObjectVersionRejectsUnsafeVersionID (0.00s) === RUN TestDeleteUnversionedObjectWithClient_MetadataOnlySkipsChunkDelete --- PASS: TestDeleteUnversionedObjectWithClient_MetadataOnlySkipsChunkDelete (0.00s) === RUN TestDeleteUnversionedObjectWithClient_FullDeletePreservesIsDeleteData --- PASS: TestDeleteUnversionedObjectWithClient_FullDeletePreservesIsDeleteData (0.00s) === RUN TestDeleteUnversionedObjectWithClient_FullPathFromBucketsRoot --- PASS: TestDeleteUnversionedObjectWithClient_FullPathFromBucketsRoot (0.00s) === RUN TestDeleteUnversionedObjectWithClientRejectsTraversal --- PASS: TestDeleteUnversionedObjectWithClientRejectsTraversal (0.00s) === RUN TestDeleteUnversionedObjectWithClient_PropagatesEntryAttributesIrrelevant --- PASS: TestDeleteUnversionedObjectWithClient_PropagatesEntryAttributesIrrelevant (0.00s) === RUN TestServeDirectoryContentContentType === RUN TestServeDirectoryContentContentType/bare_directory_answers_the_directory_marker_type === RUN TestServeDirectoryContentContentType/promoted_file_with_data_keeps_octet-stream === RUN TestServeDirectoryContentContentType/stored_mime_is_echoed_verbatim --- PASS: TestServeDirectoryContentContentType (0.00s) --- PASS: TestServeDirectoryContentContentType/bare_directory_answers_the_directory_marker_type (0.00s) --- PASS: TestServeDirectoryContentContentType/promoted_file_with_data_keeps_octet-stream (0.00s) --- PASS: TestServeDirectoryContentContentType/stored_mime_is_echoed_verbatim (0.00s) === RUN TestDirectoryListedAsCommonPrefix --- PASS: TestDirectoryListedAsCommonPrefix (0.00s) === RUN TestEmptyDirectorySurfacedAsMarker --- PASS: TestEmptyDirectorySurfacedAsMarker (0.00s) === RUN TestNonEmptyDirectoryGetsNoPhantomMarker --- PASS: TestNonEmptyDirectoryGetsNoPhantomMarker (0.00s) === RUN TestEmptyDirectoryHiddenInFlatListing --- PASS: TestEmptyDirectoryHiddenInFlatListing (0.00s) === RUN TestListObjectsHandler --- PASS: TestListObjectsHandler (0.00s) === RUN TestListObjectsV1NamespaceResponse --- PASS: TestListObjectsV1NamespaceResponse (0.00s) === RUN Test_normalizePrefixMarker === RUN Test_normalizePrefixMarker/bucket_root_listing_with_delimiter === RUN Test_normalizePrefixMarker/prefix_is_a_directory === RUN Test_normalizePrefixMarker/normal_case === RUN Test_normalizePrefixMarker/empty_prefix === RUN Test_normalizePrefixMarker/empty_directory --- PASS: Test_normalizePrefixMarker (0.00s) --- PASS: Test_normalizePrefixMarker/bucket_root_listing_with_delimiter (0.00s) --- PASS: Test_normalizePrefixMarker/prefix_is_a_directory (0.00s) --- PASS: Test_normalizePrefixMarker/normal_case (0.00s) --- PASS: Test_normalizePrefixMarker/empty_prefix (0.00s) --- PASS: Test_normalizePrefixMarker/empty_directory (0.00s) === RUN TestBuildTruncatedNextMarker === RUN TestBuildTruncatedNextMarker/does_not_duplicate_prefix_segment_in_next_continuation_token === RUN TestBuildTruncatedNextMarker/keeps_common_prefix_marker_trailing_slash === RUN TestBuildTruncatedNextMarker/includes_prefix_for_common_prefix_marker_when_request_dir_is_empty --- PASS: TestBuildTruncatedNextMarker (0.00s) --- PASS: TestBuildTruncatedNextMarker/does_not_duplicate_prefix_segment_in_next_continuation_token (0.00s) --- PASS: TestBuildTruncatedNextMarker/keeps_common_prefix_marker_trailing_slash (0.00s) --- PASS: TestBuildTruncatedNextMarker/includes_prefix_for_common_prefix_marker_when_request_dir_is_empty (0.00s) === RUN TestAllowUnorderedParameterValidation === RUN TestAllowUnorderedParameterValidation/getListObjectsV1Args_with_allow-unordered === RUN TestAllowUnorderedParameterValidation/getListObjectsV2Args_with_allow-unordered --- PASS: TestAllowUnorderedParameterValidation (0.00s) --- PASS: TestAllowUnorderedParameterValidation/getListObjectsV1Args_with_allow-unordered (0.00s) --- PASS: TestAllowUnorderedParameterValidation/getListObjectsV2Args_with_allow-unordered (0.00s) === RUN TestDoListFilerEntries_BucketRootPrefixSlashDelimiterSlash_ListsDirectories --- PASS: TestDoListFilerEntries_BucketRootPrefixSlashDelimiterSlash_ListsDirectories (0.00s) === RUN TestDoListFilerEntries_ExclusiveStartSkipsMarkerEcho --- PASS: TestDoListFilerEntries_ExclusiveStartSkipsMarkerEcho (0.00s) === RUN TestDoListFilerEntries_ExclusiveStartSkipsMarkerEchoWithSubsequentEntries --- PASS: TestDoListFilerEntries_ExclusiveStartSkipsMarkerEchoWithSubsequentEntries (0.00s) === RUN TestDoListFilerEntries_EmptyDirectoriesDoNotHideLaterEntries --- PASS: TestDoListFilerEntries_EmptyDirectoriesDoNotHideLaterEntries (0.00s) === RUN TestDoListFilerEntries_EmptyDirectoriesInSubdirectoryDoNotHideSiblings --- PASS: TestDoListFilerEntries_EmptyDirectoriesInSubdirectoryDoNotHideSiblings (0.00s) === RUN TestDoListFilerEntries_TruncationAcrossEmptyDirectories --- PASS: TestDoListFilerEntries_TruncationAcrossEmptyDirectories (0.00s) === RUN TestAllowUnorderedWithDelimiterValidation === RUN TestAllowUnorderedWithDelimiterValidation/should_return_error_when_allow-unordered=true_and_delimiter_are_both_present === RUN TestAllowUnorderedWithDelimiterValidation/should_allow_allow-unordered=true_without_delimiter === RUN TestAllowUnorderedWithDelimiterValidation/should_allow_delimiter_without_allow-unordered --- PASS: TestAllowUnorderedWithDelimiterValidation (0.00s) --- PASS: TestAllowUnorderedWithDelimiterValidation/should_return_error_when_allow-unordered=true_and_delimiter_are_both_present (0.00s) --- PASS: TestAllowUnorderedWithDelimiterValidation/should_allow_allow-unordered=true_without_delimiter (0.00s) --- PASS: TestAllowUnorderedWithDelimiterValidation/should_allow_delimiter_without_allow-unordered (0.00s) === RUN TestSanitizeV1MarkerEcho_NoProgressGuard --- PASS: TestSanitizeV1MarkerEcho_NoProgressGuard (0.00s) === RUN TestMaxKeysParameterValidation === RUN TestMaxKeysParameterValidation/valid_max-keys_values_should_work === RUN TestMaxKeysParameterValidation/invalid_max-keys_values_should_return_error === RUN TestMaxKeysParameterValidation/empty_max-keys_should_use_default --- PASS: TestMaxKeysParameterValidation (0.00s) --- PASS: TestMaxKeysParameterValidation/valid_max-keys_values_should_work (0.00s) --- PASS: TestMaxKeysParameterValidation/invalid_max-keys_values_should_return_error (0.00s) --- PASS: TestMaxKeysParameterValidation/empty_max-keys_should_use_default (0.00s) === RUN TestDelimiterWithDirectoryKeyObjects === RUN TestDelimiterWithDirectoryKeyObjects/directory_key_object_should_be_grouped_into_common_prefix_with_delimiter === RUN TestDelimiterWithDirectoryKeyObjects/directory_key_object_without_delimiter_should_be_individual_key --- PASS: TestDelimiterWithDirectoryKeyObjects (0.00s) --- PASS: TestDelimiterWithDirectoryKeyObjects/directory_key_object_should_be_grouped_into_common_prefix_with_delimiter (0.00s) --- PASS: TestDelimiterWithDirectoryKeyObjects/directory_key_object_without_delimiter_should_be_individual_key (0.00s) === RUN TestObjectLevelListPermissions === RUN TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions === RUN TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/allowed_prefix_exact_match === RUN TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/allowed_prefix_subdirectory === RUN TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/denied_different_prefix === RUN TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/denied_different_bucket === RUN TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/denied_root_level === RUN TestObjectLevelListPermissions/Bucket_Level_Permissions_Still_Work === RUN TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic === RUN TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/empty_object_with_prefix === RUN TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/slash_object_with_prefix === RUN TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/object_already_set === RUN TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/no_prefix_provided === RUN TestObjectLevelListPermissions/Issue_7039_Scenario === NAME TestObjectLevelListPermissions s3api_object_handlers_list_test.go:851: This test validates the fix for issue #7039 s3api_object_handlers_list_test.go:852: Object-level List permissions like 'List:bucket/prefix/*' now work correctly s3api_object_handlers_list_test.go:853: Middleware properly extracts prefix for permission validation --- PASS: TestObjectLevelListPermissions (0.00s) --- PASS: TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions (0.00s) --- PASS: TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/allowed_prefix_exact_match (0.00s) --- PASS: TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/allowed_prefix_subdirectory (0.00s) --- PASS: TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/denied_different_prefix (0.00s) --- PASS: TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/denied_different_bucket (0.00s) --- PASS: TestObjectLevelListPermissions/Identity_CanDo_Object_Level_Permissions/denied_root_level (0.00s) --- PASS: TestObjectLevelListPermissions/Bucket_Level_Permissions_Still_Work (0.00s) --- PASS: TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic (0.00s) --- PASS: TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/empty_object_with_prefix (0.00s) --- PASS: TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/slash_object_with_prefix (0.00s) --- PASS: TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/object_already_set (0.00s) --- PASS: TestObjectLevelListPermissions/Empty_Object_With_Prefix_Logic/no_prefix_provided (0.00s) --- PASS: TestObjectLevelListPermissions/Issue_7039_Scenario (0.00s) === RUN TestListObjectsV2_Regression --- PASS: TestListObjectsV2_Regression (0.00s) === RUN TestListObjectsV2_Regression_Sorting --- PASS: TestListObjectsV2_Regression_Sorting (0.00s) === RUN TestListObjectsV2_PrefixEndingWithSlash_DoesNotMatchSiblings --- PASS: TestListObjectsV2_PrefixEndingWithSlash_DoesNotMatchSiblings (0.00s) === RUN TestListObjectsV2_PrefixEndingWithSlash_WithDelimiter --- PASS: TestListObjectsV2_PrefixEndingWithSlash_WithDelimiter (0.00s) === RUN TestListObjectsWithVersionedObjects === RUN TestListObjectsWithVersionedObjects/List_all_objects_including_versioned_(no_delimiter) === RUN TestListObjectsWithVersionedObjects/List_bucket_root_with_delimiter --- PASS: TestListObjectsWithVersionedObjects (0.00s) --- PASS: TestListObjectsWithVersionedObjects/List_all_objects_including_versioned_(no_delimiter) (0.00s) --- PASS: TestListObjectsWithVersionedObjects/List_bucket_root_with_delimiter (0.00s) === RUN TestVersionedObjectsNoDuplication --- PASS: TestVersionedObjectsNoDuplication (0.00s) === RUN TestVersionedObjectsWithDeleteMarker --- PASS: TestVersionedObjectsWithDeleteMarker (0.00s) === RUN TestVersionedObjectsMaxKeys --- PASS: TestVersionedObjectsMaxKeys (0.00s) === RUN TestVersionsDirectoryNotTraversed --- PASS: TestVersionsDirectoryNotTraversed (0.00s) === RUN TestListObjectVersionsResult_XMLInterleaving --- PASS: TestListObjectVersionsResult_XMLInterleaving (0.00s) === RUN TestListObjectVersionsResult_XMLInterleavingMultipleKeys --- PASS: TestListObjectVersionsResult_XMLInterleavingMultipleKeys (0.00s) === RUN TestListObjectVersionsResult_XMLCommonPrefixes --- PASS: TestListObjectVersionsResult_XMLCommonPrefixes (0.00s) === RUN TestListObjectVersions_PrefixWithLeadingSlash === RUN TestListObjectVersions_PrefixWithLeadingSlash/Prefix_without_leading_slash_matches_file === RUN TestListObjectVersions_PrefixWithLeadingSlash/Prefix_with_leading_slash_(bug_fix_test)_-_normalized_and_matches_file === RUN TestListObjectVersions_PrefixWithLeadingSlash/Normalized_prefix_matches_subdirectory_file === RUN TestListObjectVersions_PrefixWithLeadingSlash/Normalized_prefix_does_not_match_different_path === RUN TestListObjectVersions_PrefixWithLeadingSlash/Prefix_with_leading_slash_allows_descending_into_directory === RUN TestListObjectVersions_PrefixWithLeadingSlash/Prefix_with_leading_slash_matches_directory_with_trailing_slash --- PASS: TestListObjectVersions_PrefixWithLeadingSlash (0.00s) --- PASS: TestListObjectVersions_PrefixWithLeadingSlash/Prefix_without_leading_slash_matches_file (0.00s) --- PASS: TestListObjectVersions_PrefixWithLeadingSlash/Prefix_with_leading_slash_(bug_fix_test)_-_normalized_and_matches_file (0.00s) --- PASS: TestListObjectVersions_PrefixWithLeadingSlash/Normalized_prefix_matches_subdirectory_file (0.00s) --- PASS: TestListObjectVersions_PrefixWithLeadingSlash/Normalized_prefix_does_not_match_different_path (0.00s) --- PASS: TestListObjectVersions_PrefixWithLeadingSlash/Prefix_with_leading_slash_allows_descending_into_directory (0.00s) --- PASS: TestListObjectVersions_PrefixWithLeadingSlash/Prefix_with_leading_slash_matches_directory_with_trailing_slash (0.00s) === RUN TestComputeStartFrom === RUN TestComputeStartFrom/empty_marker === RUN TestComputeStartFrom/empty_marker_with_path === RUN TestComputeStartFrom/root_level_file === RUN TestComputeStartFrom/root_level_with_subpath === RUN TestComputeStartFrom/matching_subdir === RUN TestComputeStartFrom/deeper_subdir === RUN TestComputeStartFrom/at_leaf_level === RUN TestComputeStartFrom/unrelated_directory === RUN TestComputeStartFrom/marker_equals_relativePath === RUN TestComputeStartFrom/marker_before_directory --- PASS: TestComputeStartFrom (0.00s) --- PASS: TestComputeStartFrom/empty_marker (0.00s) --- PASS: TestComputeStartFrom/empty_marker_with_path (0.00s) --- PASS: TestComputeStartFrom/root_level_file (0.00s) --- PASS: TestComputeStartFrom/root_level_with_subpath (0.00s) --- PASS: TestComputeStartFrom/matching_subdir (0.00s) --- PASS: TestComputeStartFrom/deeper_subdir (0.00s) --- PASS: TestComputeStartFrom/at_leaf_level (0.00s) --- PASS: TestComputeStartFrom/unrelated_directory (0.00s) --- PASS: TestComputeStartFrom/marker_equals_relativePath (0.00s) --- PASS: TestComputeStartFrom/marker_before_directory (0.00s) === RUN TestProcessDirectorySkipsBeforeMarker === RUN TestProcessDirectorySkipsBeforeMarker/no_marker === RUN TestProcessDirectorySkipsBeforeMarker/dir_before_marker === RUN TestProcessDirectorySkipsBeforeMarker/marker_descends_into_dir === RUN TestProcessDirectorySkipsBeforeMarker/dir_after_marker === RUN TestProcessDirectorySkipsBeforeMarker/same_prefix_different_suffix --- PASS: TestProcessDirectorySkipsBeforeMarker (0.00s) --- PASS: TestProcessDirectorySkipsBeforeMarker/no_marker (0.00s) --- PASS: TestProcessDirectorySkipsBeforeMarker/dir_before_marker (0.00s) --- PASS: TestProcessDirectorySkipsBeforeMarker/marker_descends_into_dir (0.00s) --- PASS: TestProcessDirectorySkipsBeforeMarker/dir_after_marker (0.00s) --- PASS: TestProcessDirectorySkipsBeforeMarker/same_prefix_different_suffix (0.00s) === RUN TestPostPolicyKeyNormalization === RUN TestPostPolicyKeyNormalization/key_without_leading_slash === RUN TestPostPolicyKeyNormalization/key_with_leading_slash === RUN TestPostPolicyKeyNormalization/key_with_path_without_leading_slash === RUN TestPostPolicyKeyNormalization/key_with_path_with_leading_slash === RUN TestPostPolicyKeyNormalization/simple_filename === RUN TestPostPolicyKeyNormalization/key_with_duplicate_slashes === RUN TestPostPolicyKeyNormalization/key_with_leading_duplicate_slashes === RUN TestPostPolicyKeyNormalization/key_with_trailing_slash --- PASS: TestPostPolicyKeyNormalization (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_without_leading_slash (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_with_leading_slash (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_with_path_without_leading_slash (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_with_path_with_leading_slash (0.00s) --- PASS: TestPostPolicyKeyNormalization/simple_filename (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_with_duplicate_slashes (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_with_leading_duplicate_slashes (0.00s) --- PASS: TestPostPolicyKeyNormalization/key_with_trailing_slash (0.00s) === RUN TestNormalizeObjectKey === RUN TestNormalizeObjectKey/empty_string === RUN TestNormalizeObjectKey/simple_file === RUN TestNormalizeObjectKey/with_leading_slash === RUN TestNormalizeObjectKey/path_without_slash === RUN TestNormalizeObjectKey/path_with_slash === RUN TestNormalizeObjectKey/duplicate_slashes === RUN TestNormalizeObjectKey/leading_duplicates === RUN TestNormalizeObjectKey/all_duplicates === RUN TestNormalizeObjectKey/just_slashes === RUN TestNormalizeObjectKey/trailing_slash === RUN TestNormalizeObjectKey/backslash_to_forward_slash === RUN TestNormalizeObjectKey/windows_path === RUN TestNormalizeObjectKey/mixed_slashes --- PASS: TestNormalizeObjectKey (0.00s) --- PASS: TestNormalizeObjectKey/empty_string (0.00s) --- PASS: TestNormalizeObjectKey/simple_file (0.00s) --- PASS: TestNormalizeObjectKey/with_leading_slash (0.00s) --- PASS: TestNormalizeObjectKey/path_without_slash (0.00s) --- PASS: TestNormalizeObjectKey/path_with_slash (0.00s) --- PASS: TestNormalizeObjectKey/duplicate_slashes (0.00s) --- PASS: TestNormalizeObjectKey/leading_duplicates (0.00s) --- PASS: TestNormalizeObjectKey/all_duplicates (0.00s) --- PASS: TestNormalizeObjectKey/just_slashes (0.00s) --- PASS: TestNormalizeObjectKey/trailing_slash (0.00s) --- PASS: TestNormalizeObjectKey/backslash_to_forward_slash (0.00s) --- PASS: TestNormalizeObjectKey/windows_path (0.00s) --- PASS: TestNormalizeObjectKey/mixed_slashes (0.00s) === RUN TestPostPolicyFilenameSubstitution === RUN TestPostPolicyFilenameSubstitution/filename_at_end === RUN TestPostPolicyFilenameSubstitution/filename_in_middle === RUN TestPostPolicyFilenameSubstitution/no_substitution_needed === RUN TestPostPolicyFilenameSubstitution/filename_only --- PASS: TestPostPolicyFilenameSubstitution (0.00s) --- PASS: TestPostPolicyFilenameSubstitution/filename_at_end (0.00s) --- PASS: TestPostPolicyFilenameSubstitution/filename_in_middle (0.00s) --- PASS: TestPostPolicyFilenameSubstitution/no_substitution_needed (0.00s) --- PASS: TestPostPolicyFilenameSubstitution/filename_only (0.00s) === RUN TestExtractPostPolicyFormValues === RUN TestExtractPostPolicyFormValues/basic_upload === RUN TestExtractPostPolicyFormValues/upload_with_path_key --- PASS: TestExtractPostPolicyFormValues (0.00s) --- PASS: TestExtractPostPolicyFormValues/basic_upload (0.00s) --- PASS: TestExtractPostPolicyFormValues/upload_with_path_key (0.00s) === RUN TestPostPolicyPathConstruction === RUN TestPostPolicyPathConstruction/simple_key_without_slash_-_the_bug_case === RUN TestPostPolicyPathConstruction/simple_key_with_slash === RUN TestPostPolicyPathConstruction/nested_path_without_leading_slash === RUN TestPostPolicyPathConstruction/nested_path_with_leading_slash === RUN TestPostPolicyPathConstruction/key_with_duplicate_slashes --- PASS: TestPostPolicyPathConstruction (0.00s) --- PASS: TestPostPolicyPathConstruction/simple_key_without_slash_-_the_bug_case (0.00s) --- PASS: TestPostPolicyPathConstruction/simple_key_with_slash (0.00s) --- PASS: TestPostPolicyPathConstruction/nested_path_without_leading_slash (0.00s) --- PASS: TestPostPolicyPathConstruction/nested_path_with_leading_slash (0.00s) --- PASS: TestPostPolicyPathConstruction/key_with_duplicate_slashes (0.00s) === RUN TestApplyPostPolicyFormHeaders_ForwardsAcl --- PASS: TestApplyPostPolicyFormHeaders_ForwardsAcl (0.00s) === RUN TestApplyPostPolicyFormHeaders_ForwardsContentHeaders === RUN TestApplyPostPolicyFormHeaders_ForwardsContentHeaders/Content-Encoding === RUN TestApplyPostPolicyFormHeaders_ForwardsContentHeaders/Content-Language --- PASS: TestApplyPostPolicyFormHeaders_ForwardsContentHeaders (0.00s) --- PASS: TestApplyPostPolicyFormHeaders_ForwardsContentHeaders/Content-Encoding (0.00s) --- PASS: TestApplyPostPolicyFormHeaders_ForwardsContentHeaders/Content-Language (0.00s) === RUN TestApplyPostPolicyFormHeaders_ForwardsXAmzHeaders --- PASS: TestApplyPostPolicyFormHeaders_ForwardsXAmzHeaders (0.00s) === RUN TestApplyPostPolicyFormHeaders_SkipsReserved --- PASS: TestApplyPostPolicyFormHeaders_SkipsReserved (0.00s) === RUN TestApplyPostPolicyFormHeaders_KeepsExistingCacheControl --- PASS: TestApplyPostPolicyFormHeaders_KeepsExistingCacheControl (0.00s) === RUN TestApplyPostPolicyFormHeaders_IgnoresContentType --- PASS: TestApplyPostPolicyFormHeaders_IgnoresContentType (0.00s) === RUN TestPostPolicyBucketHandlerKeyExtraction === RUN TestPostPolicyBucketHandlerKeyExtraction/key_without_leading_slash === RUN TestPostPolicyBucketHandlerKeyExtraction/key_with_leading_slash === RUN TestPostPolicyBucketHandlerKeyExtraction/key_with_duplicate_slashes --- PASS: TestPostPolicyBucketHandlerKeyExtraction (0.00s) --- PASS: TestPostPolicyBucketHandlerKeyExtraction/key_without_leading_slash (0.00s) --- PASS: TestPostPolicyBucketHandlerKeyExtraction/key_with_leading_slash (0.00s) --- PASS: TestPostPolicyBucketHandlerKeyExtraction/key_with_duplicate_slashes (0.00s) === RUN TestPostPolicyBucketHandlerRejectsTraversalKey --- PASS: TestPostPolicyBucketHandlerRejectsTraversalKey (0.00s) === RUN TestPostPolicyBucketHandler_PolicyViolationReturns403 --- PASS: TestPostPolicyBucketHandler_PolicyViolationReturns403 (0.00s) === RUN TestDetectRequestedChecksumAlgorithm === RUN TestDetectRequestedChecksumAlgorithm/sdk_algorithm_header === RUN TestDetectRequestedChecksumAlgorithm/algorithm_header === RUN TestDetectRequestedChecksumAlgorithm/presigned_url_hoists_sdk_algorithm_to_query === RUN TestDetectRequestedChecksumAlgorithm/presigned_url_hoists_checksum-algorithm_to_query_(lowercase) === RUN TestDetectRequestedChecksumAlgorithm/presigned_url_hoists_individual_checksum_value === RUN TestDetectRequestedChecksumAlgorithm/unsupported_in_query_returns_error W0720 21:30:04.693709 s3api_object_handlers_put.go:1028 unsupported checksum algorithm in X-Amz-Sdk-Checksum-Algorithm: "MD5" === RUN TestDetectRequestedChecksumAlgorithm/no_checksum --- PASS: TestDetectRequestedChecksumAlgorithm (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/sdk_algorithm_header (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/algorithm_header (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/presigned_url_hoists_sdk_algorithm_to_query (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/presigned_url_hoists_checksum-algorithm_to_query_(lowercase) (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/presigned_url_hoists_individual_checksum_value (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/unsupported_in_query_returns_error (0.00s) --- PASS: TestDetectRequestedChecksumAlgorithm/no_checksum (0.00s) === RUN TestLookupHeaderOrQueryCaseInsensitive --- PASS: TestLookupHeaderOrQueryCaseInsensitive (0.00s) === RUN TestMapChunkedUploadErrorToS3Error === RUN TestMapChunkedUploadErrorToS3Error/truncated_source_read_maps_to_IncompleteBody === RUN TestMapChunkedUploadErrorToS3Error/volume_upload_unexpected_EOF_maps_to_InternalError === RUN TestMapChunkedUploadErrorToS3Error/payload_checksum_mismatch_maps_to_InvalidDigest === RUN TestMapChunkedUploadErrorToS3Error/other_errors_map_to_InternalError --- PASS: TestMapChunkedUploadErrorToS3Error (0.00s) --- PASS: TestMapChunkedUploadErrorToS3Error/truncated_source_read_maps_to_IncompleteBody (0.00s) --- PASS: TestMapChunkedUploadErrorToS3Error/volume_upload_unexpected_EOF_maps_to_InternalError (0.00s) --- PASS: TestMapChunkedUploadErrorToS3Error/payload_checksum_mismatch_maps_to_InvalidDigest (0.00s) --- PASS: TestMapChunkedUploadErrorToS3Error/other_errors_map_to_InternalError (0.00s) === RUN TestNewLifecycleTTLResolver_NilOnEmpty --- PASS: TestNewLifecycleTTLResolver_NilOnEmpty (0.00s) === RUN TestNewLifecycleTTLResolver_NilOnVersionedBucket --- PASS: TestNewLifecycleTTLResolver_NilOnVersionedBucket (0.00s) === RUN TestNewLifecycleTTLResolver_DropsTagFilteredRules --- PASS: TestNewLifecycleTTLResolver_DropsTagFilteredRules (0.00s) === RUN TestNewLifecycleTTLResolver_DropsDisabledAndNonExpirationDays --- PASS: TestNewLifecycleTTLResolver_DropsDisabledAndNonExpirationDays (0.00s) === RUN TestResolve_PrefixMatch --- PASS: TestResolve_PrefixMatch (0.00s) === RUN TestResolve_OverlappingRulesShorterExpirationWins --- PASS: TestResolve_OverlappingRulesShorterExpirationWins (0.00s) === RUN TestResolve_OverflowDefersToWorker --- PASS: TestResolve_OverflowDefersToWorker (0.00s) === RUN TestResolve_OverflowSkipsButShorterStillFires --- PASS: TestResolve_OverflowSkipsButShorterStillFires (0.00s) === RUN TestResolve_SizeFilter --- PASS: TestResolve_SizeFilter (0.00s) === RUN TestResolve_NilReceiverReturnsZero --- PASS: TestResolve_NilReceiverReturnsZero (0.00s) === RUN TestNewBucketConfigFromEntry_RefreshesLifecycleTTL --- PASS: TestNewBucketConfigFromEntry_RefreshesLifecycleTTL (0.00s) === RUN TestNewBucketConfigFromEntry_ObjectLockTreatedAsVersioned --- PASS: TestNewBucketConfigFromEntry_ObjectLockTreatedAsVersioned (0.00s) === RUN TestNewBucketConfigFromEntry_TtlFastPathOptIn --- PASS: TestNewBucketConfigFromEntry_TtlFastPathOptIn (0.00s) === RUN TestVeeamObjectLockBugFix === RUN TestVeeamObjectLockBugFix/Bug_case:_bucket_with_no_extended_attributes === RUN TestVeeamObjectLockBugFix/Fix_verification:_bucket_with_Object_Lock_enabled_via_boolean_flag === RUN TestVeeamObjectLockBugFix/Fix_verification:_bucket_with_Object_Lock_enabled_via_Enabled_constant --- PASS: TestVeeamObjectLockBugFix (0.00s) --- PASS: TestVeeamObjectLockBugFix/Bug_case:_bucket_with_no_extended_attributes (0.00s) --- PASS: TestVeeamObjectLockBugFix/Fix_verification:_bucket_with_Object_Lock_enabled_via_boolean_flag (0.00s) --- PASS: TestVeeamObjectLockBugFix/Fix_verification:_bucket_with_Object_Lock_enabled_via_Enabled_constant (0.00s) === RUN TestExtractObjectLockMetadataFromRequest === RUN TestExtractObjectLockMetadataFromRequest/Extract_COMPLIANCE_mode_and_retention_date === RUN TestExtractObjectLockMetadataFromRequest/Extract_GOVERNANCE_mode_and_retention_date === RUN TestExtractObjectLockMetadataFromRequest/Extract_legal_hold_ON === RUN TestExtractObjectLockMetadataFromRequest/Extract_legal_hold_OFF === RUN TestExtractObjectLockMetadataFromRequest/Handle_all_object_lock_headers_together === RUN TestExtractObjectLockMetadataFromRequest/Handle_no_object_lock_headers === RUN TestExtractObjectLockMetadataFromRequest/Handle_invalid_retention_date_-_should_return_error E0720 21:30:04.694278 s3api_object_handlers_put.go:1628 extractObjectLockMetadataFromRequest: failed to parse retention until date, expected format: 2006-01-02T15:04:05Z07:00, error: parsing time "invalid-date" as "2006-01-02T15:04:05Z07:00": cannot parse "invalid-date" as "2006" === RUN TestExtractObjectLockMetadataFromRequest/Handle_invalid_legal_hold_value_-_should_return_error E0720 21:30:04.694304 s3api_object_handlers_put.go:1642 extractObjectLockMetadataFromRequest: unexpected legal hold value provided, expected 'ON' or 'OFF' --- PASS: TestExtractObjectLockMetadataFromRequest (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Extract_COMPLIANCE_mode_and_retention_date (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Extract_GOVERNANCE_mode_and_retention_date (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Extract_legal_hold_ON (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Extract_legal_hold_OFF (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Handle_all_object_lock_headers_together (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Handle_no_object_lock_headers (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Handle_invalid_retention_date_-_should_return_error (0.00s) --- PASS: TestExtractObjectLockMetadataFromRequest/Handle_invalid_legal_hold_value_-_should_return_error (0.00s) === RUN TestAddObjectLockHeadersToResponse === RUN TestAddObjectLockHeadersToResponse/Add_COMPLIANCE_mode_and_retention_date_to_response === RUN TestAddObjectLockHeadersToResponse/Add_GOVERNANCE_mode_to_response === RUN TestAddObjectLockHeadersToResponse/Add_legal_hold_ON_to_response === RUN TestAddObjectLockHeadersToResponse/Add_legal_hold_OFF_to_response === RUN TestAddObjectLockHeadersToResponse/Add_all_object_lock_headers_to_response === RUN TestAddObjectLockHeadersToResponse/Handle_entry_with_no_object_lock_metadata === RUN TestAddObjectLockHeadersToResponse/Handle_entry_with_object_lock_mode_but_no_legal_hold_-_should_default_to_OFF === RUN TestAddObjectLockHeadersToResponse/Handle_entry_with_retention_date_but_no_legal_hold_-_should_default_to_OFF === RUN TestAddObjectLockHeadersToResponse/Handle_nil_entry_gracefully === RUN TestAddObjectLockHeadersToResponse/Handle_entry_with_nil_Extended_map_gracefully === RUN TestAddObjectLockHeadersToResponse/Handle_invalid_retention_timestamp_gracefully E0720 21:30:04.694457 s3api_object_handlers.go:2512 addObjectLockHeadersToResponse: failed to parse retention until date from stored metadata (dateStr: invalid-timestamp): strconv.ParseInt: parsing "invalid-timestamp": invalid syntax --- PASS: TestAddObjectLockHeadersToResponse (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Add_COMPLIANCE_mode_and_retention_date_to_response (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Add_GOVERNANCE_mode_to_response (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Add_legal_hold_ON_to_response (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Add_legal_hold_OFF_to_response (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Add_all_object_lock_headers_to_response (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Handle_entry_with_no_object_lock_metadata (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Handle_entry_with_object_lock_mode_but_no_legal_hold_-_should_default_to_OFF (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Handle_entry_with_retention_date_but_no_legal_hold_-_should_default_to_OFF (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Handle_nil_entry_gracefully (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Handle_entry_with_nil_Extended_map_gracefully (0.00s) --- PASS: TestAddObjectLockHeadersToResponse/Handle_invalid_retention_timestamp_gracefully (0.00s) === RUN TestObjectLockHeaderRoundTrip === RUN TestObjectLockHeaderRoundTrip/Complete_round_trip_for_COMPLIANCE_mode === RUN TestObjectLockHeaderRoundTrip/Complete_round_trip_for_GOVERNANCE_mode --- PASS: TestObjectLockHeaderRoundTrip (0.00s) --- PASS: TestObjectLockHeaderRoundTrip/Complete_round_trip_for_COMPLIANCE_mode (0.00s) --- PASS: TestObjectLockHeaderRoundTrip/Complete_round_trip_for_GOVERNANCE_mode (0.00s) === RUN TestValidateObjectLockHeaders === RUN TestValidateObjectLockHeaders/Valid_COMPLIANCE_mode_with_retention_date_on_versioned_bucket === RUN TestValidateObjectLockHeaders/Valid_GOVERNANCE_mode_with_retention_date_on_versioned_bucket === RUN TestValidateObjectLockHeaders/Valid_legal_hold_ON_on_versioned_bucket === RUN TestValidateObjectLockHeaders/Valid_legal_hold_OFF_on_versioned_bucket === RUN TestValidateObjectLockHeaders/Invalid_object_lock_mode === RUN TestValidateObjectLockHeaders/Invalid_legal_hold_status === RUN TestValidateObjectLockHeaders/Object_lock_headers_on_non-versioned_bucket === RUN TestValidateObjectLockHeaders/Invalid_retention_date_format === RUN TestValidateObjectLockHeaders/Retention_date_in_the_past === RUN TestValidateObjectLockHeaders/Mode_without_retention_date === RUN TestValidateObjectLockHeaders/Retention_date_without_mode === RUN TestValidateObjectLockHeaders/Governance_bypass_header_on_non-versioned_bucket === RUN TestValidateObjectLockHeaders/Governance_bypass_header_on_versioned_bucket_should_pass === RUN TestValidateObjectLockHeaders/No_object_lock_headers_should_pass === RUN TestValidateObjectLockHeaders/Mixed_valid_headers_should_pass --- PASS: TestValidateObjectLockHeaders (0.00s) --- PASS: TestValidateObjectLockHeaders/Valid_COMPLIANCE_mode_with_retention_date_on_versioned_bucket (0.00s) --- PASS: TestValidateObjectLockHeaders/Valid_GOVERNANCE_mode_with_retention_date_on_versioned_bucket (0.00s) --- PASS: TestValidateObjectLockHeaders/Valid_legal_hold_ON_on_versioned_bucket (0.00s) --- PASS: TestValidateObjectLockHeaders/Valid_legal_hold_OFF_on_versioned_bucket (0.00s) --- PASS: TestValidateObjectLockHeaders/Invalid_object_lock_mode (0.00s) --- PASS: TestValidateObjectLockHeaders/Invalid_legal_hold_status (0.00s) --- PASS: TestValidateObjectLockHeaders/Object_lock_headers_on_non-versioned_bucket (0.00s) --- PASS: TestValidateObjectLockHeaders/Invalid_retention_date_format (0.00s) --- PASS: TestValidateObjectLockHeaders/Retention_date_in_the_past (0.00s) --- PASS: TestValidateObjectLockHeaders/Mode_without_retention_date (0.00s) --- PASS: TestValidateObjectLockHeaders/Retention_date_without_mode (0.00s) --- PASS: TestValidateObjectLockHeaders/Governance_bypass_header_on_non-versioned_bucket (0.00s) --- PASS: TestValidateObjectLockHeaders/Governance_bypass_header_on_versioned_bucket_should_pass (0.00s) --- PASS: TestValidateObjectLockHeaders/No_object_lock_headers_should_pass (0.00s) --- PASS: TestValidateObjectLockHeaders/Mixed_valid_headers_should_pass (0.00s) === RUN TestMapValidationErrorToS3Error === RUN TestMapValidationErrorToS3Error/ErrObjectLockVersioningRequired === RUN TestMapValidationErrorToS3Error/ErrInvalidObjectLockMode === RUN TestMapValidationErrorToS3Error/ErrInvalidLegalHoldStatus === RUN TestMapValidationErrorToS3Error/ErrInvalidRetentionDateFormat === RUN TestMapValidationErrorToS3Error/ErrRetentionDateMustBeFuture === RUN TestMapValidationErrorToS3Error/ErrObjectLockModeRequiresDate === RUN TestMapValidationErrorToS3Error/ErrRetentionDateRequiresMode === RUN TestMapValidationErrorToS3Error/ErrGovernanceBypassVersioningRequired === RUN TestMapValidationErrorToS3Error/Unknown_error_defaults_to_ErrInvalidRequest --- PASS: TestMapValidationErrorToS3Error (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrObjectLockVersioningRequired (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrInvalidObjectLockMode (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrInvalidLegalHoldStatus (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrInvalidRetentionDateFormat (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrRetentionDateMustBeFuture (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrObjectLockModeRequiresDate (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrRetentionDateRequiresMode (0.00s) --- PASS: TestMapValidationErrorToS3Error/ErrGovernanceBypassVersioningRequired (0.00s) --- PASS: TestMapValidationErrorToS3Error/Unknown_error_defaults_to_ErrInvalidRequest (0.00s) === RUN TestObjectLockPermissionLogic === RUN TestObjectLockPermissionLogic/Non-versioned_bucket_PUT_operation_logic s3api_object_lock_headers_test.go:626: For non-versioned buckets: s3api_object_lock_headers_test.go:627: - PUT operations overwrite existing objects s3api_object_lock_headers_test.go:628: - Must check existing object lock protections before allowing overwrite s3api_object_lock_headers_test.go:629: - Governance bypass headers can be used to override GOVERNANCE mode retention s3api_object_lock_headers_test.go:630: - COMPLIANCE mode retention and legal holds cannot be bypassed === RUN TestObjectLockPermissionLogic/Versioned_bucket_PUT_operation_logic s3api_object_lock_headers_test.go:643: For versioned buckets: s3api_object_lock_headers_test.go:644: - PUT operations create new versions without overwriting existing objects s3api_object_lock_headers_test.go:645: - No need to check existing object lock protections s3api_object_lock_headers_test.go:646: - Only validate object lock headers for the new version being created s3api_object_lock_headers_test.go:647: - Each version has independent object lock settings === RUN TestObjectLockPermissionLogic/Governance_bypass_header_validation s3api_object_lock_headers_test.go:656: Governance bypass behavior: s3api_object_lock_headers_test.go:657: - Only valid on versioned buckets (header validation) s3api_object_lock_headers_test.go:658: - For non-versioned buckets: Allows overwriting objects under GOVERNANCE retention s3api_object_lock_headers_test.go:659: - For versioned buckets: Not typically needed for PUT operations s3api_object_lock_headers_test.go:660: - Must have s3:BypassGovernanceRetention permission --- PASS: TestObjectLockPermissionLogic (0.00s) --- PASS: TestObjectLockPermissionLogic/Non-versioned_bucket_PUT_operation_logic (0.00s) --- PASS: TestObjectLockPermissionLogic/Versioned_bucket_PUT_operation_logic (0.00s) --- PASS: TestObjectLockPermissionLogic/Governance_bypass_header_validation (0.00s) === RUN TestSetObjectOwnerFromRequest === RUN TestSetObjectOwnerFromRequest/BucketOwnerEnforced_-_use_bucket_owner === RUN TestSetObjectOwnerFromRequest/ObjectWriter_-_use_uploader === RUN TestSetObjectOwnerFromRequest/BucketOwnerPreferred_-_use_uploader === RUN TestSetObjectOwnerFromRequest/BucketOwnerEnforced_but_owner_is_nil_-_fallback_to_uploader === RUN TestSetObjectOwnerFromRequest/BucketOwnerEnforced_but_owner_ID_is_nil_-_fallback_to_uploader === RUN TestSetObjectOwnerFromRequest/Bucket_metadata_error_-_fallback_to_uploader === RUN TestSetObjectOwnerFromRequest/Bucket_registry_is_nil_-_fallback_to_uploader === RUN TestSetObjectOwnerFromRequest/Empty_uploader_account_ID_-_no_owner_set --- PASS: TestSetObjectOwnerFromRequest (0.00s) --- PASS: TestSetObjectOwnerFromRequest/BucketOwnerEnforced_-_use_bucket_owner (0.00s) --- PASS: TestSetObjectOwnerFromRequest/ObjectWriter_-_use_uploader (0.00s) --- PASS: TestSetObjectOwnerFromRequest/BucketOwnerPreferred_-_use_uploader (0.00s) --- PASS: TestSetObjectOwnerFromRequest/BucketOwnerEnforced_but_owner_is_nil_-_fallback_to_uploader (0.00s) --- PASS: TestSetObjectOwnerFromRequest/BucketOwnerEnforced_but_owner_ID_is_nil_-_fallback_to_uploader (0.00s) --- PASS: TestSetObjectOwnerFromRequest/Bucket_metadata_error_-_fallback_to_uploader (0.00s) --- PASS: TestSetObjectOwnerFromRequest/Bucket_registry_is_nil_-_fallback_to_uploader (0.00s) --- PASS: TestSetObjectOwnerFromRequest/Empty_uploader_account_ID_-_no_owner_set (0.00s) === RUN TestParseAndValidateRangeRejectsMalformedNumericOffsets === RUN TestParseAndValidateRangeRejectsMalformedNumericOffsets/invalid_start === RUN TestParseAndValidateRangeRejectsMalformedNumericOffsets/invalid_end === RUN TestParseAndValidateRangeRejectsMalformedNumericOffsets/invalid_open_ended_start --- PASS: TestParseAndValidateRangeRejectsMalformedNumericOffsets (0.00s) --- PASS: TestParseAndValidateRangeRejectsMalformedNumericOffsets/invalid_start (0.00s) --- PASS: TestParseAndValidateRangeRejectsMalformedNumericOffsets/invalid_end (0.00s) --- PASS: TestParseAndValidateRangeRejectsMalformedNumericOffsets/invalid_open_ended_start (0.00s) === RUN TestValidateRetention === RUN TestValidateRetention/Valid_GOVERNANCE_retention === RUN TestValidateRetention/Valid_COMPLIANCE_retention === RUN TestValidateRetention/Missing_Mode === RUN TestValidateRetention/Missing_RetainUntilDate === RUN TestValidateRetention/Invalid_Mode === RUN TestValidateRetention/Past_RetainUntilDate === RUN TestValidateRetention/Empty_retention --- PASS: TestValidateRetention (0.00s) --- PASS: TestValidateRetention/Valid_GOVERNANCE_retention (0.00s) --- PASS: TestValidateRetention/Valid_COMPLIANCE_retention (0.00s) --- PASS: TestValidateRetention/Missing_Mode (0.00s) --- PASS: TestValidateRetention/Missing_RetainUntilDate (0.00s) --- PASS: TestValidateRetention/Invalid_Mode (0.00s) --- PASS: TestValidateRetention/Past_RetainUntilDate (0.00s) --- PASS: TestValidateRetention/Empty_retention (0.00s) === RUN TestValidateLegalHold === RUN TestValidateLegalHold/Valid_ON_status === RUN TestValidateLegalHold/Valid_OFF_status === RUN TestValidateLegalHold/Invalid_status === RUN TestValidateLegalHold/Empty_status === RUN TestValidateLegalHold/Lowercase_on === RUN TestValidateLegalHold/Lowercase_off --- PASS: TestValidateLegalHold (0.00s) --- PASS: TestValidateLegalHold/Valid_ON_status (0.00s) --- PASS: TestValidateLegalHold/Valid_OFF_status (0.00s) --- PASS: TestValidateLegalHold/Invalid_status (0.00s) --- PASS: TestValidateLegalHold/Empty_status (0.00s) --- PASS: TestValidateLegalHold/Lowercase_on (0.00s) --- PASS: TestValidateLegalHold/Lowercase_off (0.00s) === RUN TestParseObjectRetention === RUN TestParseObjectRetention/Valid_retention_XML === RUN TestParseObjectRetention/Valid_compliance_retention_XML === RUN TestParseObjectRetention/Valid_retention_XML_without_namespace_(Veeam_compatibility) === RUN TestParseObjectRetention/Valid_compliance_retention_XML_without_namespace_(Veeam_compatibility) === RUN TestParseObjectRetention/Empty_XML_body === RUN TestParseObjectRetention/Invalid_XML === RUN TestParseObjectRetention/Malformed_XML === RUN TestParseObjectRetention/Missing_Mode === RUN TestParseObjectRetention/Missing_RetainUntilDate --- PASS: TestParseObjectRetention (0.00s) --- PASS: TestParseObjectRetention/Valid_retention_XML (0.00s) --- PASS: TestParseObjectRetention/Valid_compliance_retention_XML (0.00s) --- PASS: TestParseObjectRetention/Valid_retention_XML_without_namespace_(Veeam_compatibility) (0.00s) --- PASS: TestParseObjectRetention/Valid_compliance_retention_XML_without_namespace_(Veeam_compatibility) (0.00s) --- PASS: TestParseObjectRetention/Empty_XML_body (0.00s) --- PASS: TestParseObjectRetention/Invalid_XML (0.00s) --- PASS: TestParseObjectRetention/Malformed_XML (0.00s) --- PASS: TestParseObjectRetention/Missing_Mode (0.00s) --- PASS: TestParseObjectRetention/Missing_RetainUntilDate (0.00s) === RUN TestParseObjectLegalHold === RUN TestParseObjectLegalHold/Valid_legal_hold_ON === RUN TestParseObjectLegalHold/Valid_legal_hold_OFF === RUN TestParseObjectLegalHold/Valid_legal_hold_ON_without_namespace === RUN TestParseObjectLegalHold/Valid_legal_hold_OFF_without_namespace === RUN TestParseObjectLegalHold/Empty_XML_body === RUN TestParseObjectLegalHold/Invalid_XML === RUN TestParseObjectLegalHold/Missing_Status --- PASS: TestParseObjectLegalHold (0.00s) --- PASS: TestParseObjectLegalHold/Valid_legal_hold_ON (0.00s) --- PASS: TestParseObjectLegalHold/Valid_legal_hold_OFF (0.00s) --- PASS: TestParseObjectLegalHold/Valid_legal_hold_ON_without_namespace (0.00s) --- PASS: TestParseObjectLegalHold/Valid_legal_hold_OFF_without_namespace (0.00s) --- PASS: TestParseObjectLegalHold/Empty_XML_body (0.00s) --- PASS: TestParseObjectLegalHold/Invalid_XML (0.00s) --- PASS: TestParseObjectLegalHold/Missing_Status (0.00s) === RUN TestParseObjectLockConfiguration === RUN TestParseObjectLockConfiguration/Valid_object_lock_configuration === RUN TestParseObjectLockConfiguration/Valid_object_lock_configuration_with_rule === RUN TestParseObjectLockConfiguration/Valid_object_lock_configuration_without_namespace === RUN TestParseObjectLockConfiguration/Valid_object_lock_configuration_with_rule_without_namespace === RUN TestParseObjectLockConfiguration/Empty_XML_body === RUN TestParseObjectLockConfiguration/Invalid_XML --- PASS: TestParseObjectLockConfiguration (0.00s) --- PASS: TestParseObjectLockConfiguration/Valid_object_lock_configuration (0.00s) --- PASS: TestParseObjectLockConfiguration/Valid_object_lock_configuration_with_rule (0.00s) --- PASS: TestParseObjectLockConfiguration/Valid_object_lock_configuration_without_namespace (0.00s) --- PASS: TestParseObjectLockConfiguration/Valid_object_lock_configuration_with_rule_without_namespace (0.00s) --- PASS: TestParseObjectLockConfiguration/Empty_XML_body (0.00s) --- PASS: TestParseObjectLockConfiguration/Invalid_XML (0.00s) === RUN TestValidateObjectLockConfiguration === RUN TestValidateObjectLockConfiguration/Valid_config_with_ObjectLockEnabled_only === RUN TestValidateObjectLockConfiguration/Missing_ObjectLockEnabled === RUN TestValidateObjectLockConfiguration/Valid_config_with_rule_and_days === RUN TestValidateObjectLockConfiguration/Valid_config_with_rule_and_years === RUN TestValidateObjectLockConfiguration/Invalid_ObjectLockEnabled_value === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_missing_mode === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_both_days_and_years === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_neither_days_nor_years === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_invalid_mode === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_days_out_of_range === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_years_out_of_range === RUN TestValidateObjectLockConfiguration/Invalid_rule_-_missing_DefaultRetention --- PASS: TestValidateObjectLockConfiguration (0.00s) --- PASS: TestValidateObjectLockConfiguration/Valid_config_with_ObjectLockEnabled_only (0.00s) --- PASS: TestValidateObjectLockConfiguration/Missing_ObjectLockEnabled (0.00s) --- PASS: TestValidateObjectLockConfiguration/Valid_config_with_rule_and_days (0.00s) --- PASS: TestValidateObjectLockConfiguration/Valid_config_with_rule_and_years (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_ObjectLockEnabled_value (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_missing_mode (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_both_days_and_years (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_neither_days_nor_years (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_invalid_mode (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_days_out_of_range (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_years_out_of_range (0.00s) --- PASS: TestValidateObjectLockConfiguration/Invalid_rule_-_missing_DefaultRetention (0.00s) === RUN TestValidateDefaultRetention === RUN TestValidateDefaultRetention/Valid_retention_with_days === RUN TestValidateDefaultRetention/Valid_retention_with_years === RUN TestValidateDefaultRetention/Missing_mode === RUN TestValidateDefaultRetention/Invalid_mode === RUN TestValidateDefaultRetention/Both_days_and_years_specified === RUN TestValidateDefaultRetention/Neither_days_nor_years_specified === RUN TestValidateDefaultRetention/Zero_years --- PASS: TestValidateDefaultRetention (0.00s) --- PASS: TestValidateDefaultRetention/Valid_retention_with_days (0.00s) --- PASS: TestValidateDefaultRetention/Valid_retention_with_years (0.00s) --- PASS: TestValidateDefaultRetention/Missing_mode (0.00s) --- PASS: TestValidateDefaultRetention/Invalid_mode (0.00s) --- PASS: TestValidateDefaultRetention/Both_days_and_years_specified (0.00s) --- PASS: TestValidateDefaultRetention/Neither_days_nor_years_specified (0.00s) --- PASS: TestValidateDefaultRetention/Zero_years (0.00s) === RUN TestStoreObjectLockConfigurationClearsStaleYears --- PASS: TestStoreObjectLockConfigurationClearsStaleYears (0.00s) === RUN TestOwnerRecentlyUnreachable --- PASS: TestOwnerRecentlyUnreachable (0.00s) === RUN TestBuildWriteCondition === RUN TestBuildWriteCondition/no_headers_is_unconditional === RUN TestBuildWriteCondition/If-None-Match_*_to_IF_NOT_EXISTS === RUN TestBuildWriteCondition/If-Match_*_to_IF_EXISTS === RUN TestBuildWriteCondition/If-Match_strong_etag_to_IF_ETAG_MATCH === RUN TestBuildWriteCondition/If-None-Match_strong_etag_to_IF_ETAG_NOT_MATCH === RUN TestBuildWriteCondition/weak_etag_falls_back === RUN TestBuildWriteCondition/etag_list_falls_back === RUN TestBuildWriteCondition/both_match_and_none-match_falls_back === RUN TestBuildWriteCondition/time-based_falls_back --- PASS: TestBuildWriteCondition (0.00s) --- PASS: TestBuildWriteCondition/no_headers_is_unconditional (0.00s) --- PASS: TestBuildWriteCondition/If-None-Match_*_to_IF_NOT_EXISTS (0.00s) --- PASS: TestBuildWriteCondition/If-Match_*_to_IF_EXISTS (0.00s) --- PASS: TestBuildWriteCondition/If-Match_strong_etag_to_IF_ETAG_MATCH (0.00s) --- PASS: TestBuildWriteCondition/If-None-Match_strong_etag_to_IF_ETAG_NOT_MATCH (0.00s) --- PASS: TestBuildWriteCondition/weak_etag_falls_back (0.00s) --- PASS: TestBuildWriteCondition/etag_list_falls_back (0.00s) --- PASS: TestBuildWriteCondition/both_match_and_none-match_falls_back (0.00s) --- PASS: TestBuildWriteCondition/time-based_falls_back (0.00s) === RUN TestParseConditionalHeadersAcceptsHTTPDateFormats === RUN TestParseConditionalHeadersAcceptsHTTPDateFormats/If-Modified-Since_RFC850 === RUN TestParseConditionalHeadersAcceptsHTTPDateFormats/If-Unmodified-Since_ANSIC === RUN TestParseConditionalHeadersAcceptsHTTPDateFormats/If-Modified-Since_RFC1123_UTC_zone --- PASS: TestParseConditionalHeadersAcceptsHTTPDateFormats (0.00s) --- PASS: TestParseConditionalHeadersAcceptsHTTPDateFormats/If-Modified-Since_RFC850 (0.00s) --- PASS: TestParseConditionalHeadersAcceptsHTTPDateFormats/If-Unmodified-Since_ANSIC (0.00s) --- PASS: TestParseConditionalHeadersAcceptsHTTPDateFormats/If-Modified-Since_RFC1123_UTC_zone (0.00s) === RUN TestValidateConditionalCopyHeadersAcceptsHTTPDateFormats === RUN TestValidateConditionalCopyHeadersAcceptsHTTPDateFormats/X-Amz-Copy-Source-If-Modified-Since_RFC850 === RUN TestValidateConditionalCopyHeadersAcceptsHTTPDateFormats/X-Amz-Copy-Source-If-Unmodified-Since_ANSIC --- PASS: TestValidateConditionalCopyHeadersAcceptsHTTPDateFormats (0.00s) --- PASS: TestValidateConditionalCopyHeadersAcceptsHTTPDateFormats/X-Amz-Copy-Source-If-Modified-Since_RFC850 (0.00s) --- PASS: TestValidateConditionalCopyHeadersAcceptsHTTPDateFormats/X-Amz-Copy-Source-If-Unmodified-Since_ANSIC (0.00s) === RUN TestBuildDeleteCondition === RUN TestBuildDeleteCondition/no_If-Match_is_unconditional === RUN TestBuildDeleteCondition/If-Match_*_to_IF_EXISTS === RUN TestBuildDeleteCondition/If-Match_etag_to_IF_ETAG_MATCH === RUN TestBuildDeleteCondition/weak_etag_falls_back --- PASS: TestBuildDeleteCondition (0.00s) --- PASS: TestBuildDeleteCondition/no_If-Match_is_unconditional (0.00s) --- PASS: TestBuildDeleteCondition/If-Match_*_to_IF_EXISTS (0.00s) --- PASS: TestBuildDeleteCondition/If-Match_etag_to_IF_ETAG_MATCH (0.00s) --- PASS: TestBuildDeleteCondition/weak_etag_falls_back (0.00s) === RUN TestSingleStrongETag --- PASS: TestSingleStrongETag (0.00s) === RUN TestObjectTxnFailsOverStaleOwner --- PASS: TestObjectTxnFailsOverStaleOwner (0.00s) === RUN TestRouteWriteCondition --- PASS: TestRouteWriteCondition (0.00s) === RUN TestSelectLatestVersion_MixedFormats --- PASS: TestSelectLatestVersion_MixedFormats (0.00s) === RUN TestSelectLatestVersion_PromotesNewestDeleteMarker --- PASS: TestSelectLatestVersion_PromotesNewestDeleteMarker (0.00s) === RUN TestSelectLatestVersion_ContentWinsWhenNewer --- PASS: TestSelectLatestVersion_ContentWinsWhenNewer (0.00s) === RUN TestSelectLatestVersion_OnlyDeleteMarkers --- PASS: TestSelectLatestVersion_OnlyDeleteMarkers (0.00s) === RUN TestSelectLatestVersion_EmptyOrUntagged --- PASS: TestSelectLatestVersion_EmptyOrUntagged (0.00s) === RUN TestVersionIdFromEntry --- PASS: TestVersionIdFromEntry (0.00s) === RUN TestSelectLatestVersion_FilenameFallback --- PASS: TestSelectLatestVersion_FilenameFallback (0.00s) === RUN TestSelectLatestVersion_FilenameFallbackMixedWithTagged --- PASS: TestSelectLatestVersion_FilenameFallbackMixedWithTagged (0.00s) === RUN TestSelectLatestVersion_FilenameFallbackDeleteMarker --- PASS: TestSelectLatestVersion_FilenameFallbackDeleteMarker (0.00s) === RUN TestValidateRequestPath_RejectsTraversal === RUN TestValidateRequestPath_RejectsTraversal/clean_path_passes === RUN TestValidateRequestPath_RejectsTraversal/bucket_only_passes === RUN TestValidateRequestPath_RejectsTraversal/trailing_slash_passes === RUN TestValidateRequestPath_RejectsTraversal/leading_dotdot_rejected === RUN TestValidateRequestPath_RejectsTraversal/nested_dotdot_rejected === RUN TestValidateRequestPath_RejectsTraversal/backslash_dotdot_rejected === RUN TestValidateRequestPath_RejectsTraversal/percent-encoded_dotdot_rejected === RUN TestValidateRequestPath_RejectsTraversal/bare_dot_object_rejected === RUN TestValidateRequestPath_RejectsTraversal/dotdot_bucket_rejected --- PASS: TestValidateRequestPath_RejectsTraversal (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/clean_path_passes (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/bucket_only_passes (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/trailing_slash_passes (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/leading_dotdot_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/nested_dotdot_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/backslash_dotdot_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/percent-encoded_dotdot_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/bare_dot_object_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/dotdot_bucket_rejected (0.00s) === RUN TestValidateRequestPath_RejectsEmptyCapturedVars === RUN TestValidateRequestPath_RejectsEmptyCapturedVars/empty_bucket === RUN TestValidateRequestPath_RejectsEmptyCapturedVars/empty_object --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars (0.00s) --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars/empty_bucket (0.00s) --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars/empty_object (0.00s) === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/clean_version_ID === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/clean_upload_ID === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/empty_values === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/version_ID_encoded_slash === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/version_ID_backslash === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/upload_ID_traversal === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/unsafe_repeated_value === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/encoded_version_ID_name === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/encoded_upload_ID_name === RUN TestValidateRequestPath_RejectsUnsafePathQueryValues/unrelated_query --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/clean_version_ID (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/clean_upload_ID (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/empty_values (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/version_ID_encoded_slash (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/version_ID_backslash (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/upload_ID_traversal (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/unsafe_repeated_value (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/encoded_version_ID_name (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/encoded_upload_ID_name (0.00s) --- PASS: TestValidateRequestPath_RejectsUnsafePathQueryValues/unrelated_query (0.00s) === RUN TestHasPathSegmentQuery_CommonPathDoesNotAllocate --- PASS: TestHasPathSegmentQuery_CommonPathDoesNotAllocate (0.00s) === RUN TestGetRequestDataReader_ChunkedEncodingWithoutIAM I0720 21:30:04.700981 config_loader.go:73 Using explicit credential store: memory === RUN TestGetRequestDataReader_ChunkedEncodingWithoutIAM/RegularRequest s3api_put_object_helper_test.go:82: Test case: RegularRequest - Regular requests without chunked encoding should pass through unchanged === RUN TestGetRequestDataReader_ChunkedEncodingWithoutIAM/StreamingSignedWithoutIAM s3api_put_object_helper_test.go:82: Test case: StreamingSignedWithoutIAM - Streaming signed requests should fail when IAM is disabled === RUN TestGetRequestDataReader_ChunkedEncodingWithoutIAM/StreamingUnsignedWithoutIAM s3api_put_object_helper_test.go:82: Test case: StreamingUnsignedWithoutIAM - Streaming unsigned requests should be processed even when IAM is disabled --- PASS: TestGetRequestDataReader_ChunkedEncodingWithoutIAM (0.00s) --- PASS: TestGetRequestDataReader_ChunkedEncodingWithoutIAM/RegularRequest (0.00s) --- PASS: TestGetRequestDataReader_ChunkedEncodingWithoutIAM/StreamingSignedWithoutIAM (0.00s) --- PASS: TestGetRequestDataReader_ChunkedEncodingWithoutIAM/StreamingUnsignedWithoutIAM (0.00s) === RUN TestGetRequestDataReader_AuthTypeDetection I0720 21:30:04.701117 config_loader.go:73 Using explicit credential store: memory === RUN TestGetRequestDataReader_AuthTypeDetection/ChunkedDataWithChecksum --- PASS: TestGetRequestDataReader_AuthTypeDetection (0.00s) --- PASS: TestGetRequestDataReader_AuthTypeDetection/ChunkedDataWithChecksum (0.00s) === RUN TestGetRequestDataReader_IAMEnabled I0720 21:30:04.701161 config_loader.go:73 Using explicit credential store: memory === RUN TestGetRequestDataReader_IAMEnabled/StreamingUnsignedWithIAMEnabled --- PASS: TestGetRequestDataReader_IAMEnabled (0.00s) --- PASS: TestGetRequestDataReader_IAMEnabled/StreamingUnsignedWithIAMEnabled (0.00s) === RUN TestAuthTypeDetection === RUN TestAuthTypeDetection/StreamingUnsigned === RUN TestAuthTypeDetection/StreamingSigned === RUN TestAuthTypeDetection/Regular --- PASS: TestAuthTypeDetection (0.00s) --- PASS: TestAuthTypeDetection/StreamingUnsigned (0.00s) --- PASS: TestAuthTypeDetection/StreamingSigned (0.00s) --- PASS: TestAuthTypeDetection/Regular (0.00s) === RUN TestIsInRemoteOnly === RUN TestIsInRemoteOnly/remote-only_entry_with_no_chunks === RUN TestIsInRemoteOnly/remote_entry_with_chunks_(cached) === RUN TestIsInRemoteOnly/local_file_with_chunks_(not_remote) === RUN TestIsInRemoteOnly/empty_remote_entry_(size_0) === RUN TestIsInRemoteOnly/no_chunks_but_nil_RemoteEntry --- PASS: TestIsInRemoteOnly (0.00s) --- PASS: TestIsInRemoteOnly/remote-only_entry_with_no_chunks (0.00s) --- PASS: TestIsInRemoteOnly/remote_entry_with_chunks_(cached) (0.00s) --- PASS: TestIsInRemoteOnly/local_file_with_chunks_(not_remote) (0.00s) --- PASS: TestIsInRemoteOnly/empty_remote_entry_(size_0) (0.00s) --- PASS: TestIsInRemoteOnly/no_chunks_but_nil_RemoteEntry (0.00s) === RUN TestRemoteOnlyEntryDetection === RUN TestRemoteOnlyEntryDetection/remote-only_entry_(no_chunks,_has_remote_entry) === RUN TestRemoteOnlyEntryDetection/data_integrity_error_(no_chunks,_no_remote,_has_size) === RUN TestRemoteOnlyEntryDetection/empty_local_file_(no_chunks,_no_remote,_size_0) === RUN TestRemoteOnlyEntryDetection/normal_file_with_chunks --- PASS: TestRemoteOnlyEntryDetection (0.00s) --- PASS: TestRemoteOnlyEntryDetection/remote-only_entry_(no_chunks,_has_remote_entry) (0.00s) --- PASS: TestRemoteOnlyEntryDetection/data_integrity_error_(no_chunks,_no_remote,_has_size) (0.00s) --- PASS: TestRemoteOnlyEntryDetection/empty_local_file_(no_chunks,_no_remote,_size_0) (0.00s) --- PASS: TestRemoteOnlyEntryDetection/normal_file_with_chunks (0.00s) === RUN TestVersionedRemoteObjectPathBuilding === RUN TestVersionedRemoteObjectPathBuilding/non-versioned_object_(empty_versionId) === RUN TestVersionedRemoteObjectPathBuilding/null_version === RUN TestVersionedRemoteObjectPathBuilding/specific_version === RUN TestVersionedRemoteObjectPathBuilding/nested_object_with_version === RUN TestVersionedRemoteObjectPathBuilding/object_with_leading_slash_and_version --- PASS: TestVersionedRemoteObjectPathBuilding (0.00s) --- PASS: TestVersionedRemoteObjectPathBuilding/non-versioned_object_(empty_versionId) (0.00s) --- PASS: TestVersionedRemoteObjectPathBuilding/null_version (0.00s) --- PASS: TestVersionedRemoteObjectPathBuilding/specific_version (0.00s) --- PASS: TestVersionedRemoteObjectPathBuilding/nested_object_with_version (0.00s) --- PASS: TestVersionedRemoteObjectPathBuilding/object_with_leading_slash_and_version (0.00s) === RUN TestResolvedSourceVersionId === RUN TestResolvedSourceVersionId/explicit_request_versionId_wins === RUN TestResolvedSourceVersionId/empty_request_falls_back_to_entry_version_(latest_in_versioned_bucket) === RUN TestResolvedSourceVersionId/empty_request_and_pre-versioning_entry_stays_empty === RUN TestResolvedSourceVersionId/empty_request_and_nil_Extended_stays_empty === RUN TestResolvedSourceVersionId/nil_entry_tolerated_when_no_request_version --- PASS: TestResolvedSourceVersionId (0.00s) --- PASS: TestResolvedSourceVersionId/explicit_request_versionId_wins (0.00s) --- PASS: TestResolvedSourceVersionId/empty_request_falls_back_to_entry_version_(latest_in_versioned_bucket) (0.00s) --- PASS: TestResolvedSourceVersionId/empty_request_and_pre-versioning_entry_stays_empty (0.00s) --- PASS: TestResolvedSourceVersionId/empty_request_and_nil_Extended_stays_empty (0.00s) --- PASS: TestResolvedSourceVersionId/nil_entry_tolerated_when_no_request_version (0.00s) === RUN TestCachedEntryHasLocalData === RUN TestCachedEntryHasLocalData/nil_entry_is_not_a_hit === RUN TestCachedEntryHasLocalData/entry_with_chunks_is_a_hit === RUN TestCachedEntryHasLocalData/entry_with_inline_content_is_a_hit === RUN TestCachedEntryHasLocalData/entry_with_neither_chunks_nor_content_is_not_a_hit --- PASS: TestCachedEntryHasLocalData (0.00s) --- PASS: TestCachedEntryHasLocalData/nil_entry_is_not_a_hit (0.00s) --- PASS: TestCachedEntryHasLocalData/entry_with_chunks_is_a_hit (0.00s) --- PASS: TestCachedEntryHasLocalData/entry_with_inline_content_is_a_hit (0.00s) --- PASS: TestCachedEntryHasLocalData/entry_with_neither_chunks_nor_content_is_not_a_hit (0.00s) === RUN TestCopyObjectRemoteOnlySourceDetection === RUN TestCopyObjectRemoteOnlySourceDetection/remote-only_object_with_size_and_no_chunks/content === RUN TestCopyObjectRemoteOnlySourceDetection/local_file_with_chunks_-_copy_works_fine,_fix_does_not_engage === RUN TestCopyObjectRemoteOnlySourceDetection/small_inline_file_(no_chunks,_has_Content)_-_hits_inline_branch_but_not_broken === RUN TestCopyObjectRemoteOnlySourceDetection/remote_entry_already_cached_(has_chunks)_-_fix_does_not_engage === RUN TestCopyObjectRemoteOnlySourceDetection/zero-byte_remote_object_-_fix_does_not_engage,_inline_branch_handles_it --- PASS: TestCopyObjectRemoteOnlySourceDetection (0.00s) --- PASS: TestCopyObjectRemoteOnlySourceDetection/remote-only_object_with_size_and_no_chunks/content (0.00s) --- PASS: TestCopyObjectRemoteOnlySourceDetection/local_file_with_chunks_-_copy_works_fine,_fix_does_not_engage (0.00s) --- PASS: TestCopyObjectRemoteOnlySourceDetection/small_inline_file_(no_chunks,_has_Content)_-_hits_inline_branch_but_not_broken (0.00s) --- PASS: TestCopyObjectRemoteOnlySourceDetection/remote_entry_already_cached_(has_chunks)_-_fix_does_not_engage (0.00s) --- PASS: TestCopyObjectRemoteOnlySourceDetection/zero-byte_remote_object_-_fix_does_not_engage,_inline_branch_handles_it (0.00s) === RUN TestCacheRemoteObjectForStreamingTimeout --- PASS: TestCacheRemoteObjectForStreamingTimeout (0.20s) === RUN TestCacheRemoteObjectForStreamingCached --- PASS: TestCacheRemoteObjectForStreamingCached (0.00s) === RUN TestRouting_STSWithQueryParams I0720 21:30:04.905320 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_STSWithQueryParams (0.00s) === RUN TestRouting_STSWithBodyParams I0720 21:30:04.909515 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_STSWithBodyParams (0.00s) === RUN TestRouting_GetFederationTokenWithQueryParams I0720 21:30:04.911217 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_GetFederationTokenWithQueryParams (0.00s) === RUN TestRouting_GetFederationTokenAuthenticatedBody I0720 21:30:04.912789 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_GetFederationTokenAuthenticatedBody (0.00s) === RUN TestRouting_AuthenticatedIAM I0720 21:30:04.914735 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_AuthenticatedIAM (0.00s) === RUN TestRouting_IAMMatcherLogic === RUN TestRouting_IAMMatcherLogic/No_auth_-_anonymous I0720 21:30:04.916539 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_IAMMatcherLogic/AWS4_signature I0720 21:30:04.917801 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_IAMMatcherLogic/AWS2_signature I0720 21:30:04.920524 config_loader.go:73 Using explicit credential store: memory W0720 21:30:04.922981 auth_signature_v2.go:137 InvalidAccessKeyId (V2 signed): attempted key 'AKIA...' not found. Available keys: 1, Auth enabled: true === RUN TestRouting_IAMMatcherLogic/Presigned_V4 I0720 21:30:04.923036 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_IAMMatcherLogic/Presigned_V2 I0720 21:30:04.924291 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_IAMMatcherLogic/AWS4_signature_with_STS_action_in_body I0720 21:30:04.925699 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_IAMMatcherLogic (0.01s) --- PASS: TestRouting_IAMMatcherLogic/No_auth_-_anonymous (0.00s) --- PASS: TestRouting_IAMMatcherLogic/AWS4_signature (0.00s) --- PASS: TestRouting_IAMMatcherLogic/AWS2_signature (0.00s) --- PASS: TestRouting_IAMMatcherLogic/Presigned_V4 (0.00s) --- PASS: TestRouting_IAMMatcherLogic/Presigned_V2 (0.00s) --- PASS: TestRouting_IAMMatcherLogic/AWS4_signature_with_STS_action_in_body (0.00s) === RUN TestRouting_BucketRouteMatchesAnyHost === RUN TestRouting_BucketRouteMatchesAnyHost/virtual-host_domain_configured,_request_via_IP I0720 21:30:04.926962 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_BucketRouteMatchesAnyHost/virtual-host_domain_configured,_request_via_unrelated_host I0720 21:30:04.928883 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_BucketRouteMatchesAnyHost/path-style_+_virtual-host_configured,_request_via_IP I0720 21:30:04.933170 config_loader.go:73 Using explicit credential store: memory === RUN TestRouting_BucketRouteMatchesAnyHost/domain_configured,_request_via_the_bare_configured_host_(no_bucket_subdomain) I0720 21:30:04.935632 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_BucketRouteMatchesAnyHost (0.01s) --- PASS: TestRouting_BucketRouteMatchesAnyHost/virtual-host_domain_configured,_request_via_IP (0.00s) --- PASS: TestRouting_BucketRouteMatchesAnyHost/virtual-host_domain_configured,_request_via_unrelated_host (0.00s) --- PASS: TestRouting_BucketRouteMatchesAnyHost/path-style_+_virtual-host_configured,_request_via_IP (0.00s) --- PASS: TestRouting_BucketRouteMatchesAnyHost/domain_configured,_request_via_the_bare_configured_host_(no_bucket_subdomain) (0.00s) === RUN TestSSECDecryptChunkView_NoOffsetAdjustment s3api_sse_decrypt_test.go:63: ✓ Correct decryption (using stored IV directly) successful s3api_sse_decrypt_test.go:91: ✓ Verified: Offset-adjusted IV produces corrupted data (as expected for SSE-C) s3api_sse_decrypt_test.go:96: Plaintext: "This is a test messa" s3api_sse_decrypt_test.go:101: Corrupted: ".\x04ʼ\xdc\xe6O\x03\xc7SO\x1b~\xb2\x02Y\rL\x9c\x01" --- PASS: TestSSECDecryptChunkView_NoOffsetAdjustment (0.00s) === RUN TestSSEKMSDecryptChunkView_RequiresOffsetAdjustment s3api_sse_decrypt_test.go:161: ✓ SSE-KMS decryption with offset adjustment successful s3api_sse_decrypt_test.go:182: ✓ Verified: double-adjusted IV produces corrupted data (range path must not pre-adjust) s3api_sse_decrypt_test.go:199: ✓ Verified: Base IV without offset produces corrupted data (as expected for SSE-KMS) --- PASS: TestSSEKMSDecryptChunkView_RequiresOffsetAdjustment (0.00s) === RUN TestSSEDecryptionDifferences s3api_sse_decrypt_test.go:205: SSE-C: Random IV per part → Use stored IV DIRECTLY (no offset) s3api_sse_decrypt_test.go:206: SSE-KMS: Base IV + offset → MUST call calculateIVWithOffset(baseIV, offset) s3api_sse_decrypt_test.go:207: SSE-S3: Base IV + offset → Stores ADJUSTED IV, use directly --- PASS: TestSSEDecryptionDifferences (0.00s) === RUN TestSSES3MultipartUploadStoresDerivedIV === RUN TestSSES3MultipartUploadStoresDerivedIV/Part_1_at_offset_0 s3api_sse_s3_upload_test.go:103: ✓ Derived IV decryption successful for offset 0 === RUN TestSSES3MultipartUploadStoresDerivedIV/Part_2_at_offset_1MB s3api_sse_s3_upload_test.go:103: ✓ Derived IV decryption successful for offset 1048576 s3api_sse_s3_upload_test.go:127: ✓ Verified: Base IV produces corrupted data at offset 1048576 (bug would cause this) === RUN TestSSES3MultipartUploadStoresDerivedIV/Part_3_at_offset_5MB s3api_sse_s3_upload_test.go:103: ✓ Derived IV decryption successful for offset 5242880 s3api_sse_s3_upload_test.go:127: ✓ Verified: Base IV produces corrupted data at offset 5242880 (bug would cause this) --- PASS: TestSSES3MultipartUploadStoresDerivedIV (0.00s) --- PASS: TestSSES3MultipartUploadStoresDerivedIV/Part_1_at_offset_0 (0.00s) --- PASS: TestSSES3MultipartUploadStoresDerivedIV/Part_2_at_offset_1MB (0.00s) --- PASS: TestSSES3MultipartUploadStoresDerivedIV/Part_3_at_offset_5MB (0.00s) === RUN TestHandleSSES3MultipartEncryptionFlow s3api_sse_s3_upload_test.go:203: ✓ Key.IV correctly updated with derived IV s3api_sse_s3_upload_test.go:225: ✓ Full encrypt-update_key-decrypt cycle successful --- PASS: TestHandleSSES3MultipartEncryptionFlow (0.00s) === RUN TestSSES3HeaderEncoding --- PASS: TestSSES3HeaderEncoding (0.00s) === RUN TestShouldWriteStreamingErrorResponse === RUN TestShouldWriteStreamingErrorResponse/nil_error === RUN TestShouldWriteStreamingErrorResponse/context_canceled === RUN TestShouldWriteStreamingErrorResponse/wrapped_context_canceled === RUN TestShouldWriteStreamingErrorResponse/grpc_canceled === RUN TestShouldWriteStreamingErrorResponse/wrapped_grpc_canceled === RUN TestShouldWriteStreamingErrorResponse/deadline_exceeded === RUN TestShouldWriteStreamingErrorResponse/wrapped_deadline_exceeded --- PASS: TestShouldWriteStreamingErrorResponse (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/nil_error (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/context_canceled (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/wrapped_context_canceled (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/grpc_canceled (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/wrapped_grpc_canceled (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/deadline_exceeded (0.00s) --- PASS: TestShouldWriteStreamingErrorResponse/wrapped_deadline_exceeded (0.00s) === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/trust_policy_admits_the_caller === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/trust_policy_admits_a_specific_principal === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/trust_policy_rejects_the_caller === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/identity_policy_explicit_deny_wins_over_trust_policy === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/session_policy_explicit_deny_blocks_role_chaining === RUN TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/revoked_chained_session_cannot_assume W0720 21:30:04.939040 auth_credentials.go:2356 AssumeRole explicit-deny check failed for a7323e89b3ef004ad0c211299b067591, denying: session has been revoked --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy (0.00s) --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/trust_policy_admits_the_caller (0.00s) --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/trust_policy_admits_a_specific_principal (0.00s) --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/trust_policy_rejects_the_caller (0.00s) --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/identity_policy_explicit_deny_wins_over_trust_policy (0.00s) --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/session_policy_explicit_deny_blocks_role_chaining (0.00s) --- PASS: TestAssumeRole_NonAdminCallerAuthorizedByTrustPolicy/revoked_chained_session_cannot_assume (0.00s) === RUN TestCallerPrincipalArn --- PASS: TestCallerPrincipalArn (0.00s) === RUN TestAssumeRoleWithWebIdentity_DispatchesThroughIAMManager === RUN TestAssumeRoleWithWebIdentity_DispatchesThroughIAMManager/with_IAMManager_wired === RUN TestAssumeRoleWithWebIdentity_DispatchesThroughIAMManager/without_IAM_integration --- PASS: TestAssumeRoleWithWebIdentity_DispatchesThroughIAMManager (0.00s) --- PASS: TestAssumeRoleWithWebIdentity_DispatchesThroughIAMManager/with_IAMManager_wired (0.00s) --- PASS: TestAssumeRoleWithWebIdentity_DispatchesThroughIAMManager/without_IAM_integration (0.00s) === RUN TestAssumeRoleWithWebIdentity_AllowsEmptyRoleArn --- PASS: TestAssumeRoleWithWebIdentity_AllowsEmptyRoleArn (0.00s) === RUN TestAssumeRole_CallerIdentityFallback === RUN TestAssumeRole_CallerIdentityFallback/Caller_is_IAM_User,_No_RoleArn === RUN TestAssumeRole_CallerIdentityFallback/Caller_is_STS_Assumed_Role,_No_RoleArn === RUN TestAssumeRole_CallerIdentityFallback/Explicit_RoleArn_Provided === RUN TestAssumeRole_CallerIdentityFallback/Malformed_ARN --- PASS: TestAssumeRole_CallerIdentityFallback (0.00s) --- PASS: TestAssumeRole_CallerIdentityFallback/Caller_is_IAM_User,_No_RoleArn (0.00s) --- PASS: TestAssumeRole_CallerIdentityFallback/Caller_is_STS_Assumed_Role,_No_RoleArn (0.00s) --- PASS: TestAssumeRole_CallerIdentityFallback/Explicit_RoleArn_Provided (0.00s) --- PASS: TestAssumeRole_CallerIdentityFallback/Malformed_ARN (0.00s) === RUN TestAssumeRole_EmbedsRolePolicies === RUN TestAssumeRole_EmbedsRolePolicies/RoleWithAttachedPolicies === RUN TestAssumeRole_EmbedsRolePolicies/RoleWithoutAttachedPolicies --- PASS: TestAssumeRole_EmbedsRolePolicies (0.00s) --- PASS: TestAssumeRole_EmbedsRolePolicies/RoleWithAttachedPolicies (0.00s) --- PASS: TestAssumeRole_EmbedsRolePolicies/RoleWithoutAttachedPolicies (0.00s) === RUN TestGetCallerIdentityResponse_XMLMarshal --- PASS: TestGetCallerIdentityResponse_XMLMarshal (0.00s) === RUN TestGetFederationToken_BasicFlow --- PASS: TestGetFederationToken_BasicFlow (0.00s) === RUN TestGetFederationToken_WithSessionPolicy --- PASS: TestGetFederationToken_WithSessionPolicy (0.00s) === RUN TestGetFederationToken_RejectTemporaryCredentials === RUN TestGetFederationToken_RejectTemporaryCredentials/SessionTokenInHeader === RUN TestGetFederationToken_RejectTemporaryCredentials/SessionTokenInQuery --- PASS: TestGetFederationToken_RejectTemporaryCredentials (0.00s) --- PASS: TestGetFederationToken_RejectTemporaryCredentials/SessionTokenInHeader (0.00s) --- PASS: TestGetFederationToken_RejectTemporaryCredentials/SessionTokenInQuery (0.00s) === RUN TestGetFederationToken_MissingName --- PASS: TestGetFederationToken_MissingName (0.00s) === RUN TestGetFederationToken_NameValidation === RUN TestGetFederationToken_NameValidation/TooShort === RUN TestGetFederationToken_NameValidation/TooLong === RUN TestGetFederationToken_NameValidation/MinLength === RUN TestGetFederationToken_NameValidation/MaxLength === RUN TestGetFederationToken_NameValidation/ValidSpecialChars === RUN TestGetFederationToken_NameValidation/InvalidChars_Space === RUN TestGetFederationToken_NameValidation/InvalidChars_Slash --- PASS: TestGetFederationToken_NameValidation (0.00s) --- PASS: TestGetFederationToken_NameValidation/TooShort (0.00s) --- PASS: TestGetFederationToken_NameValidation/TooLong (0.00s) --- PASS: TestGetFederationToken_NameValidation/MinLength (0.00s) --- PASS: TestGetFederationToken_NameValidation/MaxLength (0.00s) --- PASS: TestGetFederationToken_NameValidation/ValidSpecialChars (0.00s) --- PASS: TestGetFederationToken_NameValidation/InvalidChars_Space (0.00s) --- PASS: TestGetFederationToken_NameValidation/InvalidChars_Slash (0.00s) === RUN TestGetFederationToken_DurationValidation === RUN TestGetFederationToken_DurationValidation/BelowMinimum === RUN TestGetFederationToken_DurationValidation/AboveMaximum === RUN TestGetFederationToken_DurationValidation/InvalidFormat === RUN TestGetFederationToken_DurationValidation/MinimumValid === RUN TestGetFederationToken_DurationValidation/MaximumValid_36Hours === RUN TestGetFederationToken_DurationValidation/Default12Hours --- PASS: TestGetFederationToken_DurationValidation (0.00s) --- PASS: TestGetFederationToken_DurationValidation/BelowMinimum (0.00s) --- PASS: TestGetFederationToken_DurationValidation/AboveMaximum (0.00s) --- PASS: TestGetFederationToken_DurationValidation/InvalidFormat (0.00s) --- PASS: TestGetFederationToken_DurationValidation/MinimumValid (0.00s) --- PASS: TestGetFederationToken_DurationValidation/MaximumValid_36Hours (0.00s) --- PASS: TestGetFederationToken_DurationValidation/Default12Hours (0.00s) === RUN TestGetFederationToken_ResponseFormat --- PASS: TestGetFederationToken_ResponseFormat (0.00s) === RUN TestGetFederationToken_PolicyEmbedding --- PASS: TestGetFederationToken_PolicyEmbedding (0.00s) === RUN TestGetFederationToken_PolicyIntersection --- PASS: TestGetFederationToken_PolicyIntersection (0.00s) === RUN TestGetFederationToken_MalformedPolicy === RUN TestGetFederationToken_MalformedPolicy/InvalidJSON === RUN TestGetFederationToken_MalformedPolicy/EmptyObject === RUN TestGetFederationToken_MalformedPolicy/TooLarge === RUN TestGetFederationToken_MalformedPolicy/ValidPolicy === RUN TestGetFederationToken_MalformedPolicy/EmptyString --- PASS: TestGetFederationToken_MalformedPolicy (0.00s) --- PASS: TestGetFederationToken_MalformedPolicy/InvalidJSON (0.00s) --- PASS: TestGetFederationToken_MalformedPolicy/EmptyObject (0.00s) --- PASS: TestGetFederationToken_MalformedPolicy/TooLarge (0.00s) --- PASS: TestGetFederationToken_MalformedPolicy/ValidPolicy (0.00s) --- PASS: TestGetFederationToken_MalformedPolicy/EmptyString (0.00s) === RUN TestGetFederationToken_STSNotReady --- PASS: TestGetFederationToken_STSNotReady (0.00s) === RUN TestGetFederationToken_DefaultDuration --- PASS: TestGetFederationToken_DefaultDuration (0.00s) === RUN TestGetFederationToken_GetPoliciesForUser === RUN TestGetFederationToken_GetPoliciesForUser/NoUserStore === RUN TestGetFederationToken_GetPoliciesForUser/UserNotFound === RUN TestGetFederationToken_GetPoliciesForUser/UserWithPolicies === RUN TestGetFederationToken_GetPoliciesForUser/UserWithNoPolicies --- PASS: TestGetFederationToken_GetPoliciesForUser (0.00s) --- PASS: TestGetFederationToken_GetPoliciesForUser/NoUserStore (0.00s) --- PASS: TestGetFederationToken_GetPoliciesForUser/UserNotFound (0.00s) --- PASS: TestGetFederationToken_GetPoliciesForUser/UserWithPolicies (0.00s) --- PASS: TestGetFederationToken_GetPoliciesForUser/UserWithNoPolicies (0.00s) === RUN TestGetFederationToken_PolicyMergeAndDedup --- PASS: TestGetFederationToken_PolicyMergeAndDedup (0.00s) === RUN TestGetFederationToken_PolicyMergeNoManager --- PASS: TestGetFederationToken_PolicyMergeNoManager (0.00s) === RUN TestBuildListTablesRequestRejectsInvalidNamespaceQuery === RUN TestBuildListTablesRequestRejectsInvalidNamespaceQuery/uppercase === RUN TestBuildListTablesRequestRejectsInvalidNamespaceQuery/leading_hyphen === RUN TestBuildListTablesRequestRejectsInvalidNamespaceQuery/slash --- PASS: TestBuildListTablesRequestRejectsInvalidNamespaceQuery (0.00s) --- PASS: TestBuildListTablesRequestRejectsInvalidNamespaceQuery/uppercase (0.00s) --- PASS: TestBuildListTablesRequestRejectsInvalidNamespaceQuery/leading_hyphen (0.00s) --- PASS: TestBuildListTablesRequestRejectsInvalidNamespaceQuery/slash (0.00s) === RUN TestBuildGetTableRequestRejectsInvalidNamespaceQuery === RUN TestBuildGetTableRequestRejectsInvalidNamespaceQuery/uppercase === RUN TestBuildGetTableRequestRejectsInvalidNamespaceQuery/leading_hyphen === RUN TestBuildGetTableRequestRejectsInvalidNamespaceQuery/slash --- PASS: TestBuildGetTableRequestRejectsInvalidNamespaceQuery (0.00s) --- PASS: TestBuildGetTableRequestRejectsInvalidNamespaceQuery/uppercase (0.00s) --- PASS: TestBuildGetTableRequestRejectsInvalidNamespaceQuery/leading_hyphen (0.00s) --- PASS: TestBuildGetTableRequestRejectsInvalidNamespaceQuery/slash (0.00s) === RUN TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery === RUN TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery/uppercase === RUN TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery/leading_hyphen === RUN TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery/slash --- PASS: TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery (0.00s) --- PASS: TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery/uppercase (0.00s) --- PASS: TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery/leading_hyphen (0.00s) --- PASS: TestHandleRestOperationReturnsBadRequestForInvalidNamespaceQuery/slash (0.00s) === RUN TestIsS3TablesSignedRequest === RUN TestIsS3TablesSignedRequest/regular_S3_auth_header === RUN TestIsS3TablesSignedRequest/S3_Tables_auth_header === RUN TestIsS3TablesSignedRequest/S3_Tables_presigned_query === RUN TestIsS3TablesSignedRequest/regular_S3_presigned_query === RUN TestIsS3TablesSignedRequest/unsigned_request === RUN TestIsS3TablesSignedRequest/malformed_credential === RUN TestIsS3TablesSignedRequest/service_substring_in_access_key_must_not_match --- PASS: TestIsS3TablesSignedRequest (0.00s) --- PASS: TestIsS3TablesSignedRequest/regular_S3_auth_header (0.00s) --- PASS: TestIsS3TablesSignedRequest/S3_Tables_auth_header (0.00s) --- PASS: TestIsS3TablesSignedRequest/S3_Tables_presigned_query (0.00s) --- PASS: TestIsS3TablesSignedRequest/regular_S3_presigned_query (0.00s) --- PASS: TestIsS3TablesSignedRequest/unsigned_request (0.00s) --- PASS: TestIsS3TablesSignedRequest/malformed_credential (0.00s) --- PASS: TestIsS3TablesSignedRequest/service_substring_in_access_key_must_not_match (0.00s) === RUN TestRouting_ListObjectsV2OnBucketNamedBuckets I0720 21:30:04.940966 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_ListObjectsV2OnBucketNamedBuckets (0.00s) === RUN TestRouting_S3TablesListTableBucketsStillReachable I0720 21:30:04.944030 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_S3TablesListTableBucketsStillReachable (0.00s) === RUN TestRouting_CreateBucketNamedBuckets I0720 21:30:04.945383 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_CreateBucketNamedBuckets (0.00s) === RUN TestRouting_GetObjectOnBucketNamedGetTable I0720 21:30:04.946715 config_loader.go:73 Using explicit credential store: memory --- PASS: TestRouting_GetObjectOnBucketNamedGetTable (0.00s) === RUN TestCopyObjectResponse 2026-07-20T21:30:04.94811644Z12345678 --- PASS: TestCopyObjectResponse (0.00s) === RUN TestCopyPartResponse 2026-07-20T21:30:04.948156056Z12345678 --- PASS: TestCopyPartResponse (0.00s) === RUN TestVersionIdFormatDetection === RUN TestVersionIdFormatDetection/new_format_-_inverted_timestamp === RUN TestVersionIdFormatDetection/new_format_-_high_value === RUN TestVersionIdFormatDetection/old_format_-_raw_timestamp === RUN TestVersionIdFormatDetection/old_format_-_low_value === RUN TestVersionIdFormatDetection/null_version === RUN TestVersionIdFormatDetection/short_version_ID === RUN TestVersionIdFormatDetection/empty_version_ID --- PASS: TestVersionIdFormatDetection (0.00s) --- PASS: TestVersionIdFormatDetection/new_format_-_inverted_timestamp (0.00s) --- PASS: TestVersionIdFormatDetection/new_format_-_high_value (0.00s) --- PASS: TestVersionIdFormatDetection/old_format_-_raw_timestamp (0.00s) --- PASS: TestVersionIdFormatDetection/old_format_-_low_value (0.00s) --- PASS: TestVersionIdFormatDetection/null_version (0.00s) --- PASS: TestVersionIdFormatDetection/short_version_ID (0.00s) --- PASS: TestVersionIdFormatDetection/empty_version_ID (0.00s) === RUN TestIsValidVersionID === RUN TestIsValidVersionID/#00 === RUN TestIsValidVersionID/null === RUN TestIsValidVersionID/opaque-version_123 === RUN TestIsValidVersionID/v1/../../../victim === RUN TestIsValidVersionID/v1\..\victim === RUN TestIsValidVersionID/bad\x00version --- PASS: TestIsValidVersionID (0.00s) --- PASS: TestIsValidVersionID/#00 (0.00s) --- PASS: TestIsValidVersionID/null (0.00s) --- PASS: TestIsValidVersionID/opaque-version_123 (0.00s) --- PASS: TestIsValidVersionID/v1/../../../victim (0.00s) --- PASS: TestIsValidVersionID/v1\..\victim (0.00s) --- PASS: TestIsValidVersionID/bad\x00version (0.00s) === RUN TestGenerateVersionIdFormats --- PASS: TestGenerateVersionIdFormats (0.00s) === RUN TestGetVersionTimestamp --- PASS: TestGetVersionTimestamp (0.00s) === RUN TestCompareVersionIdsSameFormatOld --- PASS: TestCompareVersionIdsSameFormatOld (0.00s) === RUN TestCompareVersionIdsSameFormatNew --- PASS: TestCompareVersionIdsSameFormatNew (0.00s) === RUN TestCompareVersionIdsMixedFormats --- PASS: TestCompareVersionIdsMixedFormats (0.00s) === RUN TestCompareVersionIdsNullHandling --- PASS: TestCompareVersionIdsNullHandling (0.00s) === RUN TestOldFormatBackwardCompatibility --- PASS: TestOldFormatBackwardCompatibility (0.00s) === RUN TestNewFormatSorting --- PASS: TestNewFormatSorting (0.00s) === RUN TestMixedFormatTransition --- PASS: TestMixedFormatTransition (0.00s) === RUN TestVersioningHealLogPrefix --- PASS: TestVersioningHealLogPrefix (0.00s) === RUN TestVersioningHealInfof_FormatStringSafe I0720 21:30:04.948440 s3api_versioning_reconciler.go:46 [versioning-heal] event=smoke bucket=a-bucket key=obj-with-%s-percent err=100% --- PASS: TestVersioningHealInfof_FormatStringSafe (0.00s) === RUN TestSanitizeHealArg === RUN TestSanitizeHealArg/plain_string === RUN TestSanitizeHealArg/underscored === RUN TestSanitizeHealArg/slashed_key === RUN TestSanitizeHealArg/space === RUN TestSanitizeHealArg/newline === RUN TestSanitizeHealArg/carriage_return === RUN TestSanitizeHealArg/tab === RUN TestSanitizeHealArg/quote === RUN TestSanitizeHealArg/backslash === RUN TestSanitizeHealArg/control_char_(DEL) === RUN TestSanitizeHealArg/event=_injection_attempt === RUN TestSanitizeHealArg/error_with_newline === RUN TestSanitizeHealArg/error_plain === RUN TestSanitizeHealArg/nil_arg === RUN TestSanitizeHealArg/int_arg === RUN TestSanitizeHealArg/bool_arg --- PASS: TestSanitizeHealArg (0.00s) --- PASS: TestSanitizeHealArg/plain_string (0.00s) --- PASS: TestSanitizeHealArg/underscored (0.00s) --- PASS: TestSanitizeHealArg/slashed_key (0.00s) --- PASS: TestSanitizeHealArg/space (0.00s) --- PASS: TestSanitizeHealArg/newline (0.00s) --- PASS: TestSanitizeHealArg/carriage_return (0.00s) --- PASS: TestSanitizeHealArg/tab (0.00s) --- PASS: TestSanitizeHealArg/quote (0.00s) --- PASS: TestSanitizeHealArg/backslash (0.00s) --- PASS: TestSanitizeHealArg/control_char_(DEL) (0.00s) --- PASS: TestSanitizeHealArg/event=_injection_attempt (0.00s) --- PASS: TestSanitizeHealArg/error_with_newline (0.00s) --- PASS: TestSanitizeHealArg/error_plain (0.00s) --- PASS: TestSanitizeHealArg/nil_arg (0.00s) --- PASS: TestSanitizeHealArg/int_arg (0.00s) --- PASS: TestSanitizeHealArg/bool_arg (0.00s) === RUN TestVersioningHealInfof_KeyWithWhitespaceStaysOneField --- PASS: TestVersioningHealInfof_KeyWithWhitespaceStaysOneField (0.00s) === RUN TestVersioningHealEventVocabulary --- PASS: TestVersioningHealEventVocabulary (0.00s) === RUN TestVersionsHealQueue_DedupOnEnqueue I0720 21:30:04.948650 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=obj queue_depth=1 --- PASS: TestVersionsHealQueue_DedupOnEnqueue (0.00s) === RUN TestVersionsHealQueue_CapacityCap I0720 21:30:04.948672 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x00" queue_depth=1 I0720 21:30:04.948676 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x01" queue_depth=2 I0720 21:30:04.948679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x02" queue_depth=3 I0720 21:30:04.948683 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x03" queue_depth=4 I0720 21:30:04.948686 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x04" queue_depth=5 I0720 21:30:04.948689 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x05" queue_depth=6 I0720 21:30:04.948693 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x06" queue_depth=7 I0720 21:30:04.948695 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\a" queue_depth=8 I0720 21:30:04.948700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\b" queue_depth=9 I0720 21:30:04.948704 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\t" queue_depth=10 I0720 21:30:04.948707 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\n" queue_depth=11 I0720 21:30:04.948710 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\v" queue_depth=12 I0720 21:30:04.948713 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\f" queue_depth=13 I0720 21:30:04.948716 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\r" queue_depth=14 I0720 21:30:04.948720 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x0e" queue_depth=15 I0720 21:30:04.948722 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x0f" queue_depth=16 I0720 21:30:04.948725 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x10" queue_depth=17 I0720 21:30:04.948728 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x11" queue_depth=18 I0720 21:30:04.948731 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x12" queue_depth=19 I0720 21:30:04.948734 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x13" queue_depth=20 I0720 21:30:04.948737 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x14" queue_depth=21 I0720 21:30:04.948740 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x15" queue_depth=22 I0720 21:30:04.948743 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x16" queue_depth=23 I0720 21:30:04.948747 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x17" queue_depth=24 I0720 21:30:04.948750 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x18" queue_depth=25 I0720 21:30:04.948753 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x19" queue_depth=26 I0720 21:30:04.948756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x1a" queue_depth=27 I0720 21:30:04.948759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x1b" queue_depth=28 I0720 21:30:04.948763 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x1c" queue_depth=29 I0720 21:30:04.948767 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x1d" queue_depth=30 I0720 21:30:04.948770 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x1e" queue_depth=31 I0720 21:30:04.948773 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x1f" queue_depth=32 I0720 21:30:04.948777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=" " queue_depth=33 I0720 21:30:04.948780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=! queue_depth=34 I0720 21:30:04.948783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\"" queue_depth=35 I0720 21:30:04.948786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=# queue_depth=36 I0720 21:30:04.948789 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=$ queue_depth=37 I0720 21:30:04.948792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=% queue_depth=38 I0720 21:30:04.948795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=& queue_depth=39 I0720 21:30:04.948798 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=' queue_depth=40 I0720 21:30:04.948801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=( queue_depth=41 I0720 21:30:04.948803 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=) queue_depth=42 I0720 21:30:04.948807 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=* queue_depth=43 I0720 21:30:04.948810 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=+ queue_depth=44 I0720 21:30:04.948813 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=, queue_depth=45 I0720 21:30:04.948816 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=- queue_depth=46 I0720 21:30:04.948819 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=. queue_depth=47 I0720 21:30:04.948822 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=/ queue_depth=48 I0720 21:30:04.948824 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=0 queue_depth=49 I0720 21:30:04.948827 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=1 queue_depth=50 I0720 21:30:04.948830 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=2 queue_depth=51 I0720 21:30:04.948833 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=3 queue_depth=52 I0720 21:30:04.948836 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=4 queue_depth=53 I0720 21:30:04.948839 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=5 queue_depth=54 I0720 21:30:04.948842 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=6 queue_depth=55 I0720 21:30:04.948844 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=7 queue_depth=56 I0720 21:30:04.948849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=8 queue_depth=57 I0720 21:30:04.948853 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=9 queue_depth=58 I0720 21:30:04.948856 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=: queue_depth=59 I0720 21:30:04.948859 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=; queue_depth=60 I0720 21:30:04.948861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=< queue_depth=61 I0720 21:30:04.948864 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key== queue_depth=62 I0720 21:30:04.948867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=> queue_depth=63 I0720 21:30:04.948870 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=? queue_depth=64 I0720 21:30:04.948873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=@ queue_depth=65 I0720 21:30:04.948875 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=A queue_depth=66 I0720 21:30:04.948878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=B queue_depth=67 I0720 21:30:04.948881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=C queue_depth=68 I0720 21:30:04.948884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=D queue_depth=69 I0720 21:30:04.948886 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=E queue_depth=70 I0720 21:30:04.948889 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=F queue_depth=71 I0720 21:30:04.948892 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=G queue_depth=72 I0720 21:30:04.948895 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=H queue_depth=73 I0720 21:30:04.948898 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=I queue_depth=74 I0720 21:30:04.948901 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=J queue_depth=75 I0720 21:30:04.948904 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=K queue_depth=76 I0720 21:30:04.948907 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=L queue_depth=77 I0720 21:30:04.948910 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=M queue_depth=78 I0720 21:30:04.948913 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=N queue_depth=79 I0720 21:30:04.948916 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=O queue_depth=80 I0720 21:30:04.948918 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=P queue_depth=81 I0720 21:30:04.948921 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Q queue_depth=82 I0720 21:30:04.948926 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=R queue_depth=83 I0720 21:30:04.948928 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=S queue_depth=84 I0720 21:30:04.948932 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=T queue_depth=85 I0720 21:30:04.948937 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=U queue_depth=86 I0720 21:30:04.948941 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=V queue_depth=87 I0720 21:30:04.948944 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=W queue_depth=88 I0720 21:30:04.948948 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=X queue_depth=89 I0720 21:30:04.948952 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Y queue_depth=90 I0720 21:30:04.948957 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Z queue_depth=91 I0720 21:30:04.948963 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=[ queue_depth=92 I0720 21:30:04.948968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\\" queue_depth=93 I0720 21:30:04.948973 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=] queue_depth=94 I0720 21:30:04.948978 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=^ queue_depth=95 I0720 21:30:04.948982 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=_ queue_depth=96 I0720 21:30:04.948986 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=` queue_depth=97 I0720 21:30:04.948991 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=a queue_depth=98 I0720 21:30:04.948996 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=b queue_depth=99 I0720 21:30:04.948999 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=c queue_depth=100 I0720 21:30:04.949001 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=d queue_depth=101 I0720 21:30:04.949004 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=e queue_depth=102 I0720 21:30:04.949007 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=f queue_depth=103 I0720 21:30:04.949009 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=g queue_depth=104 I0720 21:30:04.949012 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=h queue_depth=105 I0720 21:30:04.949014 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=i queue_depth=106 I0720 21:30:04.949017 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=j queue_depth=107 I0720 21:30:04.949020 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=k queue_depth=108 I0720 21:30:04.949027 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=l queue_depth=109 I0720 21:30:04.949030 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=m queue_depth=110 I0720 21:30:04.949033 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=n queue_depth=111 I0720 21:30:04.949036 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=o queue_depth=112 I0720 21:30:04.949046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=p queue_depth=113 I0720 21:30:04.949049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=q queue_depth=114 I0720 21:30:04.949052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=r queue_depth=115 I0720 21:30:04.949056 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=s queue_depth=116 I0720 21:30:04.949060 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=t queue_depth=117 I0720 21:30:04.949062 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=u queue_depth=118 I0720 21:30:04.949065 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=v queue_depth=119 I0720 21:30:04.949068 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=w queue_depth=120 I0720 21:30:04.949071 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=x queue_depth=121 I0720 21:30:04.949074 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=y queue_depth=122 I0720 21:30:04.949077 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=z queue_depth=123 I0720 21:30:04.949082 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key={ queue_depth=124 I0720 21:30:04.949085 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=| queue_depth=125 I0720 21:30:04.949088 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=} queue_depth=126 I0720 21:30:04.949091 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=~ queue_depth=127 I0720 21:30:04.949094 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key="\x7f" queue_depth=128 I0720 21:30:04.949097 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=€ queue_depth=129 I0720 21:30:04.949100 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key= queue_depth=130 I0720 21:30:04.949104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=‚ queue_depth=131 I0720 21:30:04.949110 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƒ queue_depth=132 I0720 21:30:04.949113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=„ queue_depth=133 I0720 21:30:04.949116 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=… queue_depth=134 I0720 21:30:04.949119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=† queue_depth=135 I0720 21:30:04.949122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=‡ queue_depth=136 I0720 21:30:04.949125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˆ queue_depth=137 I0720 21:30:04.949128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=‰ queue_depth=138 I0720 21:30:04.949131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Š queue_depth=139 I0720 21:30:04.949133 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=‹ queue_depth=140 I0720 21:30:04.949136 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Œ queue_depth=141 I0720 21:30:04.949140 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key= queue_depth=142 I0720 21:30:04.949143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ž queue_depth=143 I0720 21:30:04.949145 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key= queue_depth=144 I0720 21:30:04.949148 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key= queue_depth=145 I0720 21:30:04.949151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=‘ queue_depth=146 I0720 21:30:04.949154 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=’ queue_depth=147 I0720 21:30:04.949157 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=“ queue_depth=148 I0720 21:30:04.949160 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=” queue_depth=149 I0720 21:30:04.949162 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=• queue_depth=150 I0720 21:30:04.949165 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=– queue_depth=151 I0720 21:30:04.949168 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=— queue_depth=152 I0720 21:30:04.949172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˜ queue_depth=153 I0720 21:30:04.949174 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=™ queue_depth=154 I0720 21:30:04.949177 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=š queue_depth=155 I0720 21:30:04.949180 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=› queue_depth=156 I0720 21:30:04.949183 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=œ queue_depth=157 I0720 21:30:04.949186 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key= queue_depth=158 I0720 21:30:04.949189 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ž queue_depth=159 I0720 21:30:04.949192 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ÿ queue_depth=160 I0720 21:30:04.949195 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=  queue_depth=161 I0720 21:30:04.949198 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¡ queue_depth=162 I0720 21:30:04.949201 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¢ queue_depth=163 I0720 21:30:04.949203 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=£ queue_depth=164 I0720 21:30:04.949206 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¤ queue_depth=165 I0720 21:30:04.949209 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¥ queue_depth=166 I0720 21:30:04.949212 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¦ queue_depth=167 I0720 21:30:04.949215 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=§ queue_depth=168 I0720 21:30:04.949218 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¨ queue_depth=169 I0720 21:30:04.949221 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=© queue_depth=170 I0720 21:30:04.949224 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ª queue_depth=171 I0720 21:30:04.949227 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=« queue_depth=172 I0720 21:30:04.949231 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¬ queue_depth=173 I0720 21:30:04.949233 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=­ queue_depth=174 I0720 21:30:04.949236 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=® queue_depth=175 I0720 21:30:04.949239 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¯ queue_depth=176 I0720 21:30:04.949242 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=° queue_depth=177 I0720 21:30:04.949245 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=± queue_depth=178 I0720 21:30:04.949248 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=² queue_depth=179 I0720 21:30:04.949251 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=³ queue_depth=180 I0720 21:30:04.949253 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=´ queue_depth=181 I0720 21:30:04.949256 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=µ queue_depth=182 I0720 21:30:04.949259 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¶ queue_depth=183 I0720 21:30:04.949262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=· queue_depth=184 I0720 21:30:04.949265 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¸ queue_depth=185 I0720 21:30:04.949268 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¹ queue_depth=186 I0720 21:30:04.949271 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=º queue_depth=187 I0720 21:30:04.949274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=» queue_depth=188 I0720 21:30:04.949277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¼ queue_depth=189 I0720 21:30:04.949280 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=½ queue_depth=190 I0720 21:30:04.949283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¾ queue_depth=191 I0720 21:30:04.949288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=¿ queue_depth=192 I0720 21:30:04.949291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=À queue_depth=193 I0720 21:30:04.949293 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Á queue_depth=194 I0720 21:30:04.949296 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key= queue_depth=195 I0720 21:30:04.949299 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=à queue_depth=196 I0720 21:30:04.949302 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ä queue_depth=197 I0720 21:30:04.949306 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Å queue_depth=198 I0720 21:30:04.949309 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Æ queue_depth=199 I0720 21:30:04.949312 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ç queue_depth=200 I0720 21:30:04.949315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=È queue_depth=201 I0720 21:30:04.949318 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=É queue_depth=202 I0720 21:30:04.949321 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ê queue_depth=203 I0720 21:30:04.949324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ë queue_depth=204 I0720 21:30:04.949327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ì queue_depth=205 I0720 21:30:04.949330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Í queue_depth=206 I0720 21:30:04.949333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Î queue_depth=207 I0720 21:30:04.949336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ï queue_depth=208 I0720 21:30:04.949339 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ð queue_depth=209 I0720 21:30:04.949342 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ñ queue_depth=210 I0720 21:30:04.949345 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ò queue_depth=211 I0720 21:30:04.949348 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ó queue_depth=212 I0720 21:30:04.949351 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ô queue_depth=213 I0720 21:30:04.949354 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Õ queue_depth=214 I0720 21:30:04.949360 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ö queue_depth=215 I0720 21:30:04.949363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=× queue_depth=216 I0720 21:30:04.949366 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ø queue_depth=217 I0720 21:30:04.949369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ù queue_depth=218 I0720 21:30:04.949372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ú queue_depth=219 I0720 21:30:04.949375 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Û queue_depth=220 I0720 21:30:04.949378 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ü queue_depth=221 I0720 21:30:04.949381 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ý queue_depth=222 I0720 21:30:04.949384 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Þ queue_depth=223 I0720 21:30:04.949387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ß queue_depth=224 I0720 21:30:04.949396 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=à queue_depth=225 I0720 21:30:04.949402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=á queue_depth=226 I0720 21:30:04.949405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=â queue_depth=227 I0720 21:30:04.949408 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ã queue_depth=228 I0720 21:30:04.949410 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ä queue_depth=229 I0720 21:30:04.949413 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=å queue_depth=230 I0720 21:30:04.949415 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=æ queue_depth=231 I0720 21:30:04.949418 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ç queue_depth=232 I0720 21:30:04.949421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=è queue_depth=233 I0720 21:30:04.949424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=é queue_depth=234 I0720 21:30:04.949427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ê queue_depth=235 I0720 21:30:04.949430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ë queue_depth=236 I0720 21:30:04.949433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ì queue_depth=237 I0720 21:30:04.949436 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=í queue_depth=238 I0720 21:30:04.949439 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=î queue_depth=239 I0720 21:30:04.949442 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ï queue_depth=240 I0720 21:30:04.949445 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ð queue_depth=241 I0720 21:30:04.949448 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ñ queue_depth=242 I0720 21:30:04.949451 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ò queue_depth=243 I0720 21:30:04.949454 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ó queue_depth=244 I0720 21:30:04.949457 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ô queue_depth=245 I0720 21:30:04.949459 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=õ queue_depth=246 I0720 21:30:04.949462 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ö queue_depth=247 I0720 21:30:04.949465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=÷ queue_depth=248 I0720 21:30:04.949468 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ø queue_depth=249 I0720 21:30:04.949471 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ù queue_depth=250 I0720 21:30:04.949474 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ú queue_depth=251 I0720 21:30:04.949477 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=û queue_depth=252 I0720 21:30:04.949480 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ü queue_depth=253 I0720 21:30:04.949483 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ý queue_depth=254 I0720 21:30:04.949486 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=þ queue_depth=255 I0720 21:30:04.949489 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ÿ queue_depth=256 I0720 21:30:04.949492 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ā queue_depth=257 I0720 21:30:04.949495 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ā queue_depth=258 I0720 21:30:04.949498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ă queue_depth=259 I0720 21:30:04.949501 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ă queue_depth=260 I0720 21:30:04.949504 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ą queue_depth=261 I0720 21:30:04.949506 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ą queue_depth=262 I0720 21:30:04.949509 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ć queue_depth=263 I0720 21:30:04.949512 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ć queue_depth=264 I0720 21:30:04.949515 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĉ queue_depth=265 I0720 21:30:04.949518 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĉ queue_depth=266 I0720 21:30:04.949521 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ċ queue_depth=267 I0720 21:30:04.949524 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ċ queue_depth=268 I0720 21:30:04.949527 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Č queue_depth=269 I0720 21:30:04.949530 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=č queue_depth=270 I0720 21:30:04.949533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ď queue_depth=271 I0720 21:30:04.949535 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ď queue_depth=272 I0720 21:30:04.949538 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Đ queue_depth=273 I0720 21:30:04.949541 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=đ queue_depth=274 I0720 21:30:04.949545 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ē queue_depth=275 I0720 21:30:04.949547 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ē queue_depth=276 I0720 21:30:04.949550 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĕ queue_depth=277 I0720 21:30:04.949553 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĕ queue_depth=278 I0720 21:30:04.949556 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ė queue_depth=279 I0720 21:30:04.949559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ė queue_depth=280 I0720 21:30:04.949561 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ę queue_depth=281 I0720 21:30:04.949564 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ę queue_depth=282 I0720 21:30:04.949567 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ě queue_depth=283 I0720 21:30:04.949570 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ě queue_depth=284 I0720 21:30:04.949573 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĝ queue_depth=285 I0720 21:30:04.949576 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĝ queue_depth=286 I0720 21:30:04.949578 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ğ queue_depth=287 I0720 21:30:04.949581 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ğ queue_depth=288 I0720 21:30:04.949584 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ġ queue_depth=289 I0720 21:30:04.949587 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ġ queue_depth=290 I0720 21:30:04.949590 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ģ queue_depth=291 I0720 21:30:04.949593 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ģ queue_depth=292 I0720 21:30:04.949596 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĥ queue_depth=293 I0720 21:30:04.949599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĥ queue_depth=294 I0720 21:30:04.949602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ħ queue_depth=295 I0720 21:30:04.949605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ħ queue_depth=296 I0720 21:30:04.949608 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĩ queue_depth=297 I0720 21:30:04.949611 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĩ queue_depth=298 I0720 21:30:04.949614 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ī queue_depth=299 I0720 21:30:04.949617 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ī queue_depth=300 I0720 21:30:04.949620 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĭ queue_depth=301 I0720 21:30:04.949623 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĭ queue_depth=302 I0720 21:30:04.949625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Į queue_depth=303 I0720 21:30:04.949628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=į queue_depth=304 I0720 21:30:04.949631 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=İ queue_depth=305 I0720 21:30:04.949634 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ı queue_depth=306 I0720 21:30:04.949638 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=IJ queue_depth=307 I0720 21:30:04.949641 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ij queue_depth=308 I0720 21:30:04.949644 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĵ queue_depth=309 I0720 21:30:04.949647 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĵ queue_depth=310 I0720 21:30:04.949650 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ķ queue_depth=311 I0720 21:30:04.949652 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ķ queue_depth=312 I0720 21:30:04.949655 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĸ queue_depth=313 I0720 21:30:04.949658 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ĺ queue_depth=314 I0720 21:30:04.949661 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ĺ queue_depth=315 I0720 21:30:04.949664 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ļ queue_depth=316 I0720 21:30:04.949667 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ļ queue_depth=317 I0720 21:30:04.949670 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ľ queue_depth=318 I0720 21:30:04.949673 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ľ queue_depth=319 I0720 21:30:04.949676 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŀ queue_depth=320 I0720 21:30:04.949679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŀ queue_depth=321 I0720 21:30:04.949682 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ł queue_depth=322 I0720 21:30:04.949685 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ł queue_depth=323 I0720 21:30:04.949688 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ń queue_depth=324 I0720 21:30:04.949691 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ń queue_depth=325 I0720 21:30:04.949694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ņ queue_depth=326 I0720 21:30:04.949696 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ņ queue_depth=327 I0720 21:30:04.949700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ň queue_depth=328 I0720 21:30:04.949703 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ň queue_depth=329 I0720 21:30:04.949706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʼn queue_depth=330 I0720 21:30:04.949709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŋ queue_depth=331 I0720 21:30:04.949712 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŋ queue_depth=332 I0720 21:30:04.949715 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ō queue_depth=333 I0720 21:30:04.949718 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ō queue_depth=334 I0720 21:30:04.949721 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŏ queue_depth=335 I0720 21:30:04.949724 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŏ queue_depth=336 I0720 21:30:04.949727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ő queue_depth=337 I0720 21:30:04.949729 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ő queue_depth=338 I0720 21:30:04.949732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Œ queue_depth=339 I0720 21:30:04.949736 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=œ queue_depth=340 I0720 21:30:04.949738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŕ queue_depth=341 I0720 21:30:04.949741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŕ queue_depth=342 I0720 21:30:04.949744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŗ queue_depth=343 I0720 21:30:04.949747 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŗ queue_depth=344 I0720 21:30:04.949750 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ř queue_depth=345 I0720 21:30:04.949753 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ř queue_depth=346 I0720 21:30:04.949756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ś queue_depth=347 I0720 21:30:04.949759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ś queue_depth=348 I0720 21:30:04.949762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŝ queue_depth=349 I0720 21:30:04.949765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŝ queue_depth=350 I0720 21:30:04.949767 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ş queue_depth=351 I0720 21:30:04.949770 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ş queue_depth=352 I0720 21:30:04.949773 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Š queue_depth=353 I0720 21:30:04.949777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=š queue_depth=354 I0720 21:30:04.949779 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ţ queue_depth=355 I0720 21:30:04.949782 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ţ queue_depth=356 I0720 21:30:04.949785 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ť queue_depth=357 I0720 21:30:04.949788 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ť queue_depth=358 I0720 21:30:04.949791 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŧ queue_depth=359 I0720 21:30:04.949794 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŧ queue_depth=360 I0720 21:30:04.949797 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ũ queue_depth=361 I0720 21:30:04.949801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ũ queue_depth=362 I0720 21:30:04.949804 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ū queue_depth=363 I0720 21:30:04.949806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ū queue_depth=364 I0720 21:30:04.949809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŭ queue_depth=365 I0720 21:30:04.949812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŭ queue_depth=366 I0720 21:30:04.949815 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ů queue_depth=367 I0720 21:30:04.949818 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ů queue_depth=368 I0720 21:30:04.949821 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ű queue_depth=369 I0720 21:30:04.949824 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ű queue_depth=370 I0720 21:30:04.949826 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ų queue_depth=371 I0720 21:30:04.949829 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ų queue_depth=372 I0720 21:30:04.949833 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŵ queue_depth=373 I0720 21:30:04.949835 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŵ queue_depth=374 I0720 21:30:04.949838 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ŷ queue_depth=375 I0720 21:30:04.949840 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ŷ queue_depth=376 I0720 21:30:04.949844 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ÿ queue_depth=377 I0720 21:30:04.949846 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ź queue_depth=378 I0720 21:30:04.949849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ź queue_depth=379 I0720 21:30:04.949852 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ż queue_depth=380 I0720 21:30:04.949855 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ż queue_depth=381 I0720 21:30:04.949858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ž queue_depth=382 I0720 21:30:04.949861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ž queue_depth=383 I0720 21:30:04.949864 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ſ queue_depth=384 I0720 21:30:04.949867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƀ queue_depth=385 I0720 21:30:04.949869 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɓ queue_depth=386 I0720 21:30:04.949872 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƃ queue_depth=387 I0720 21:30:04.949875 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƃ queue_depth=388 I0720 21:30:04.949878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƅ queue_depth=389 I0720 21:30:04.949881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƅ queue_depth=390 I0720 21:30:04.949884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɔ queue_depth=391 I0720 21:30:04.949887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƈ queue_depth=392 I0720 21:30:04.949890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƈ queue_depth=393 I0720 21:30:04.949893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɖ queue_depth=394 I0720 21:30:04.949896 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɗ queue_depth=395 I0720 21:30:04.949899 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƌ queue_depth=396 I0720 21:30:04.949902 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƌ queue_depth=397 I0720 21:30:04.949905 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƍ queue_depth=398 I0720 21:30:04.949908 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǝ queue_depth=399 I0720 21:30:04.949911 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ə queue_depth=400 I0720 21:30:04.949914 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɛ queue_depth=401 I0720 21:30:04.949917 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƒ queue_depth=402 I0720 21:30:04.949919 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƒ queue_depth=403 I0720 21:30:04.949923 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɠ queue_depth=404 I0720 21:30:04.949926 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɣ queue_depth=405 I0720 21:30:04.949929 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƕ queue_depth=406 I0720 21:30:04.949932 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɩ queue_depth=407 I0720 21:30:04.949935 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɨ queue_depth=408 I0720 21:30:04.949938 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƙ queue_depth=409 I0720 21:30:04.949941 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƙ queue_depth=410 I0720 21:30:04.949944 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƚ queue_depth=411 I0720 21:30:04.949947 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƛ queue_depth=412 I0720 21:30:04.949950 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɯ queue_depth=413 I0720 21:30:04.949953 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɲ queue_depth=414 I0720 21:30:04.949956 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƞ queue_depth=415 I0720 21:30:04.949959 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɵ queue_depth=416 I0720 21:30:04.949962 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ơ queue_depth=417 I0720 21:30:04.949964 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ơ queue_depth=418 I0720 21:30:04.949967 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƣ queue_depth=419 I0720 21:30:04.949970 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƣ queue_depth=420 I0720 21:30:04.949973 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƥ queue_depth=421 I0720 21:30:04.949976 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƥ queue_depth=422 I0720 21:30:04.949979 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʀ queue_depth=423 I0720 21:30:04.949982 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƨ queue_depth=424 I0720 21:30:04.949985 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƨ queue_depth=425 I0720 21:30:04.949990 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʃ queue_depth=426 I0720 21:30:04.949993 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƪ queue_depth=427 I0720 21:30:04.949996 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƫ queue_depth=428 I0720 21:30:04.949999 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƭ queue_depth=429 I0720 21:30:04.950005 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƭ queue_depth=430 I0720 21:30:04.950008 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʈ queue_depth=431 I0720 21:30:04.950011 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ư queue_depth=432 I0720 21:30:04.950014 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ư queue_depth=433 I0720 21:30:04.950017 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʊ queue_depth=434 I0720 21:30:04.950020 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʋ queue_depth=435 I0720 21:30:04.950023 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƴ queue_depth=436 I0720 21:30:04.950026 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƴ queue_depth=437 I0720 21:30:04.950029 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƶ queue_depth=438 I0720 21:30:04.950033 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƶ queue_depth=439 I0720 21:30:04.950036 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʒ queue_depth=440 I0720 21:30:04.950038 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƹ queue_depth=441 I0720 21:30:04.950041 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƹ queue_depth=442 I0720 21:30:04.950044 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƺ queue_depth=443 I0720 21:30:04.950047 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƻ queue_depth=444 I0720 21:30:04.950050 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƽ queue_depth=445 I0720 21:30:04.950053 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƽ queue_depth=446 I0720 21:30:04.950056 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƾ queue_depth=447 I0720 21:30:04.950059 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ƿ queue_depth=448 I0720 21:30:04.950077 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǀ queue_depth=449 I0720 21:30:04.950082 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǁ queue_depth=450 I0720 21:30:04.950085 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǂ queue_depth=451 I0720 21:30:04.950088 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǃ queue_depth=452 I0720 21:30:04.950091 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=DŽ queue_depth=453 I0720 21:30:04.950094 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Dž queue_depth=454 I0720 21:30:04.950097 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=dž queue_depth=455 I0720 21:30:04.950099 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=LJ queue_depth=456 I0720 21:30:04.950102 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Lj queue_depth=457 I0720 21:30:04.950105 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=lj queue_depth=458 I0720 21:30:04.950108 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=NJ queue_depth=459 I0720 21:30:04.950111 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Nj queue_depth=460 I0720 21:30:04.950114 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=nj queue_depth=461 I0720 21:30:04.950117 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǎ queue_depth=462 I0720 21:30:04.950119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǎ queue_depth=463 I0720 21:30:04.950122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǐ queue_depth=464 I0720 21:30:04.950125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǐ queue_depth=465 I0720 21:30:04.950128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǒ queue_depth=466 I0720 21:30:04.950132 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǒ queue_depth=467 I0720 21:30:04.950137 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǔ queue_depth=468 I0720 21:30:04.950140 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǔ queue_depth=469 I0720 21:30:04.950143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǖ queue_depth=470 I0720 21:30:04.950146 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǖ queue_depth=471 I0720 21:30:04.950150 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǘ queue_depth=472 I0720 21:30:04.950153 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǘ queue_depth=473 I0720 21:30:04.950155 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǚ queue_depth=474 I0720 21:30:04.950158 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǚ queue_depth=475 I0720 21:30:04.950161 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǜ queue_depth=476 I0720 21:30:04.950164 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǜ queue_depth=477 I0720 21:30:04.950167 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǝ queue_depth=478 I0720 21:30:04.950170 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǟ queue_depth=479 I0720 21:30:04.950172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǟ queue_depth=480 I0720 21:30:04.950175 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǡ queue_depth=481 I0720 21:30:04.950178 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǡ queue_depth=482 I0720 21:30:04.950181 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǣ queue_depth=483 I0720 21:30:04.950184 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǣ queue_depth=484 I0720 21:30:04.950187 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǥ queue_depth=485 I0720 21:30:04.950189 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǥ queue_depth=486 I0720 21:30:04.950192 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǧ queue_depth=487 I0720 21:30:04.950195 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǧ queue_depth=488 I0720 21:30:04.950198 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǩ queue_depth=489 I0720 21:30:04.950201 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǩ queue_depth=490 I0720 21:30:04.950204 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǫ queue_depth=491 I0720 21:30:04.950207 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǫ queue_depth=492 I0720 21:30:04.950210 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǭ queue_depth=493 I0720 21:30:04.950213 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǭ queue_depth=494 I0720 21:30:04.950216 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǯ queue_depth=495 I0720 21:30:04.950218 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǯ queue_depth=496 I0720 21:30:04.950221 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǰ queue_depth=497 I0720 21:30:04.950224 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=DZ queue_depth=498 I0720 21:30:04.950229 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Dz queue_depth=499 I0720 21:30:04.950232 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=dz queue_depth=500 I0720 21:30:04.950235 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǵ queue_depth=501 I0720 21:30:04.950238 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǵ queue_depth=502 I0720 21:30:04.950241 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƕ queue_depth=503 I0720 21:30:04.950244 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƿ queue_depth=504 I0720 21:30:04.950247 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǹ queue_depth=505 I0720 21:30:04.950250 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǹ queue_depth=506 I0720 21:30:04.950253 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǻ queue_depth=507 I0720 21:30:04.950255 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǻ queue_depth=508 I0720 21:30:04.950259 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǽ queue_depth=509 I0720 21:30:04.950262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǽ queue_depth=510 I0720 21:30:04.950265 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ǿ queue_depth=511 I0720 21:30:04.950268 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ǿ queue_depth=512 I0720 21:30:04.950271 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȁ queue_depth=513 I0720 21:30:04.950274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȁ queue_depth=514 I0720 21:30:04.950277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȃ queue_depth=515 I0720 21:30:04.950280 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȃ queue_depth=516 I0720 21:30:04.950283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȅ queue_depth=517 I0720 21:30:04.950286 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȅ queue_depth=518 I0720 21:30:04.950288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȇ queue_depth=519 I0720 21:30:04.950291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȇ queue_depth=520 I0720 21:30:04.950294 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȉ queue_depth=521 I0720 21:30:04.950298 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȉ queue_depth=522 I0720 21:30:04.950301 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȋ queue_depth=523 I0720 21:30:04.950304 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȋ queue_depth=524 I0720 21:30:04.950306 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȍ queue_depth=525 I0720 21:30:04.950309 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȍ queue_depth=526 I0720 21:30:04.950312 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȏ queue_depth=527 I0720 21:30:04.950315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȏ queue_depth=528 I0720 21:30:04.950318 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȑ queue_depth=529 I0720 21:30:04.950321 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȑ queue_depth=530 I0720 21:30:04.950324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȓ queue_depth=531 I0720 21:30:04.950327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȓ queue_depth=532 I0720 21:30:04.950330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȕ queue_depth=533 I0720 21:30:04.950333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȕ queue_depth=534 I0720 21:30:04.950336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȗ queue_depth=535 I0720 21:30:04.950339 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȗ queue_depth=536 I0720 21:30:04.950342 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ș queue_depth=537 I0720 21:30:04.950345 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ș queue_depth=538 I0720 21:30:04.950348 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ț queue_depth=539 I0720 21:30:04.950351 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ț queue_depth=540 I0720 21:30:04.950355 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȝ queue_depth=541 I0720 21:30:04.950357 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȝ queue_depth=542 I0720 21:30:04.950360 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȟ queue_depth=543 I0720 21:30:04.950363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȟ queue_depth=544 I0720 21:30:04.950366 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƞ queue_depth=545 I0720 21:30:04.950369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȡ queue_depth=546 I0720 21:30:04.950372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȣ queue_depth=547 I0720 21:30:04.950375 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȣ queue_depth=548 I0720 21:30:04.950377 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȥ queue_depth=549 I0720 21:30:04.950380 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȥ queue_depth=550 I0720 21:30:04.950383 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȧ queue_depth=551 I0720 21:30:04.950386 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȧ queue_depth=552 I0720 21:30:04.950389 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȩ queue_depth=553 I0720 21:30:04.950392 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȩ queue_depth=554 I0720 21:30:04.950395 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȫ queue_depth=555 I0720 21:30:04.950398 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȫ queue_depth=556 I0720 21:30:04.950401 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȭ queue_depth=557 I0720 21:30:04.950403 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȭ queue_depth=558 I0720 21:30:04.950406 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȯ queue_depth=559 I0720 21:30:04.950409 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȯ queue_depth=560 I0720 21:30:04.950412 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȱ queue_depth=561 I0720 21:30:04.950415 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȱ queue_depth=562 I0720 21:30:04.950418 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȳ queue_depth=563 I0720 21:30:04.950421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȳ queue_depth=564 I0720 21:30:04.950424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȴ queue_depth=565 I0720 21:30:04.950439 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȵ queue_depth=566 I0720 21:30:04.950444 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȶ queue_depth=567 I0720 21:30:04.950447 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȷ queue_depth=568 I0720 21:30:04.950450 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȸ queue_depth=569 I0720 21:30:04.950453 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȹ queue_depth=570 I0720 21:30:04.950457 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ⱥ queue_depth=571 I0720 21:30:04.950460 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ȼ queue_depth=572 I0720 21:30:04.950463 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȼ queue_depth=573 I0720 21:30:04.950465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƚ queue_depth=574 I0720 21:30:04.950468 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ⱦ queue_depth=575 I0720 21:30:04.950471 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ȿ queue_depth=576 I0720 21:30:04.950474 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɀ queue_depth=577 I0720 21:30:04.950477 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɂ queue_depth=578 I0720 21:30:04.950480 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɂ queue_depth=579 I0720 21:30:04.950483 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ƀ queue_depth=580 I0720 21:30:04.950486 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʉ queue_depth=581 I0720 21:30:04.950489 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ʌ queue_depth=582 I0720 21:30:04.950492 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɇ queue_depth=583 I0720 21:30:04.950495 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɇ queue_depth=584 I0720 21:30:04.950498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɉ queue_depth=585 I0720 21:30:04.950500 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɉ queue_depth=586 I0720 21:30:04.950503 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɋ queue_depth=587 I0720 21:30:04.950507 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɋ queue_depth=588 I0720 21:30:04.950510 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɍ queue_depth=589 I0720 21:30:04.950513 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɍ queue_depth=590 I0720 21:30:04.950516 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ɏ queue_depth=591 I0720 21:30:04.950519 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɏ queue_depth=592 I0720 21:30:04.950521 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɐ queue_depth=593 I0720 21:30:04.950524 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɑ queue_depth=594 I0720 21:30:04.950527 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɒ queue_depth=595 I0720 21:30:04.950530 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɓ queue_depth=596 I0720 21:30:04.950533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɔ queue_depth=597 I0720 21:30:04.950536 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɕ queue_depth=598 I0720 21:30:04.950538 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɖ queue_depth=599 I0720 21:30:04.950541 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɗ queue_depth=600 I0720 21:30:04.950544 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɘ queue_depth=601 I0720 21:30:04.950547 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ə queue_depth=602 I0720 21:30:04.950550 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɚ queue_depth=603 I0720 21:30:04.950553 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɛ queue_depth=604 I0720 21:30:04.950556 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɜ queue_depth=605 I0720 21:30:04.950579 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɝ queue_depth=606 I0720 21:30:04.950584 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɞ queue_depth=607 I0720 21:30:04.950587 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɟ queue_depth=608 I0720 21:30:04.950590 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɠ queue_depth=609 I0720 21:30:04.950593 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɡ queue_depth=610 I0720 21:30:04.950596 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɢ queue_depth=611 I0720 21:30:04.950599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɣ queue_depth=612 I0720 21:30:04.950601 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɤ queue_depth=613 I0720 21:30:04.950604 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɥ queue_depth=614 I0720 21:30:04.950607 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɦ queue_depth=615 I0720 21:30:04.950610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɧ queue_depth=616 I0720 21:30:04.950613 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɨ queue_depth=617 I0720 21:30:04.950616 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɩ queue_depth=618 I0720 21:30:04.950619 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɪ queue_depth=619 I0720 21:30:04.950622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɫ queue_depth=620 I0720 21:30:04.950625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɬ queue_depth=621 I0720 21:30:04.950628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɭ queue_depth=622 I0720 21:30:04.950631 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɮ queue_depth=623 I0720 21:30:04.950634 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɯ queue_depth=624 I0720 21:30:04.950637 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɰ queue_depth=625 I0720 21:30:04.950640 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɱ queue_depth=626 I0720 21:30:04.950643 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɲ queue_depth=627 I0720 21:30:04.950646 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɳ queue_depth=628 I0720 21:30:04.950649 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɴ queue_depth=629 I0720 21:30:04.950652 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɵ queue_depth=630 I0720 21:30:04.950655 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɶ queue_depth=631 I0720 21:30:04.950658 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɷ queue_depth=632 I0720 21:30:04.950661 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɸ queue_depth=633 I0720 21:30:04.950664 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɹ queue_depth=634 I0720 21:30:04.950667 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɺ queue_depth=635 I0720 21:30:04.950670 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɻ queue_depth=636 I0720 21:30:04.950673 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɼ queue_depth=637 I0720 21:30:04.950676 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɽ queue_depth=638 I0720 21:30:04.950679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɾ queue_depth=639 I0720 21:30:04.950682 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ɿ queue_depth=640 I0720 21:30:04.950685 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʀ queue_depth=641 I0720 21:30:04.950688 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʁ queue_depth=642 I0720 21:30:04.950691 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʂ queue_depth=643 I0720 21:30:04.950694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʃ queue_depth=644 I0720 21:30:04.950697 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʄ queue_depth=645 I0720 21:30:04.950700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʅ queue_depth=646 I0720 21:30:04.950703 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʆ queue_depth=647 I0720 21:30:04.950706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʇ queue_depth=648 I0720 21:30:04.950709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʈ queue_depth=649 I0720 21:30:04.950712 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʉ queue_depth=650 I0720 21:30:04.950714 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʊ queue_depth=651 I0720 21:30:04.950717 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʋ queue_depth=652 I0720 21:30:04.950720 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʌ queue_depth=653 I0720 21:30:04.950723 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʍ queue_depth=654 I0720 21:30:04.950726 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʎ queue_depth=655 I0720 21:30:04.950729 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʏ queue_depth=656 I0720 21:30:04.950732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʐ queue_depth=657 I0720 21:30:04.950735 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʑ queue_depth=658 I0720 21:30:04.950738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʒ queue_depth=659 I0720 21:30:04.950741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʓ queue_depth=660 I0720 21:30:04.950743 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʔ queue_depth=661 I0720 21:30:04.950746 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʕ queue_depth=662 I0720 21:30:04.950749 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʖ queue_depth=663 I0720 21:30:04.950752 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʗ queue_depth=664 I0720 21:30:04.950755 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʘ queue_depth=665 I0720 21:30:04.950758 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʙ queue_depth=666 I0720 21:30:04.950761 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʚ queue_depth=667 I0720 21:30:04.950764 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʛ queue_depth=668 I0720 21:30:04.950767 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʜ queue_depth=669 I0720 21:30:04.950771 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʝ queue_depth=670 I0720 21:30:04.950774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʞ queue_depth=671 I0720 21:30:04.950777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʟ queue_depth=672 I0720 21:30:04.950780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʠ queue_depth=673 I0720 21:30:04.950783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʡ queue_depth=674 I0720 21:30:04.950787 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʢ queue_depth=675 I0720 21:30:04.950790 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʣ queue_depth=676 I0720 21:30:04.950793 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʤ queue_depth=677 I0720 21:30:04.950796 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʥ queue_depth=678 I0720 21:30:04.950799 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʦ queue_depth=679 I0720 21:30:04.950802 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʧ queue_depth=680 I0720 21:30:04.950805 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʨ queue_depth=681 I0720 21:30:04.950808 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʩ queue_depth=682 I0720 21:30:04.950811 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʪ queue_depth=683 I0720 21:30:04.950814 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʫ queue_depth=684 I0720 21:30:04.950816 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʬ queue_depth=685 I0720 21:30:04.950820 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʭ queue_depth=686 I0720 21:30:04.950823 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʮ queue_depth=687 I0720 21:30:04.950825 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʯ queue_depth=688 I0720 21:30:04.950828 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʰ queue_depth=689 I0720 21:30:04.950831 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʱ queue_depth=690 I0720 21:30:04.950834 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʲ queue_depth=691 I0720 21:30:04.950837 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʳ queue_depth=692 I0720 21:30:04.950840 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʴ queue_depth=693 I0720 21:30:04.950842 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʵ queue_depth=694 I0720 21:30:04.950845 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʶ queue_depth=695 I0720 21:30:04.950848 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʷ queue_depth=696 I0720 21:30:04.950851 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʸ queue_depth=697 I0720 21:30:04.950854 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʹ queue_depth=698 I0720 21:30:04.950857 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʺ queue_depth=699 I0720 21:30:04.950860 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʻ queue_depth=700 I0720 21:30:04.950863 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʼ queue_depth=701 I0720 21:30:04.950866 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʽ queue_depth=702 I0720 21:30:04.950870 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʾ queue_depth=703 I0720 21:30:04.950873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʿ queue_depth=704 I0720 21:30:04.950881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˀ queue_depth=705 I0720 21:30:04.950884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˁ queue_depth=706 I0720 21:30:04.950887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˂ queue_depth=707 I0720 21:30:04.950890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˃ queue_depth=708 I0720 21:30:04.950894 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˄ queue_depth=709 I0720 21:30:04.950897 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˅ queue_depth=710 I0720 21:30:04.950900 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˆ queue_depth=711 I0720 21:30:04.950903 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˇ queue_depth=712 I0720 21:30:04.950906 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˈ queue_depth=713 I0720 21:30:04.950909 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˉ queue_depth=714 I0720 21:30:04.950912 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˊ queue_depth=715 I0720 21:30:04.950915 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˋ queue_depth=716 I0720 21:30:04.950918 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˌ queue_depth=717 I0720 21:30:04.950921 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˍ queue_depth=718 I0720 21:30:04.950924 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˎ queue_depth=719 I0720 21:30:04.950927 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˏ queue_depth=720 I0720 21:30:04.950930 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ː queue_depth=721 I0720 21:30:04.950933 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˑ queue_depth=722 I0720 21:30:04.950935 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˒ queue_depth=723 I0720 21:30:04.950938 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˓ queue_depth=724 I0720 21:30:04.950941 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˔ queue_depth=725 I0720 21:30:04.950944 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˕ queue_depth=726 I0720 21:30:04.950946 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˖ queue_depth=727 I0720 21:30:04.950949 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˗ queue_depth=728 I0720 21:30:04.950952 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˘ queue_depth=729 I0720 21:30:04.950955 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˙ queue_depth=730 I0720 21:30:04.950957 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˚ queue_depth=731 I0720 21:30:04.950960 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˛ queue_depth=732 I0720 21:30:04.950963 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˜ queue_depth=733 I0720 21:30:04.950966 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˝ queue_depth=734 I0720 21:30:04.950969 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˞ queue_depth=735 I0720 21:30:04.950972 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˟ queue_depth=736 I0720 21:30:04.950975 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˠ queue_depth=737 I0720 21:30:04.950978 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˡ queue_depth=738 I0720 21:30:04.950981 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˢ queue_depth=739 I0720 21:30:04.950984 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˣ queue_depth=740 I0720 21:30:04.950987 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˤ queue_depth=741 I0720 21:30:04.950990 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˥ queue_depth=742 I0720 21:30:04.950994 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˦ queue_depth=743 I0720 21:30:04.950997 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˧ queue_depth=744 I0720 21:30:04.951000 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˨ queue_depth=745 I0720 21:30:04.951003 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˩ queue_depth=746 I0720 21:30:04.951006 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˪ queue_depth=747 I0720 21:30:04.951008 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˫ queue_depth=748 I0720 21:30:04.951011 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˬ queue_depth=749 I0720 21:30:04.951014 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˭ queue_depth=750 I0720 21:30:04.951017 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ˮ queue_depth=751 I0720 21:30:04.951020 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˯ queue_depth=752 I0720 21:30:04.951023 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˰ queue_depth=753 I0720 21:30:04.951025 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˱ queue_depth=754 I0720 21:30:04.951028 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˲ queue_depth=755 I0720 21:30:04.951031 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˳ queue_depth=756 I0720 21:30:04.951034 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˴ queue_depth=757 I0720 21:30:04.951037 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˵ queue_depth=758 I0720 21:30:04.951040 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˶ queue_depth=759 I0720 21:30:04.951043 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˷ queue_depth=760 I0720 21:30:04.951046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˸ queue_depth=761 I0720 21:30:04.951049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˹ queue_depth=762 I0720 21:30:04.951051 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˺ queue_depth=763 I0720 21:30:04.951055 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˻ queue_depth=764 I0720 21:30:04.951057 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˼ queue_depth=765 I0720 21:30:04.951060 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˽ queue_depth=766 I0720 21:30:04.951063 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˾ queue_depth=767 I0720 21:30:04.951066 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=˿ queue_depth=768 I0720 21:30:04.951069 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̀ queue_depth=769 I0720 21:30:04.951072 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=́ queue_depth=770 I0720 21:30:04.951075 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̂ queue_depth=771 I0720 21:30:04.951078 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̃ queue_depth=772 I0720 21:30:04.951080 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̄ queue_depth=773 I0720 21:30:04.951083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̅ queue_depth=774 I0720 21:30:04.951086 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̆ queue_depth=775 I0720 21:30:04.951089 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̇ queue_depth=776 I0720 21:30:04.951092 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̈ queue_depth=777 I0720 21:30:04.951095 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̉ queue_depth=778 I0720 21:30:04.951098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̊ queue_depth=779 I0720 21:30:04.951101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̋ queue_depth=780 I0720 21:30:04.951104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̌ queue_depth=781 I0720 21:30:04.951107 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̍ queue_depth=782 I0720 21:30:04.951110 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̎ queue_depth=783 I0720 21:30:04.951113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̏ queue_depth=784 I0720 21:30:04.951115 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̐ queue_depth=785 I0720 21:30:04.951118 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̑ queue_depth=786 I0720 21:30:04.951121 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̒ queue_depth=787 I0720 21:30:04.951125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̓ queue_depth=788 I0720 21:30:04.951128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̔ queue_depth=789 I0720 21:30:04.951131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̕ queue_depth=790 I0720 21:30:04.951134 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̖ queue_depth=791 I0720 21:30:04.951136 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̗ queue_depth=792 I0720 21:30:04.951139 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̘ queue_depth=793 I0720 21:30:04.951142 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̙ queue_depth=794 I0720 21:30:04.951145 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̚ queue_depth=795 I0720 21:30:04.951148 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̛ queue_depth=796 I0720 21:30:04.951151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̜ queue_depth=797 I0720 21:30:04.951154 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̝ queue_depth=798 I0720 21:30:04.951160 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̞ queue_depth=799 I0720 21:30:04.951164 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̟ queue_depth=800 I0720 21:30:04.951167 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̠ queue_depth=801 I0720 21:30:04.951170 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̡ queue_depth=802 I0720 21:30:04.951173 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̢ queue_depth=803 I0720 21:30:04.951176 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̣ queue_depth=804 I0720 21:30:04.951179 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̤ queue_depth=805 I0720 21:30:04.951182 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̥ queue_depth=806 I0720 21:30:04.951184 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̦ queue_depth=807 I0720 21:30:04.951187 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̧ queue_depth=808 I0720 21:30:04.951190 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̨ queue_depth=809 I0720 21:30:04.951193 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̩ queue_depth=810 I0720 21:30:04.951197 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̪ queue_depth=811 I0720 21:30:04.951200 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̫ queue_depth=812 I0720 21:30:04.951203 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̬ queue_depth=813 I0720 21:30:04.951205 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̭ queue_depth=814 I0720 21:30:04.951208 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̮ queue_depth=815 I0720 21:30:04.951211 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̯ queue_depth=816 I0720 21:30:04.951214 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̰ queue_depth=817 I0720 21:30:04.951217 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̱ queue_depth=818 I0720 21:30:04.951219 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̲ queue_depth=819 I0720 21:30:04.951222 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̳ queue_depth=820 I0720 21:30:04.951226 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̴ queue_depth=821 I0720 21:30:04.951229 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̵ queue_depth=822 I0720 21:30:04.951231 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̶ queue_depth=823 I0720 21:30:04.951234 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̷ queue_depth=824 I0720 21:30:04.951237 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key≠ queue_depth=825 I0720 21:30:04.951240 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̹ queue_depth=826 I0720 21:30:04.951243 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̺ queue_depth=827 I0720 21:30:04.951245 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̻ queue_depth=828 I0720 21:30:04.951248 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̼ queue_depth=829 I0720 21:30:04.951251 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̽ queue_depth=830 I0720 21:30:04.951254 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̾ queue_depth=831 I0720 21:30:04.951257 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̿ queue_depth=832 I0720 21:30:04.951260 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̀ queue_depth=833 I0720 21:30:04.951262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=́ queue_depth=834 I0720 21:30:04.951266 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͂ queue_depth=835 I0720 21:30:04.951269 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̓ queue_depth=836 I0720 21:30:04.951272 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=̈́ queue_depth=837 I0720 21:30:04.951274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͅ queue_depth=838 I0720 21:30:04.951277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͆ queue_depth=839 I0720 21:30:04.951280 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͇ queue_depth=840 I0720 21:30:04.951283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͈ queue_depth=841 I0720 21:30:04.951286 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͉ queue_depth=842 I0720 21:30:04.951289 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͊ queue_depth=843 I0720 21:30:04.951292 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͋ queue_depth=844 I0720 21:30:04.951296 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͌ queue_depth=845 I0720 21:30:04.951299 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͍ queue_depth=846 I0720 21:30:04.951302 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͎ queue_depth=847 I0720 21:30:04.951304 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͏ queue_depth=848 I0720 21:30:04.951307 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͐ queue_depth=849 I0720 21:30:04.951310 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͑ queue_depth=850 I0720 21:30:04.951313 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͒ queue_depth=851 I0720 21:30:04.951316 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͓ queue_depth=852 I0720 21:30:04.951319 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͔ queue_depth=853 I0720 21:30:04.951322 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͕ queue_depth=854 I0720 21:30:04.951324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͖ queue_depth=855 I0720 21:30:04.951327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͗ queue_depth=856 I0720 21:30:04.951330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͘ queue_depth=857 I0720 21:30:04.951333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͙ queue_depth=858 I0720 21:30:04.951335 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͚ queue_depth=859 I0720 21:30:04.951338 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͛ queue_depth=860 I0720 21:30:04.951341 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͜ queue_depth=861 I0720 21:30:04.951344 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͝ queue_depth=862 I0720 21:30:04.951347 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͞ queue_depth=863 I0720 21:30:04.951350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͟ queue_depth=864 I0720 21:30:04.951352 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͠ queue_depth=865 I0720 21:30:04.951355 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͡ queue_depth=866 I0720 21:30:04.951358 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͢ queue_depth=867 I0720 21:30:04.951361 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͣ queue_depth=868 I0720 21:30:04.951364 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͤ queue_depth=869 I0720 21:30:04.951369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͥ queue_depth=870 I0720 21:30:04.951372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͦ queue_depth=871 I0720 21:30:04.951375 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͧ queue_depth=872 I0720 21:30:04.951377 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͨ queue_depth=873 I0720 21:30:04.951380 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͩ queue_depth=874 I0720 21:30:04.951383 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͪ queue_depth=875 I0720 21:30:04.951386 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͫ queue_depth=876 I0720 21:30:04.951389 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͬ queue_depth=877 I0720 21:30:04.951392 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͭ queue_depth=878 I0720 21:30:04.951396 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͮ queue_depth=879 I0720 21:30:04.951399 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͯ queue_depth=880 I0720 21:30:04.951402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ͱ queue_depth=881 I0720 21:30:04.951405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͱ queue_depth=882 I0720 21:30:04.951408 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ͳ queue_depth=883 I0720 21:30:04.951410 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͳ queue_depth=884 I0720 21:30:04.951413 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ʹ queue_depth=885 I0720 21:30:04.951416 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͵ queue_depth=886 I0720 21:30:04.951419 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ͷ queue_depth=887 I0720 21:30:04.951422 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͷ queue_depth=888 I0720 21:30:04.951425 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͸ queue_depth=889 I0720 21:30:04.951427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=͹ queue_depth=890 I0720 21:30:04.951430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͺ queue_depth=891 I0720 21:30:04.951433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͻ queue_depth=892 I0720 21:30:04.951436 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͼ queue_depth=893 I0720 21:30:04.951439 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ͽ queue_depth=894 I0720 21:30:04.951441 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=; queue_depth=895 I0720 21:30:04.951445 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϳ queue_depth=896 I0720 21:30:04.951496 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΀ queue_depth=897 I0720 21:30:04.951498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΁ queue_depth=898 I0720 21:30:04.951501 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΂ queue_depth=899 I0720 21:30:04.951504 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΃ queue_depth=900 I0720 21:30:04.951508 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΄ queue_depth=901 I0720 21:30:04.951510 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΅ queue_depth=902 I0720 21:30:04.951513 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ά queue_depth=903 I0720 21:30:04.951516 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=· queue_depth=904 I0720 21:30:04.951519 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Έ queue_depth=905 I0720 21:30:04.951521 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ή queue_depth=906 I0720 21:30:04.951524 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ί queue_depth=907 I0720 21:30:04.951528 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΋ queue_depth=908 I0720 21:30:04.951531 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ό queue_depth=909 I0720 21:30:04.951533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΍ queue_depth=910 I0720 21:30:04.951536 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ύ queue_depth=911 I0720 21:30:04.951539 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ώ queue_depth=912 I0720 21:30:04.951543 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ΐ queue_depth=913 I0720 21:30:04.951545 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Α queue_depth=914 I0720 21:30:04.951548 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Β queue_depth=915 I0720 21:30:04.951551 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Γ queue_depth=916 I0720 21:30:04.951554 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Δ queue_depth=917 I0720 21:30:04.951557 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ε queue_depth=918 I0720 21:30:04.951559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ζ queue_depth=919 I0720 21:30:04.951562 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Η queue_depth=920 I0720 21:30:04.951565 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Θ queue_depth=921 I0720 21:30:04.951568 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ι queue_depth=922 I0720 21:30:04.951571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Κ queue_depth=923 I0720 21:30:04.951573 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Λ queue_depth=924 I0720 21:30:04.951576 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Μ queue_depth=925 I0720 21:30:04.951579 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ν queue_depth=926 I0720 21:30:04.951582 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ξ queue_depth=927 I0720 21:30:04.951585 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ο queue_depth=928 I0720 21:30:04.951588 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Π queue_depth=929 I0720 21:30:04.951591 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ρ queue_depth=930 I0720 21:30:04.951593 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=΢ queue_depth=931 I0720 21:30:04.951596 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Σ queue_depth=932 I0720 21:30:04.951599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Τ queue_depth=933 I0720 21:30:04.951602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Υ queue_depth=934 I0720 21:30:04.951605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Φ queue_depth=935 I0720 21:30:04.951608 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Χ queue_depth=936 I0720 21:30:04.951611 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ψ queue_depth=937 I0720 21:30:04.951614 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ω queue_depth=938 I0720 21:30:04.951616 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϊ queue_depth=939 I0720 21:30:04.951619 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϋ queue_depth=940 I0720 21:30:04.951622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ά queue_depth=941 I0720 21:30:04.951625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=έ queue_depth=942 I0720 21:30:04.951628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ή queue_depth=943 I0720 21:30:04.951631 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ί queue_depth=944 I0720 21:30:04.951633 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ΰ queue_depth=945 I0720 21:30:04.951636 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=α queue_depth=946 I0720 21:30:04.951640 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=β queue_depth=947 I0720 21:30:04.951643 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=γ queue_depth=948 I0720 21:30:04.951645 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=δ queue_depth=949 I0720 21:30:04.951648 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ε queue_depth=950 I0720 21:30:04.951651 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ζ queue_depth=951 I0720 21:30:04.951654 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=η queue_depth=952 I0720 21:30:04.951657 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=θ queue_depth=953 I0720 21:30:04.951660 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ι queue_depth=954 I0720 21:30:04.951662 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=κ queue_depth=955 I0720 21:30:04.951665 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=λ queue_depth=956 I0720 21:30:04.951668 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=μ queue_depth=957 I0720 21:30:04.951671 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ν queue_depth=958 I0720 21:30:04.951674 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ξ queue_depth=959 I0720 21:30:04.951676 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ο queue_depth=960 I0720 21:30:04.951679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=π queue_depth=961 I0720 21:30:04.951682 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ρ queue_depth=962 I0720 21:30:04.951685 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ς queue_depth=963 I0720 21:30:04.951688 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=σ queue_depth=964 I0720 21:30:04.951691 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=τ queue_depth=965 I0720 21:30:04.951694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=υ queue_depth=966 I0720 21:30:04.951697 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=φ queue_depth=967 I0720 21:30:04.951700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=χ queue_depth=968 I0720 21:30:04.951703 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ψ queue_depth=969 I0720 21:30:04.951706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ω queue_depth=970 I0720 21:30:04.951708 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϊ queue_depth=971 I0720 21:30:04.951711 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϋ queue_depth=972 I0720 21:30:04.951713 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ό queue_depth=973 I0720 21:30:04.951716 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ύ queue_depth=974 I0720 21:30:04.951719 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ώ queue_depth=975 I0720 21:30:04.951722 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϗ queue_depth=976 I0720 21:30:04.951725 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϐ queue_depth=977 I0720 21:30:04.951728 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϑ queue_depth=978 I0720 21:30:04.951731 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϒ queue_depth=979 I0720 21:30:04.951734 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϓ queue_depth=980 I0720 21:30:04.951737 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϔ queue_depth=981 I0720 21:30:04.951740 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϕ queue_depth=982 I0720 21:30:04.951743 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϖ queue_depth=983 I0720 21:30:04.951746 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϗ queue_depth=984 I0720 21:30:04.951749 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϙ queue_depth=985 I0720 21:30:04.951751 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϙ queue_depth=986 I0720 21:30:04.951754 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϛ queue_depth=987 I0720 21:30:04.951757 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϛ queue_depth=988 I0720 21:30:04.951760 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϝ queue_depth=989 I0720 21:30:04.951763 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϝ queue_depth=990 I0720 21:30:04.951766 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϟ queue_depth=991 I0720 21:30:04.951769 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϟ queue_depth=992 I0720 21:30:04.951771 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϡ queue_depth=993 I0720 21:30:04.951774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϡ queue_depth=994 I0720 21:30:04.951777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϣ queue_depth=995 I0720 21:30:04.951780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϣ queue_depth=996 I0720 21:30:04.951783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϥ queue_depth=997 I0720 21:30:04.951785 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϥ queue_depth=998 I0720 21:30:04.951788 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϧ queue_depth=999 I0720 21:30:04.951791 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϧ queue_depth=1000 I0720 21:30:04.951794 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϩ queue_depth=1001 I0720 21:30:04.951797 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϩ queue_depth=1002 I0720 21:30:04.951800 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϫ queue_depth=1003 I0720 21:30:04.951803 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϫ queue_depth=1004 I0720 21:30:04.951806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϭ queue_depth=1005 I0720 21:30:04.951809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϭ queue_depth=1006 I0720 21:30:04.951812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϯ queue_depth=1007 I0720 21:30:04.951814 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϯ queue_depth=1008 I0720 21:30:04.951817 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϰ queue_depth=1009 I0720 21:30:04.951820 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϱ queue_depth=1010 I0720 21:30:04.951823 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϲ queue_depth=1011 I0720 21:30:04.951826 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϳ queue_depth=1012 I0720 21:30:04.951829 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϴ queue_depth=1013 I0720 21:30:04.951832 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϵ queue_depth=1014 I0720 21:30:04.951835 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=϶ queue_depth=1015 I0720 21:30:04.951838 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϸ queue_depth=1016 I0720 21:30:04.951841 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϸ queue_depth=1017 I0720 21:30:04.951844 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϲ queue_depth=1018 I0720 21:30:04.951847 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ϻ queue_depth=1019 I0720 21:30:04.951849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϻ queue_depth=1020 I0720 21:30:04.951852 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ϼ queue_depth=1021 I0720 21:30:04.951855 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ͻ queue_depth=1022 I0720 21:30:04.951858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ͼ queue_depth=1023 I0720 21:30:04.951861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ͽ queue_depth=1024 I0720 21:30:04.951864 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѐ queue_depth=1025 I0720 21:30:04.951867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ё queue_depth=1026 I0720 21:30:04.951870 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ђ queue_depth=1027 I0720 21:30:04.951873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѓ queue_depth=1028 I0720 21:30:04.951876 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Є queue_depth=1029 I0720 21:30:04.951878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѕ queue_depth=1030 I0720 21:30:04.951881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=І queue_depth=1031 I0720 21:30:04.951884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ї queue_depth=1032 I0720 21:30:04.951887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ј queue_depth=1033 I0720 21:30:04.951890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Љ queue_depth=1034 I0720 21:30:04.951893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Њ queue_depth=1035 I0720 21:30:04.951896 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ћ queue_depth=1036 I0720 21:30:04.951899 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ќ queue_depth=1037 I0720 21:30:04.951902 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѝ queue_depth=1038 I0720 21:30:04.951905 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ў queue_depth=1039 I0720 21:30:04.951908 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Џ queue_depth=1040 I0720 21:30:04.951911 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=А queue_depth=1041 I0720 21:30:04.951913 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Б queue_depth=1042 I0720 21:30:04.951916 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=В queue_depth=1043 I0720 21:30:04.951919 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Г queue_depth=1044 I0720 21:30:04.951921 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Д queue_depth=1045 I0720 21:30:04.951924 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Е queue_depth=1046 I0720 21:30:04.951927 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ж queue_depth=1047 I0720 21:30:04.951930 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=З queue_depth=1048 I0720 21:30:04.951933 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=И queue_depth=1049 I0720 21:30:04.951936 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Й queue_depth=1050 I0720 21:30:04.951939 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=К queue_depth=1051 I0720 21:30:04.951942 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Л queue_depth=1052 I0720 21:30:04.951945 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=М queue_depth=1053 I0720 21:30:04.951948 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Н queue_depth=1054 I0720 21:30:04.951950 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=О queue_depth=1055 I0720 21:30:04.951953 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=П queue_depth=1056 I0720 21:30:04.951956 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Р queue_depth=1057 I0720 21:30:04.951959 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=С queue_depth=1058 I0720 21:30:04.951962 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Т queue_depth=1059 I0720 21:30:04.951965 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=У queue_depth=1060 I0720 21:30:04.951968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ф queue_depth=1061 I0720 21:30:04.951970 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Х queue_depth=1062 I0720 21:30:04.951973 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ц queue_depth=1063 I0720 21:30:04.951976 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ч queue_depth=1064 I0720 21:30:04.951979 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ш queue_depth=1065 I0720 21:30:04.951982 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Щ queue_depth=1066 I0720 21:30:04.951985 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ъ queue_depth=1067 I0720 21:30:04.951988 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ы queue_depth=1068 I0720 21:30:04.951991 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ь queue_depth=1069 I0720 21:30:04.951994 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Э queue_depth=1070 I0720 21:30:04.951997 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ю queue_depth=1071 I0720 21:30:04.952000 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Я queue_depth=1072 I0720 21:30:04.952002 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=а queue_depth=1073 I0720 21:30:04.952005 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=б queue_depth=1074 I0720 21:30:04.952008 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=в queue_depth=1075 I0720 21:30:04.952011 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=г queue_depth=1076 I0720 21:30:04.952014 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=д queue_depth=1077 I0720 21:30:04.952017 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=е queue_depth=1078 I0720 21:30:04.952020 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ж queue_depth=1079 I0720 21:30:04.952022 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=з queue_depth=1080 I0720 21:30:04.952026 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=и queue_depth=1081 I0720 21:30:04.952029 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=й queue_depth=1082 I0720 21:30:04.952032 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=к queue_depth=1083 I0720 21:30:04.952034 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=л queue_depth=1084 I0720 21:30:04.952037 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=м queue_depth=1085 I0720 21:30:04.952040 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=н queue_depth=1086 I0720 21:30:04.952043 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=о queue_depth=1087 I0720 21:30:04.952046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=п queue_depth=1088 I0720 21:30:04.952049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=р queue_depth=1089 I0720 21:30:04.952051 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=с queue_depth=1090 I0720 21:30:04.952054 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=т queue_depth=1091 I0720 21:30:04.952057 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=у queue_depth=1092 I0720 21:30:04.952060 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ф queue_depth=1093 I0720 21:30:04.952063 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=х queue_depth=1094 I0720 21:30:04.952065 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ц queue_depth=1095 I0720 21:30:04.952068 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ч queue_depth=1096 I0720 21:30:04.952071 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ш queue_depth=1097 I0720 21:30:04.952074 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=щ queue_depth=1098 I0720 21:30:04.952077 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ъ queue_depth=1099 I0720 21:30:04.952080 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ы queue_depth=1100 I0720 21:30:04.952083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ь queue_depth=1101 I0720 21:30:04.952086 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=э queue_depth=1102 I0720 21:30:04.952088 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ю queue_depth=1103 I0720 21:30:04.952091 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=я queue_depth=1104 I0720 21:30:04.952094 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѐ queue_depth=1105 I0720 21:30:04.952097 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ё queue_depth=1106 I0720 21:30:04.952100 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ђ queue_depth=1107 I0720 21:30:04.952102 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѓ queue_depth=1108 I0720 21:30:04.952105 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=є queue_depth=1109 I0720 21:30:04.952108 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѕ queue_depth=1110 I0720 21:30:04.952110 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=і queue_depth=1111 I0720 21:30:04.952113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ї queue_depth=1112 I0720 21:30:04.952116 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ј queue_depth=1113 I0720 21:30:04.952120 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=љ queue_depth=1114 I0720 21:30:04.952122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=њ queue_depth=1115 I0720 21:30:04.952125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ћ queue_depth=1116 I0720 21:30:04.952128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ќ queue_depth=1117 I0720 21:30:04.952132 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѝ queue_depth=1118 I0720 21:30:04.952134 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ў queue_depth=1119 I0720 21:30:04.952137 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=џ queue_depth=1120 I0720 21:30:04.952140 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѡ queue_depth=1121 I0720 21:30:04.952143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѡ queue_depth=1122 I0720 21:30:04.952146 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѣ queue_depth=1123 I0720 21:30:04.952148 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѣ queue_depth=1124 I0720 21:30:04.952151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѥ queue_depth=1125 I0720 21:30:04.952154 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѥ queue_depth=1126 I0720 21:30:04.952159 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѧ queue_depth=1127 I0720 21:30:04.952164 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѧ queue_depth=1128 I0720 21:30:04.952167 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѩ queue_depth=1129 I0720 21:30:04.952170 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѩ queue_depth=1130 I0720 21:30:04.952172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѫ queue_depth=1131 I0720 21:30:04.952176 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѫ queue_depth=1132 I0720 21:30:04.952179 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѭ queue_depth=1133 I0720 21:30:04.952182 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѭ queue_depth=1134 I0720 21:30:04.952185 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѯ queue_depth=1135 I0720 21:30:04.952188 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѯ queue_depth=1136 I0720 21:30:04.952190 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѱ queue_depth=1137 I0720 21:30:04.952193 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѱ queue_depth=1138 I0720 21:30:04.952196 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѳ queue_depth=1139 I0720 21:30:04.952199 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѳ queue_depth=1140 I0720 21:30:04.952202 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѵ queue_depth=1141 I0720 21:30:04.952204 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѵ queue_depth=1142 I0720 21:30:04.952207 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѷ queue_depth=1143 I0720 21:30:04.952210 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѷ queue_depth=1144 I0720 21:30:04.952213 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѹ queue_depth=1145 I0720 21:30:04.952215 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѹ queue_depth=1146 I0720 21:30:04.952250 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѻ queue_depth=1147 I0720 21:30:04.952253 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѻ queue_depth=1148 I0720 21:30:04.952256 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѽ queue_depth=1149 I0720 21:30:04.952259 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѽ queue_depth=1150 I0720 21:30:04.952262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ѿ queue_depth=1151 I0720 21:30:04.952265 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ѿ queue_depth=1152 I0720 21:30:04.952268 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҁ queue_depth=1153 I0720 21:30:04.952271 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҁ queue_depth=1154 I0720 21:30:04.952274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҂ queue_depth=1155 I0720 21:30:04.952277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҃ queue_depth=1156 I0720 21:30:04.952279 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҄ queue_depth=1157 I0720 21:30:04.952282 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҅ queue_depth=1158 I0720 21:30:04.952285 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҆ queue_depth=1159 I0720 21:30:04.952288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҇ queue_depth=1160 I0720 21:30:04.952291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҈ queue_depth=1161 I0720 21:30:04.952294 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=҉ queue_depth=1162 I0720 21:30:04.952297 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҋ queue_depth=1163 I0720 21:30:04.952301 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҋ queue_depth=1164 I0720 21:30:04.952304 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҍ queue_depth=1165 I0720 21:30:04.952307 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҍ queue_depth=1166 I0720 21:30:04.952310 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҏ queue_depth=1167 I0720 21:30:04.952313 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҏ queue_depth=1168 I0720 21:30:04.952315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ґ queue_depth=1169 I0720 21:30:04.952319 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ґ queue_depth=1170 I0720 21:30:04.952321 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ғ queue_depth=1171 I0720 21:30:04.952324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ғ queue_depth=1172 I0720 21:30:04.952327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҕ queue_depth=1173 I0720 21:30:04.952330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҕ queue_depth=1174 I0720 21:30:04.952333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Җ queue_depth=1175 I0720 21:30:04.952336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=җ queue_depth=1176 I0720 21:30:04.952339 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҙ queue_depth=1177 I0720 21:30:04.952341 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҙ queue_depth=1178 I0720 21:30:04.952344 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Қ queue_depth=1179 I0720 21:30:04.952347 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=қ queue_depth=1180 I0720 21:30:04.952350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҝ queue_depth=1181 I0720 21:30:04.952353 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҝ queue_depth=1182 I0720 21:30:04.952356 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҟ queue_depth=1183 I0720 21:30:04.952359 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҟ queue_depth=1184 I0720 21:30:04.952362 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҡ queue_depth=1185 I0720 21:30:04.952365 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҡ queue_depth=1186 I0720 21:30:04.952367 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ң queue_depth=1187 I0720 21:30:04.952370 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ң queue_depth=1188 I0720 21:30:04.952373 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҥ queue_depth=1189 I0720 21:30:04.952376 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҥ queue_depth=1190 I0720 21:30:04.952379 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҧ queue_depth=1191 I0720 21:30:04.952382 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҧ queue_depth=1192 I0720 21:30:04.952384 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҩ queue_depth=1193 I0720 21:30:04.952387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҩ queue_depth=1194 I0720 21:30:04.952390 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҫ queue_depth=1195 I0720 21:30:04.952393 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҫ queue_depth=1196 I0720 21:30:04.952396 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҭ queue_depth=1197 I0720 21:30:04.952399 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҭ queue_depth=1198 I0720 21:30:04.952402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ү queue_depth=1199 I0720 21:30:04.952405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ү queue_depth=1200 I0720 21:30:04.952408 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ұ queue_depth=1201 I0720 21:30:04.952412 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ұ queue_depth=1202 I0720 21:30:04.952415 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҳ queue_depth=1203 I0720 21:30:04.952417 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҳ queue_depth=1204 I0720 21:30:04.952420 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҵ queue_depth=1205 I0720 21:30:04.952423 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҵ queue_depth=1206 I0720 21:30:04.952427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҷ queue_depth=1207 I0720 21:30:04.952430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҷ queue_depth=1208 I0720 21:30:04.952432 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҹ queue_depth=1209 I0720 21:30:04.952435 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҹ queue_depth=1210 I0720 21:30:04.952438 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Һ queue_depth=1211 I0720 21:30:04.952441 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=һ queue_depth=1212 I0720 21:30:04.952445 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҽ queue_depth=1213 I0720 21:30:04.952448 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҽ queue_depth=1214 I0720 21:30:04.952450 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ҿ queue_depth=1215 I0720 21:30:04.952453 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ҿ queue_depth=1216 I0720 21:30:04.952456 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӏ queue_depth=1217 I0720 21:30:04.952459 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӂ queue_depth=1218 I0720 21:30:04.952462 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӂ queue_depth=1219 I0720 21:30:04.952465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӄ queue_depth=1220 I0720 21:30:04.952468 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӄ queue_depth=1221 I0720 21:30:04.952470 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӆ queue_depth=1222 I0720 21:30:04.952473 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӆ queue_depth=1223 I0720 21:30:04.952476 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӈ queue_depth=1224 I0720 21:30:04.952478 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӈ queue_depth=1225 I0720 21:30:04.952481 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӊ queue_depth=1226 I0720 21:30:04.952484 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӊ queue_depth=1227 I0720 21:30:04.952487 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӌ queue_depth=1228 I0720 21:30:04.952490 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӌ queue_depth=1229 I0720 21:30:04.952492 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӎ queue_depth=1230 I0720 21:30:04.952496 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӎ queue_depth=1231 I0720 21:30:04.952499 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӏ queue_depth=1232 I0720 21:30:04.952502 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӑ queue_depth=1233 I0720 21:30:04.952505 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӑ queue_depth=1234 I0720 21:30:04.952508 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӓ queue_depth=1235 I0720 21:30:04.952510 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӓ queue_depth=1236 I0720 21:30:04.952513 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӕ queue_depth=1237 I0720 21:30:04.952516 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӕ queue_depth=1238 I0720 21:30:04.952519 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӗ queue_depth=1239 I0720 21:30:04.952522 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӗ queue_depth=1240 I0720 21:30:04.952525 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ә queue_depth=1241 I0720 21:30:04.952527 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ә queue_depth=1242 I0720 21:30:04.952530 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӛ queue_depth=1243 I0720 21:30:04.952533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӛ queue_depth=1244 I0720 21:30:04.952536 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӝ queue_depth=1245 I0720 21:30:04.952539 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӝ queue_depth=1246 I0720 21:30:04.952542 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӟ queue_depth=1247 I0720 21:30:04.952545 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӟ queue_depth=1248 I0720 21:30:04.952548 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӡ queue_depth=1249 I0720 21:30:04.952551 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӡ queue_depth=1250 I0720 21:30:04.952553 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӣ queue_depth=1251 I0720 21:30:04.952556 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӣ queue_depth=1252 I0720 21:30:04.952559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӥ queue_depth=1253 I0720 21:30:04.952562 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӥ queue_depth=1254 I0720 21:30:04.952565 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӧ queue_depth=1255 I0720 21:30:04.952568 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӧ queue_depth=1256 I0720 21:30:04.952571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ө queue_depth=1257 I0720 21:30:04.952573 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ө queue_depth=1258 I0720 21:30:04.952576 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӫ queue_depth=1259 I0720 21:30:04.952579 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӫ queue_depth=1260 I0720 21:30:04.952582 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӭ queue_depth=1261 I0720 21:30:04.952585 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӭ queue_depth=1262 I0720 21:30:04.952588 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӯ queue_depth=1263 I0720 21:30:04.952591 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӯ queue_depth=1264 I0720 21:30:04.952594 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӱ queue_depth=1265 I0720 21:30:04.952597 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӱ queue_depth=1266 I0720 21:30:04.952600 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӳ queue_depth=1267 I0720 21:30:04.952603 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӳ queue_depth=1268 I0720 21:30:04.952605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӵ queue_depth=1269 I0720 21:30:04.952609 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӵ queue_depth=1270 I0720 21:30:04.952611 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӷ queue_depth=1271 I0720 21:30:04.952614 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӷ queue_depth=1272 I0720 21:30:04.952617 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӹ queue_depth=1273 I0720 21:30:04.952620 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӹ queue_depth=1274 I0720 21:30:04.952623 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӻ queue_depth=1275 I0720 21:30:04.952626 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӻ queue_depth=1276 I0720 21:30:04.952628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӽ queue_depth=1277 I0720 21:30:04.952631 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӽ queue_depth=1278 I0720 21:30:04.952635 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ӿ queue_depth=1279 I0720 21:30:04.952637 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ӿ queue_depth=1280 I0720 21:30:04.952640 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԁ queue_depth=1281 I0720 21:30:04.952643 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԁ queue_depth=1282 I0720 21:30:04.952646 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԃ queue_depth=1283 I0720 21:30:04.952649 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԃ queue_depth=1284 I0720 21:30:04.952652 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԅ queue_depth=1285 I0720 21:30:04.952655 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԅ queue_depth=1286 I0720 21:30:04.952658 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԇ queue_depth=1287 I0720 21:30:04.952661 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԇ queue_depth=1288 I0720 21:30:04.952664 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԉ queue_depth=1289 I0720 21:30:04.952667 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԉ queue_depth=1290 I0720 21:30:04.952670 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԋ queue_depth=1291 I0720 21:30:04.952673 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԋ queue_depth=1292 I0720 21:30:04.952675 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԍ queue_depth=1293 I0720 21:30:04.952678 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԍ queue_depth=1294 I0720 21:30:04.952681 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԏ queue_depth=1295 I0720 21:30:04.952684 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԏ queue_depth=1296 I0720 21:30:04.952687 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԑ queue_depth=1297 I0720 21:30:04.952690 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԑ queue_depth=1298 I0720 21:30:04.952693 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԓ queue_depth=1299 I0720 21:30:04.952696 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԓ queue_depth=1300 I0720 21:30:04.952698 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԕ queue_depth=1301 I0720 21:30:04.952701 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԕ queue_depth=1302 I0720 21:30:04.952704 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԗ queue_depth=1303 I0720 21:30:04.952707 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԗ queue_depth=1304 I0720 21:30:04.952710 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԙ queue_depth=1305 I0720 21:30:04.952713 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԙ queue_depth=1306 I0720 21:30:04.952716 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԛ queue_depth=1307 I0720 21:30:04.952718 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԛ queue_depth=1308 I0720 21:30:04.952721 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԝ queue_depth=1309 I0720 21:30:04.952724 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԝ queue_depth=1310 I0720 21:30:04.952727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԟ queue_depth=1311 I0720 21:30:04.952730 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԟ queue_depth=1312 I0720 21:30:04.952733 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԡ queue_depth=1313 I0720 21:30:04.952736 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԡ queue_depth=1314 I0720 21:30:04.952739 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԣ queue_depth=1315 I0720 21:30:04.952741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԣ queue_depth=1316 I0720 21:30:04.952744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԥ queue_depth=1317 I0720 21:30:04.952747 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԥ queue_depth=1318 I0720 21:30:04.952750 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԧ queue_depth=1319 I0720 21:30:04.952753 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԧ queue_depth=1320 I0720 21:30:04.952756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԩ queue_depth=1321 I0720 21:30:04.952759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԩ queue_depth=1322 I0720 21:30:04.952762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԫ queue_depth=1323 I0720 21:30:04.952765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԫ queue_depth=1324 I0720 21:30:04.952768 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԭ queue_depth=1325 I0720 21:30:04.952771 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԭ queue_depth=1326 I0720 21:30:04.952774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ԯ queue_depth=1327 I0720 21:30:04.952777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ԯ queue_depth=1328 I0720 21:30:04.952780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=԰ queue_depth=1329 I0720 21:30:04.952783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ա queue_depth=1330 I0720 21:30:04.952786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Բ queue_depth=1331 I0720 21:30:04.952789 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Գ queue_depth=1332 I0720 21:30:04.952792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Դ queue_depth=1333 I0720 21:30:04.952795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ե queue_depth=1334 I0720 21:30:04.952797 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Զ queue_depth=1335 I0720 21:30:04.952800 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Է queue_depth=1336 I0720 21:30:04.952803 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ը queue_depth=1337 I0720 21:30:04.952806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Թ queue_depth=1338 I0720 21:30:04.952808 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ժ queue_depth=1339 I0720 21:30:04.952811 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ի queue_depth=1340 I0720 21:30:04.952814 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Լ queue_depth=1341 I0720 21:30:04.952817 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Խ queue_depth=1342 I0720 21:30:04.952820 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ծ queue_depth=1343 I0720 21:30:04.952823 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Կ queue_depth=1344 I0720 21:30:04.952826 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Հ queue_depth=1345 I0720 21:30:04.952829 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ձ queue_depth=1346 I0720 21:30:04.952832 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ղ queue_depth=1347 I0720 21:30:04.952834 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ճ queue_depth=1348 I0720 21:30:04.952837 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Մ queue_depth=1349 I0720 21:30:04.952840 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Յ queue_depth=1350 I0720 21:30:04.952843 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ն queue_depth=1351 I0720 21:30:04.952846 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Շ queue_depth=1352 I0720 21:30:04.952849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ո queue_depth=1353 I0720 21:30:04.952852 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Չ queue_depth=1354 I0720 21:30:04.952854 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Պ queue_depth=1355 I0720 21:30:04.952858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ջ queue_depth=1356 I0720 21:30:04.952861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ռ queue_depth=1357 I0720 21:30:04.952863 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ս queue_depth=1358 I0720 21:30:04.952866 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Վ queue_depth=1359 I0720 21:30:04.952869 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Տ queue_depth=1360 I0720 21:30:04.952872 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ր queue_depth=1361 I0720 21:30:04.952875 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ց queue_depth=1362 I0720 21:30:04.952878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ւ queue_depth=1363 I0720 21:30:04.952881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Փ queue_depth=1364 I0720 21:30:04.952884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ք queue_depth=1365 I0720 21:30:04.952887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Օ queue_depth=1366 I0720 21:30:04.952890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=Ֆ queue_depth=1367 I0720 21:30:04.952893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՗ queue_depth=1368 I0720 21:30:04.952896 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՘ queue_depth=1369 I0720 21:30:04.952899 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ՙ queue_depth=1370 I0720 21:30:04.952902 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՚ queue_depth=1371 I0720 21:30:04.952905 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՛ queue_depth=1372 I0720 21:30:04.952908 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՜ queue_depth=1373 I0720 21:30:04.952911 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՝ queue_depth=1374 I0720 21:30:04.952914 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՞ queue_depth=1375 I0720 21:30:04.952917 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=՟ queue_depth=1376 I0720 21:30:04.952919 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ՠ queue_depth=1377 I0720 21:30:04.952923 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ա queue_depth=1378 I0720 21:30:04.952926 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=բ queue_depth=1379 I0720 21:30:04.952929 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=գ queue_depth=1380 I0720 21:30:04.952931 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=դ queue_depth=1381 I0720 21:30:04.952934 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ե queue_depth=1382 I0720 21:30:04.952937 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=զ queue_depth=1383 I0720 21:30:04.952939 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=է queue_depth=1384 I0720 21:30:04.952942 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ը queue_depth=1385 I0720 21:30:04.952945 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=թ queue_depth=1386 I0720 21:30:04.952948 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ժ queue_depth=1387 I0720 21:30:04.952951 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ի queue_depth=1388 I0720 21:30:04.952954 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=լ queue_depth=1389 I0720 21:30:04.952956 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=խ queue_depth=1390 I0720 21:30:04.952959 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ծ queue_depth=1391 I0720 21:30:04.952962 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=կ queue_depth=1392 I0720 21:30:04.952965 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=հ queue_depth=1393 I0720 21:30:04.952968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ձ queue_depth=1394 I0720 21:30:04.952971 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ղ queue_depth=1395 I0720 21:30:04.952974 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ճ queue_depth=1396 I0720 21:30:04.952977 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=մ queue_depth=1397 I0720 21:30:04.952980 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=յ queue_depth=1398 I0720 21:30:04.952983 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ն queue_depth=1399 I0720 21:30:04.952985 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=շ queue_depth=1400 I0720 21:30:04.952988 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ո queue_depth=1401 I0720 21:30:04.952991 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=չ queue_depth=1402 I0720 21:30:04.952994 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=պ queue_depth=1403 I0720 21:30:04.952996 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ջ queue_depth=1404 I0720 21:30:04.952999 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ռ queue_depth=1405 I0720 21:30:04.953002 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ս queue_depth=1406 I0720 21:30:04.953005 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=վ queue_depth=1407 I0720 21:30:04.953008 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=տ queue_depth=1408 I0720 21:30:04.953010 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ր queue_depth=1409 I0720 21:30:04.953013 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ց queue_depth=1410 I0720 21:30:04.953017 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ւ queue_depth=1411 I0720 21:30:04.953020 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=փ queue_depth=1412 I0720 21:30:04.953024 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ք queue_depth=1413 I0720 21:30:04.953026 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=օ queue_depth=1414 I0720 21:30:04.953029 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֆ queue_depth=1415 I0720 21:30:04.953032 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=և queue_depth=1416 I0720 21:30:04.953035 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֈ queue_depth=1417 I0720 21:30:04.953038 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=։ queue_depth=1418 I0720 21:30:04.953041 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֊ queue_depth=1419 I0720 21:30:04.953043 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֋ queue_depth=1420 I0720 21:30:04.953046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֌ queue_depth=1421 I0720 21:30:04.953049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֍ queue_depth=1422 I0720 21:30:04.953052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֎ queue_depth=1423 I0720 21:30:04.953054 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֏ queue_depth=1424 I0720 21:30:04.953057 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֐ queue_depth=1425 I0720 21:30:04.953060 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֑ queue_depth=1426 I0720 21:30:04.953063 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֒ queue_depth=1427 I0720 21:30:04.953066 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֓ queue_depth=1428 I0720 21:30:04.953069 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֔ queue_depth=1429 I0720 21:30:04.953072 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֕ queue_depth=1430 I0720 21:30:04.953075 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֖ queue_depth=1431 I0720 21:30:04.953078 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֗ queue_depth=1432 I0720 21:30:04.953080 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֘ queue_depth=1433 I0720 21:30:04.953083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֙ queue_depth=1434 I0720 21:30:04.953086 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֚ queue_depth=1435 I0720 21:30:04.953089 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֛ queue_depth=1436 I0720 21:30:04.953092 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֜ queue_depth=1437 I0720 21:30:04.953095 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֝ queue_depth=1438 I0720 21:30:04.953098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֞ queue_depth=1439 I0720 21:30:04.953101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֟ queue_depth=1440 I0720 21:30:04.953103 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֠ queue_depth=1441 I0720 21:30:04.953106 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֡ queue_depth=1442 I0720 21:30:04.953109 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֢ queue_depth=1443 I0720 21:30:04.953113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֣ queue_depth=1444 I0720 21:30:04.953116 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֤ queue_depth=1445 I0720 21:30:04.953119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֥ queue_depth=1446 I0720 21:30:04.953122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֦ queue_depth=1447 I0720 21:30:04.953125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֧ queue_depth=1448 I0720 21:30:04.953128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֨ queue_depth=1449 I0720 21:30:04.953132 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֩ queue_depth=1450 I0720 21:30:04.953134 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֪ queue_depth=1451 I0720 21:30:04.953137 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֫ queue_depth=1452 I0720 21:30:04.953141 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֬ queue_depth=1453 I0720 21:30:04.953144 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֭ queue_depth=1454 I0720 21:30:04.953147 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֮ queue_depth=1455 I0720 21:30:04.953150 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=֯ queue_depth=1456 I0720 21:30:04.953153 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ְ queue_depth=1457 I0720 21:30:04.953156 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֱ queue_depth=1458 I0720 21:30:04.953159 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֲ queue_depth=1459 I0720 21:30:04.953161 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֳ queue_depth=1460 I0720 21:30:04.953164 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ִ queue_depth=1461 I0720 21:30:04.953168 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֵ queue_depth=1462 I0720 21:30:04.953170 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֶ queue_depth=1463 I0720 21:30:04.953173 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ַ queue_depth=1464 I0720 21:30:04.953176 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ָ queue_depth=1465 I0720 21:30:04.953179 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֹ queue_depth=1466 I0720 21:30:04.953182 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֺ queue_depth=1467 I0720 21:30:04.953185 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֻ queue_depth=1468 I0720 21:30:04.953187 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ּ queue_depth=1469 I0720 21:30:04.953190 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֽ queue_depth=1470 I0720 21:30:04.953193 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=־ queue_depth=1471 I0720 21:30:04.953196 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ֿ queue_depth=1472 I0720 21:30:04.953199 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׀ queue_depth=1473 I0720 21:30:04.953202 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ׁ queue_depth=1474 I0720 21:30:04.953205 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ׂ queue_depth=1475 I0720 21:30:04.953208 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׃ queue_depth=1476 I0720 21:30:04.953211 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ׄ queue_depth=1477 I0720 21:30:04.953214 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ׅ queue_depth=1478 I0720 21:30:04.953217 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׆ queue_depth=1479 I0720 21:30:04.953220 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ׇ queue_depth=1480 I0720 21:30:04.953222 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׈ queue_depth=1481 I0720 21:30:04.953225 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׉ queue_depth=1482 I0720 21:30:04.953228 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׊ queue_depth=1483 I0720 21:30:04.953231 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׋ queue_depth=1484 I0720 21:30:04.953234 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׌ queue_depth=1485 I0720 21:30:04.953237 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׍ queue_depth=1486 I0720 21:30:04.953240 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׎ queue_depth=1487 I0720 21:30:04.953243 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׏ queue_depth=1488 I0720 21:30:04.953246 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=א queue_depth=1489 I0720 21:30:04.953248 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ב queue_depth=1490 I0720 21:30:04.953251 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ג queue_depth=1491 I0720 21:30:04.953254 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ד queue_depth=1492 I0720 21:30:04.953257 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ה queue_depth=1493 I0720 21:30:04.953260 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ו queue_depth=1494 I0720 21:30:04.953263 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ז queue_depth=1495 I0720 21:30:04.953266 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ח queue_depth=1496 I0720 21:30:04.953269 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ט queue_depth=1497 I0720 21:30:04.953272 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=י queue_depth=1498 I0720 21:30:04.953275 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ך queue_depth=1499 I0720 21:30:04.953277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=כ queue_depth=1500 I0720 21:30:04.953280 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ל queue_depth=1501 I0720 21:30:04.953283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ם queue_depth=1502 I0720 21:30:04.953286 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=מ queue_depth=1503 I0720 21:30:04.953289 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ן queue_depth=1504 I0720 21:30:04.953292 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=נ queue_depth=1505 I0720 21:30:04.953294 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ס queue_depth=1506 I0720 21:30:04.953297 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ע queue_depth=1507 I0720 21:30:04.953300 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ף queue_depth=1508 I0720 21:30:04.953303 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=פ queue_depth=1509 I0720 21:30:04.953306 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ץ queue_depth=1510 I0720 21:30:04.953309 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=צ queue_depth=1511 I0720 21:30:04.953312 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ק queue_depth=1512 I0720 21:30:04.953315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ר queue_depth=1513 I0720 21:30:04.953318 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ש queue_depth=1514 I0720 21:30:04.953321 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ת queue_depth=1515 I0720 21:30:04.953324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׫ queue_depth=1516 I0720 21:30:04.953327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׬ queue_depth=1517 I0720 21:30:04.953330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׭ queue_depth=1518 I0720 21:30:04.953333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׮ queue_depth=1519 I0720 21:30:04.953336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ׯ queue_depth=1520 I0720 21:30:04.953338 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=װ queue_depth=1521 I0720 21:30:04.953341 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ױ queue_depth=1522 I0720 21:30:04.953344 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ײ queue_depth=1523 I0720 21:30:04.953347 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׳ queue_depth=1524 I0720 21:30:04.953350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=״ queue_depth=1525 I0720 21:30:04.953353 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׵ queue_depth=1526 I0720 21:30:04.953355 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׶ queue_depth=1527 I0720 21:30:04.953359 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׷ queue_depth=1528 I0720 21:30:04.953362 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׸ queue_depth=1529 I0720 21:30:04.953365 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׹ queue_depth=1530 I0720 21:30:04.953368 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׺ queue_depth=1531 I0720 21:30:04.953371 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׻ queue_depth=1532 I0720 21:30:04.953374 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׼ queue_depth=1533 I0720 21:30:04.953376 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׽ queue_depth=1534 I0720 21:30:04.953379 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׾ queue_depth=1535 I0720 21:30:04.953382 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=׿ queue_depth=1536 I0720 21:30:04.953385 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؀ queue_depth=1537 I0720 21:30:04.953388 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؁ queue_depth=1538 I0720 21:30:04.953391 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؂ queue_depth=1539 I0720 21:30:04.953394 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؃ queue_depth=1540 I0720 21:30:04.953397 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؄ queue_depth=1541 I0720 21:30:04.953400 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؅ queue_depth=1542 I0720 21:30:04.953403 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؆ queue_depth=1543 I0720 21:30:04.953406 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؇ queue_depth=1544 I0720 21:30:04.953409 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؈ queue_depth=1545 I0720 21:30:04.953412 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؉ queue_depth=1546 I0720 21:30:04.953415 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؊ queue_depth=1547 I0720 21:30:04.953418 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؋ queue_depth=1548 I0720 21:30:04.953421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=، queue_depth=1549 I0720 21:30:04.953424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؍ queue_depth=1550 I0720 21:30:04.953427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؎ queue_depth=1551 I0720 21:30:04.953430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؏ queue_depth=1552 I0720 21:30:04.953433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؐ queue_depth=1553 I0720 21:30:04.953435 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؑ queue_depth=1554 I0720 21:30:04.953438 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؒ queue_depth=1555 I0720 21:30:04.953441 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؓ queue_depth=1556 I0720 21:30:04.953444 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؔ queue_depth=1557 I0720 21:30:04.953447 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؕ queue_depth=1558 I0720 21:30:04.953450 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؖ queue_depth=1559 I0720 21:30:04.953452 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؗ queue_depth=1560 I0720 21:30:04.953456 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؘ queue_depth=1561 I0720 21:30:04.953458 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؙ queue_depth=1562 I0720 21:30:04.953462 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؚ queue_depth=1563 I0720 21:30:04.953465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؛ queue_depth=1564 I0720 21:30:04.953468 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؜ queue_depth=1565 I0720 21:30:04.953471 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؝ queue_depth=1566 I0720 21:30:04.953474 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؞ queue_depth=1567 I0720 21:30:04.953477 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=؟ queue_depth=1568 I0720 21:30:04.953479 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؠ queue_depth=1569 I0720 21:30:04.953482 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ء queue_depth=1570 I0720 21:30:04.953485 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=آ queue_depth=1571 I0720 21:30:04.953488 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=أ queue_depth=1572 I0720 21:30:04.953491 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؤ queue_depth=1573 I0720 21:30:04.953494 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=إ queue_depth=1574 I0720 21:30:04.953497 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ئ queue_depth=1575 I0720 21:30:04.953500 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ا queue_depth=1576 I0720 21:30:04.953503 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ب queue_depth=1577 I0720 21:30:04.953506 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ة queue_depth=1578 I0720 21:30:04.953511 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ت queue_depth=1579 I0720 21:30:04.953515 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ث queue_depth=1580 I0720 21:30:04.953518 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ج queue_depth=1581 I0720 21:30:04.953527 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ح queue_depth=1582 I0720 21:30:04.953530 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=خ queue_depth=1583 I0720 21:30:04.953533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=د queue_depth=1584 I0720 21:30:04.953536 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ذ queue_depth=1585 I0720 21:30:04.953539 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ر queue_depth=1586 I0720 21:30:04.953542 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ز queue_depth=1587 I0720 21:30:04.953544 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=س queue_depth=1588 I0720 21:30:04.953547 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ش queue_depth=1589 I0720 21:30:04.953550 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ص queue_depth=1590 I0720 21:30:04.953553 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ض queue_depth=1591 I0720 21:30:04.953556 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ط queue_depth=1592 I0720 21:30:04.953559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ظ queue_depth=1593 I0720 21:30:04.953562 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ع queue_depth=1594 I0720 21:30:04.953567 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=غ queue_depth=1595 I0720 21:30:04.953570 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ػ queue_depth=1596 I0720 21:30:04.953573 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؼ queue_depth=1597 I0720 21:30:04.953577 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؽ queue_depth=1598 I0720 21:30:04.953581 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؾ queue_depth=1599 I0720 21:30:04.953584 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ؿ queue_depth=1600 I0720 21:30:04.953587 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ـ queue_depth=1601 I0720 21:30:04.953590 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ف queue_depth=1602 I0720 21:30:04.953593 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ق queue_depth=1603 I0720 21:30:04.953596 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ك queue_depth=1604 I0720 21:30:04.953599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ل queue_depth=1605 I0720 21:30:04.953601 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=م queue_depth=1606 I0720 21:30:04.953604 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ن queue_depth=1607 I0720 21:30:04.953607 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ه queue_depth=1608 I0720 21:30:04.953610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=و queue_depth=1609 I0720 21:30:04.953613 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ى queue_depth=1610 I0720 21:30:04.953616 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ي queue_depth=1611 I0720 21:30:04.953619 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ً queue_depth=1612 I0720 21:30:04.953622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٌ queue_depth=1613 I0720 21:30:04.953625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٍ queue_depth=1614 I0720 21:30:04.953628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=َ queue_depth=1615 I0720 21:30:04.953632 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ُ queue_depth=1616 I0720 21:30:04.953635 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ِ queue_depth=1617 I0720 21:30:04.953638 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ّ queue_depth=1618 I0720 21:30:04.953641 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ْ queue_depth=1619 I0720 21:30:04.953644 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٓ queue_depth=1620 I0720 21:30:04.953647 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٔ queue_depth=1621 I0720 21:30:04.953650 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٕ queue_depth=1622 I0720 21:30:04.953653 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٖ queue_depth=1623 I0720 21:30:04.953656 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٗ queue_depth=1624 I0720 21:30:04.953659 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٘ queue_depth=1625 I0720 21:30:04.953662 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٙ queue_depth=1626 I0720 21:30:04.953665 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٚ queue_depth=1627 I0720 21:30:04.953668 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٛ queue_depth=1628 I0720 21:30:04.953671 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٜ queue_depth=1629 I0720 21:30:04.953674 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٝ queue_depth=1630 I0720 21:30:04.953677 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٞ queue_depth=1631 I0720 21:30:04.953680 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٟ queue_depth=1632 I0720 21:30:04.953683 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٠ queue_depth=1633 I0720 21:30:04.953686 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=١ queue_depth=1634 I0720 21:30:04.953688 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٢ queue_depth=1635 I0720 21:30:04.953691 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٣ queue_depth=1636 I0720 21:30:04.953694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٤ queue_depth=1637 I0720 21:30:04.953697 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٥ queue_depth=1638 I0720 21:30:04.953700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٦ queue_depth=1639 I0720 21:30:04.953703 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٧ queue_depth=1640 I0720 21:30:04.953706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٨ queue_depth=1641 I0720 21:30:04.953709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٩ queue_depth=1642 I0720 21:30:04.953712 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٪ queue_depth=1643 I0720 21:30:04.953715 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٫ queue_depth=1644 I0720 21:30:04.953718 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٬ queue_depth=1645 I0720 21:30:04.953721 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=٭ queue_depth=1646 I0720 21:30:04.953724 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٮ queue_depth=1647 I0720 21:30:04.953727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٯ queue_depth=1648 I0720 21:30:04.953730 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٰ queue_depth=1649 I0720 21:30:04.953733 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٱ queue_depth=1650 I0720 21:30:04.953736 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٲ queue_depth=1651 I0720 21:30:04.953739 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٳ queue_depth=1652 I0720 21:30:04.953742 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٴ queue_depth=1653 I0720 21:30:04.953745 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٵ queue_depth=1654 I0720 21:30:04.953748 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٶ queue_depth=1655 I0720 21:30:04.953751 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٷ queue_depth=1656 I0720 21:30:04.953754 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٸ queue_depth=1657 I0720 21:30:04.953757 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٹ queue_depth=1658 I0720 21:30:04.953760 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٺ queue_depth=1659 I0720 21:30:04.953764 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٻ queue_depth=1660 I0720 21:30:04.953767 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ټ queue_depth=1661 I0720 21:30:04.953770 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٽ queue_depth=1662 I0720 21:30:04.953773 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=پ queue_depth=1663 I0720 21:30:04.953776 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ٿ queue_depth=1664 I0720 21:30:04.953782 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڀ queue_depth=1665 I0720 21:30:04.953785 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ځ queue_depth=1666 I0720 21:30:04.953787 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڂ queue_depth=1667 I0720 21:30:04.953790 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڃ queue_depth=1668 I0720 21:30:04.953793 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڄ queue_depth=1669 I0720 21:30:04.953796 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=څ queue_depth=1670 I0720 21:30:04.953799 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=چ queue_depth=1671 I0720 21:30:04.953802 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڇ queue_depth=1672 I0720 21:30:04.953805 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڈ queue_depth=1673 I0720 21:30:04.953808 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ډ queue_depth=1674 I0720 21:30:04.953811 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڊ queue_depth=1675 I0720 21:30:04.953814 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڋ queue_depth=1676 I0720 21:30:04.953817 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڌ queue_depth=1677 I0720 21:30:04.953820 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڍ queue_depth=1678 I0720 21:30:04.953823 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڎ queue_depth=1679 I0720 21:30:04.953826 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڏ queue_depth=1680 I0720 21:30:04.953828 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڐ queue_depth=1681 I0720 21:30:04.953832 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڑ queue_depth=1682 I0720 21:30:04.953835 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڒ queue_depth=1683 I0720 21:30:04.953838 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ړ queue_depth=1684 I0720 21:30:04.953841 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڔ queue_depth=1685 I0720 21:30:04.953845 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڕ queue_depth=1686 I0720 21:30:04.953848 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ږ queue_depth=1687 I0720 21:30:04.953851 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڗ queue_depth=1688 I0720 21:30:04.953854 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ژ queue_depth=1689 I0720 21:30:04.953857 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڙ queue_depth=1690 I0720 21:30:04.953860 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ښ queue_depth=1691 I0720 21:30:04.953862 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڛ queue_depth=1692 I0720 21:30:04.953866 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڜ queue_depth=1693 I0720 21:30:04.953869 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڝ queue_depth=1694 I0720 21:30:04.953871 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڞ queue_depth=1695 I0720 21:30:04.953874 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڟ queue_depth=1696 I0720 21:30:04.953878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڠ queue_depth=1697 I0720 21:30:04.953881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڡ queue_depth=1698 I0720 21:30:04.953884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڢ queue_depth=1699 I0720 21:30:04.953887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڣ queue_depth=1700 I0720 21:30:04.953890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڤ queue_depth=1701 I0720 21:30:04.953901 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڥ queue_depth=1702 I0720 21:30:04.953904 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڦ queue_depth=1703 I0720 21:30:04.953907 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڧ queue_depth=1704 I0720 21:30:04.953910 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڨ queue_depth=1705 I0720 21:30:04.953913 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ک queue_depth=1706 I0720 21:30:04.953916 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڪ queue_depth=1707 I0720 21:30:04.953918 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ګ queue_depth=1708 I0720 21:30:04.953921 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڬ queue_depth=1709 I0720 21:30:04.953924 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڭ queue_depth=1710 I0720 21:30:04.953927 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڮ queue_depth=1711 I0720 21:30:04.953930 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=گ queue_depth=1712 I0720 21:30:04.953933 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڰ queue_depth=1713 I0720 21:30:04.953936 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڱ queue_depth=1714 I0720 21:30:04.953939 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڲ queue_depth=1715 I0720 21:30:04.953942 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڳ queue_depth=1716 I0720 21:30:04.953945 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڴ queue_depth=1717 I0720 21:30:04.953948 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڵ queue_depth=1718 I0720 21:30:04.953951 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڶ queue_depth=1719 I0720 21:30:04.953954 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڷ queue_depth=1720 I0720 21:30:04.953957 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڸ queue_depth=1721 I0720 21:30:04.953960 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڹ queue_depth=1722 I0720 21:30:04.953963 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ں queue_depth=1723 I0720 21:30:04.953965 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڻ queue_depth=1724 I0720 21:30:04.953968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڼ queue_depth=1725 I0720 21:30:04.953972 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڽ queue_depth=1726 I0720 21:30:04.953975 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ھ queue_depth=1727 I0720 21:30:04.953978 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ڿ queue_depth=1728 I0720 21:30:04.953981 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۀ queue_depth=1729 I0720 21:30:04.953984 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ہ queue_depth=1730 I0720 21:30:04.953987 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۂ queue_depth=1731 I0720 21:30:04.953990 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۃ queue_depth=1732 I0720 21:30:04.953992 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۄ queue_depth=1733 I0720 21:30:04.953995 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۅ queue_depth=1734 I0720 21:30:04.953998 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۆ queue_depth=1735 I0720 21:30:04.954001 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۇ queue_depth=1736 I0720 21:30:04.954004 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۈ queue_depth=1737 I0720 21:30:04.954007 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۉ queue_depth=1738 I0720 21:30:04.954010 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۊ queue_depth=1739 I0720 21:30:04.954013 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۋ queue_depth=1740 I0720 21:30:04.954016 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ی queue_depth=1741 I0720 21:30:04.954019 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۍ queue_depth=1742 I0720 21:30:04.954022 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ێ queue_depth=1743 I0720 21:30:04.954025 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۏ queue_depth=1744 I0720 21:30:04.954028 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ې queue_depth=1745 I0720 21:30:04.954030 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۑ queue_depth=1746 I0720 21:30:04.954033 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ے queue_depth=1747 I0720 21:30:04.954036 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۓ queue_depth=1748 I0720 21:30:04.954039 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۔ queue_depth=1749 I0720 21:30:04.954042 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ە queue_depth=1750 I0720 21:30:04.954045 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۖ queue_depth=1751 I0720 21:30:04.954048 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۗ queue_depth=1752 I0720 21:30:04.954051 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۘ queue_depth=1753 I0720 21:30:04.954054 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۙ queue_depth=1754 I0720 21:30:04.954057 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۚ queue_depth=1755 I0720 21:30:04.954060 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۛ queue_depth=1756 I0720 21:30:04.954063 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۜ queue_depth=1757 I0720 21:30:04.954065 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۝ queue_depth=1758 I0720 21:30:04.954069 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۞ queue_depth=1759 I0720 21:30:04.954072 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۟ queue_depth=1760 I0720 21:30:04.954075 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۠ queue_depth=1761 I0720 21:30:04.954078 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۡ queue_depth=1762 I0720 21:30:04.954081 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۢ queue_depth=1763 I0720 21:30:04.954084 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۣ queue_depth=1764 I0720 21:30:04.954087 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۤ queue_depth=1765 I0720 21:30:04.954090 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۥ queue_depth=1766 I0720 21:30:04.954093 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۦ queue_depth=1767 I0720 21:30:04.954095 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۧ queue_depth=1768 I0720 21:30:04.954098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۨ queue_depth=1769 I0720 21:30:04.954101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۩ queue_depth=1770 I0720 21:30:04.954104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۪ queue_depth=1771 I0720 21:30:04.954107 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۫ queue_depth=1772 I0720 21:30:04.954110 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۬ queue_depth=1773 I0720 21:30:04.954113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۭ queue_depth=1774 I0720 21:30:04.954116 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۮ queue_depth=1775 I0720 21:30:04.954119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۯ queue_depth=1776 I0720 21:30:04.954121 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۰ queue_depth=1777 I0720 21:30:04.954124 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۱ queue_depth=1778 I0720 21:30:04.954128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۲ queue_depth=1779 I0720 21:30:04.954131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۳ queue_depth=1780 I0720 21:30:04.954134 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۴ queue_depth=1781 I0720 21:30:04.954139 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۵ queue_depth=1782 I0720 21:30:04.954142 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۶ queue_depth=1783 I0720 21:30:04.954145 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۷ queue_depth=1784 I0720 21:30:04.954148 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۸ queue_depth=1785 I0720 21:30:04.954151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۹ queue_depth=1786 I0720 21:30:04.954154 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۺ queue_depth=1787 I0720 21:30:04.954157 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۻ queue_depth=1788 I0720 21:30:04.954160 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۼ queue_depth=1789 I0720 21:30:04.954209 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۽ queue_depth=1790 I0720 21:30:04.954214 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=۾ queue_depth=1791 I0720 21:30:04.954218 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ۿ queue_depth=1792 I0720 21:30:04.954221 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܀ queue_depth=1793 I0720 21:30:04.954224 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܁ queue_depth=1794 I0720 21:30:04.954227 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܂ queue_depth=1795 I0720 21:30:04.954258 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܃ queue_depth=1796 I0720 21:30:04.954261 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܄ queue_depth=1797 I0720 21:30:04.954264 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܅ queue_depth=1798 I0720 21:30:04.954267 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܆ queue_depth=1799 I0720 21:30:04.954270 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܇ queue_depth=1800 I0720 21:30:04.954272 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܈ queue_depth=1801 I0720 21:30:04.954275 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܉ queue_depth=1802 I0720 21:30:04.954278 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܊ queue_depth=1803 I0720 21:30:04.954281 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܋ queue_depth=1804 I0720 21:30:04.954285 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܌ queue_depth=1805 I0720 21:30:04.954287 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܍ queue_depth=1806 I0720 21:30:04.954290 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܎ queue_depth=1807 I0720 21:30:04.954293 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=܏ queue_depth=1808 I0720 21:30:04.954299 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܐ queue_depth=1809 I0720 21:30:04.954302 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܑ queue_depth=1810 I0720 21:30:04.954305 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܒ queue_depth=1811 I0720 21:30:04.954307 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܓ queue_depth=1812 I0720 21:30:04.954310 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܔ queue_depth=1813 I0720 21:30:04.954313 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܕ queue_depth=1814 I0720 21:30:04.954316 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܖ queue_depth=1815 I0720 21:30:04.954319 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܗ queue_depth=1816 I0720 21:30:04.954322 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܘ queue_depth=1817 I0720 21:30:04.954325 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܙ queue_depth=1818 I0720 21:30:04.954328 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܚ queue_depth=1819 I0720 21:30:04.954331 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܛ queue_depth=1820 I0720 21:30:04.954334 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܜ queue_depth=1821 I0720 21:30:04.954337 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܝ queue_depth=1822 I0720 21:30:04.954340 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܞ queue_depth=1823 I0720 21:30:04.954345 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܟ queue_depth=1824 I0720 21:30:04.954355 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܠ queue_depth=1825 I0720 21:30:04.954360 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܡ queue_depth=1826 I0720 21:30:04.954363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܢ queue_depth=1827 I0720 21:30:04.954366 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܣ queue_depth=1828 I0720 21:30:04.954369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܤ queue_depth=1829 I0720 21:30:04.954372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܥ queue_depth=1830 I0720 21:30:04.954374 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܦ queue_depth=1831 I0720 21:30:04.954377 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܧ queue_depth=1832 I0720 21:30:04.954380 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܨ queue_depth=1833 I0720 21:30:04.954383 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܩ queue_depth=1834 I0720 21:30:04.954386 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܪ queue_depth=1835 I0720 21:30:04.954389 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܫ queue_depth=1836 I0720 21:30:04.954391 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܬ queue_depth=1837 I0720 21:30:04.954394 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܭ queue_depth=1838 I0720 21:30:04.954397 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܮ queue_depth=1839 I0720 21:30:04.954400 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܯ queue_depth=1840 I0720 21:30:04.954403 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܰ queue_depth=1841 I0720 21:30:04.954406 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܱ queue_depth=1842 I0720 21:30:04.954409 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܲ queue_depth=1843 I0720 21:30:04.954412 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܳ queue_depth=1844 I0720 21:30:04.954414 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܴ queue_depth=1845 I0720 21:30:04.954417 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܵ queue_depth=1846 I0720 21:30:04.954420 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܶ queue_depth=1847 I0720 21:30:04.954422 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܷ queue_depth=1848 I0720 21:30:04.954425 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܸ queue_depth=1849 I0720 21:30:04.954428 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܹ queue_depth=1850 I0720 21:30:04.954431 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܺ queue_depth=1851 I0720 21:30:04.954433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܻ queue_depth=1852 I0720 21:30:04.954438 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܼ queue_depth=1853 I0720 21:30:04.954441 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܽ queue_depth=1854 I0720 21:30:04.954444 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܾ queue_depth=1855 I0720 21:30:04.954447 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ܿ queue_depth=1856 I0720 21:30:04.954450 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݀ queue_depth=1857 I0720 21:30:04.954453 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݁ queue_depth=1858 I0720 21:30:04.954456 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݂ queue_depth=1859 I0720 21:30:04.954459 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݃ queue_depth=1860 I0720 21:30:04.954462 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݄ queue_depth=1861 I0720 21:30:04.954465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݅ queue_depth=1862 I0720 21:30:04.954469 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݆ queue_depth=1863 I0720 21:30:04.954472 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݇ queue_depth=1864 I0720 21:30:04.954475 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݈ queue_depth=1865 I0720 21:30:04.954478 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݉ queue_depth=1866 I0720 21:30:04.954481 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݊ queue_depth=1867 I0720 21:30:04.954483 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݋ queue_depth=1868 I0720 21:30:04.954486 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=݌ queue_depth=1869 I0720 21:30:04.954489 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݍ queue_depth=1870 I0720 21:30:04.954492 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݎ queue_depth=1871 I0720 21:30:04.954495 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݏ queue_depth=1872 I0720 21:30:04.954498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݐ queue_depth=1873 I0720 21:30:04.954501 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݑ queue_depth=1874 I0720 21:30:04.954504 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݒ queue_depth=1875 I0720 21:30:04.954506 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݓ queue_depth=1876 I0720 21:30:04.954509 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݔ queue_depth=1877 I0720 21:30:04.954512 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݕ queue_depth=1878 I0720 21:30:04.954515 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݖ queue_depth=1879 I0720 21:30:04.954518 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݗ queue_depth=1880 I0720 21:30:04.954521 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݘ queue_depth=1881 I0720 21:30:04.954524 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݙ queue_depth=1882 I0720 21:30:04.954526 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݚ queue_depth=1883 I0720 21:30:04.954529 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݛ queue_depth=1884 I0720 21:30:04.954532 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݜ queue_depth=1885 I0720 21:30:04.954535 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݝ queue_depth=1886 I0720 21:30:04.954538 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݞ queue_depth=1887 I0720 21:30:04.954541 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݟ queue_depth=1888 I0720 21:30:04.954544 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݠ queue_depth=1889 I0720 21:30:04.954546 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݡ queue_depth=1890 I0720 21:30:04.954549 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݢ queue_depth=1891 I0720 21:30:04.954553 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݣ queue_depth=1892 I0720 21:30:04.954556 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݤ queue_depth=1893 I0720 21:30:04.954559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݥ queue_depth=1894 I0720 21:30:04.954562 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݦ queue_depth=1895 I0720 21:30:04.954565 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݧ queue_depth=1896 I0720 21:30:04.954568 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݨ queue_depth=1897 I0720 21:30:04.954571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݩ queue_depth=1898 I0720 21:30:04.954574 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݪ queue_depth=1899 I0720 21:30:04.954577 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݫ queue_depth=1900 I0720 21:30:04.954580 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݬ queue_depth=1901 I0720 21:30:04.954583 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݭ queue_depth=1902 I0720 21:30:04.954586 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݮ queue_depth=1903 I0720 21:30:04.954588 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݯ queue_depth=1904 I0720 21:30:04.954591 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݰ queue_depth=1905 I0720 21:30:04.954594 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݱ queue_depth=1906 I0720 21:30:04.954597 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݲ queue_depth=1907 I0720 21:30:04.954600 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݳ queue_depth=1908 I0720 21:30:04.954602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݴ queue_depth=1909 I0720 21:30:04.954605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݵ queue_depth=1910 I0720 21:30:04.954608 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݶ queue_depth=1911 I0720 21:30:04.954611 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݷ queue_depth=1912 I0720 21:30:04.954613 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݸ queue_depth=1913 I0720 21:30:04.954616 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݹ queue_depth=1914 I0720 21:30:04.954619 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݺ queue_depth=1915 I0720 21:30:04.954622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݻ queue_depth=1916 I0720 21:30:04.954625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݼ queue_depth=1917 I0720 21:30:04.954628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݽ queue_depth=1918 I0720 21:30:04.954631 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݾ queue_depth=1919 I0720 21:30:04.954634 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ݿ queue_depth=1920 I0720 21:30:04.954637 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ހ queue_depth=1921 I0720 21:30:04.954640 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ށ queue_depth=1922 I0720 21:30:04.954642 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ނ queue_depth=1923 I0720 21:30:04.954646 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ރ queue_depth=1924 I0720 21:30:04.954649 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ބ queue_depth=1925 I0720 21:30:04.954652 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޅ queue_depth=1926 I0720 21:30:04.954655 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ކ queue_depth=1927 I0720 21:30:04.954658 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=އ queue_depth=1928 I0720 21:30:04.954661 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ވ queue_depth=1929 I0720 21:30:04.954664 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=މ queue_depth=1930 I0720 21:30:04.954667 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ފ queue_depth=1931 I0720 21:30:04.954670 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ދ queue_depth=1932 I0720 21:30:04.954673 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ތ queue_depth=1933 I0720 21:30:04.954678 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ލ queue_depth=1934 I0720 21:30:04.954681 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ގ queue_depth=1935 I0720 21:30:04.954684 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޏ queue_depth=1936 I0720 21:30:04.954687 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ސ queue_depth=1937 I0720 21:30:04.954691 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޑ queue_depth=1938 I0720 21:30:04.954694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޒ queue_depth=1939 I0720 21:30:04.954697 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޓ queue_depth=1940 I0720 21:30:04.954700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޔ queue_depth=1941 I0720 21:30:04.954703 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޕ queue_depth=1942 I0720 21:30:04.954706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޖ queue_depth=1943 I0720 21:30:04.954709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޗ queue_depth=1944 I0720 21:30:04.954713 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޘ queue_depth=1945 I0720 21:30:04.954716 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޙ queue_depth=1946 I0720 21:30:04.954718 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޚ queue_depth=1947 I0720 21:30:04.954721 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޛ queue_depth=1948 I0720 21:30:04.954724 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޜ queue_depth=1949 I0720 21:30:04.954727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޝ queue_depth=1950 I0720 21:30:04.954730 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޞ queue_depth=1951 I0720 21:30:04.954734 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޟ queue_depth=1952 I0720 21:30:04.954737 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޠ queue_depth=1953 I0720 21:30:04.954740 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޡ queue_depth=1954 I0720 21:30:04.954743 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޢ queue_depth=1955 I0720 21:30:04.954746 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޣ queue_depth=1956 I0720 21:30:04.954750 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޤ queue_depth=1957 I0720 21:30:04.954753 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޥ queue_depth=1958 I0720 21:30:04.954756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ަ queue_depth=1959 I0720 21:30:04.954759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ާ queue_depth=1960 I0720 21:30:04.954762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ި queue_depth=1961 I0720 21:30:04.954765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ީ queue_depth=1962 I0720 21:30:04.954768 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ު queue_depth=1963 I0720 21:30:04.954770 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޫ queue_depth=1964 I0720 21:30:04.954773 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ެ queue_depth=1965 I0720 21:30:04.954777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޭ queue_depth=1966 I0720 21:30:04.954780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޮ queue_depth=1967 I0720 21:30:04.954783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޯ queue_depth=1968 I0720 21:30:04.954786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ް queue_depth=1969 I0720 21:30:04.954789 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ޱ queue_depth=1970 I0720 21:30:04.954792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޲ queue_depth=1971 I0720 21:30:04.954795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޳ queue_depth=1972 I0720 21:30:04.954798 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޴ queue_depth=1973 I0720 21:30:04.954801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޵ queue_depth=1974 I0720 21:30:04.954803 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޶ queue_depth=1975 I0720 21:30:04.954806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޷ queue_depth=1976 I0720 21:30:04.954809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޸ queue_depth=1977 I0720 21:30:04.954812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޹ queue_depth=1978 I0720 21:30:04.954815 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޺ queue_depth=1979 I0720 21:30:04.954818 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޻ queue_depth=1980 I0720 21:30:04.954821 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޼ queue_depth=1981 I0720 21:30:04.954824 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޽ queue_depth=1982 I0720 21:30:04.954827 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޾ queue_depth=1983 I0720 21:30:04.954830 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=޿ queue_depth=1984 I0720 21:30:04.954832 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߀ queue_depth=1985 I0720 21:30:04.954835 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߁ queue_depth=1986 I0720 21:30:04.954838 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߂ queue_depth=1987 I0720 21:30:04.954841 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߃ queue_depth=1988 I0720 21:30:04.954845 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߄ queue_depth=1989 I0720 21:30:04.954848 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߅ queue_depth=1990 I0720 21:30:04.954851 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߆ queue_depth=1991 I0720 21:30:04.954854 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߇ queue_depth=1992 I0720 21:30:04.954857 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߈ queue_depth=1993 I0720 21:30:04.954859 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߉ queue_depth=1994 I0720 21:30:04.954862 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߊ queue_depth=1995 I0720 21:30:04.954865 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߋ queue_depth=1996 I0720 21:30:04.954868 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߌ queue_depth=1997 I0720 21:30:04.954871 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߍ queue_depth=1998 I0720 21:30:04.954873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߎ queue_depth=1999 I0720 21:30:04.954876 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߏ queue_depth=2000 I0720 21:30:04.954879 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߐ queue_depth=2001 I0720 21:30:04.954882 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߑ queue_depth=2002 I0720 21:30:04.954885 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߒ queue_depth=2003 I0720 21:30:04.954888 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߓ queue_depth=2004 I0720 21:30:04.954891 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߔ queue_depth=2005 I0720 21:30:04.954893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߕ queue_depth=2006 I0720 21:30:04.954897 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߖ queue_depth=2007 I0720 21:30:04.954900 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߗ queue_depth=2008 I0720 21:30:04.955009 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߘ queue_depth=2009 I0720 21:30:04.955015 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߙ queue_depth=2010 I0720 21:30:04.955019 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߚ queue_depth=2011 I0720 21:30:04.955022 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߛ queue_depth=2012 I0720 21:30:04.955025 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߜ queue_depth=2013 I0720 21:30:04.955028 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߝ queue_depth=2014 I0720 21:30:04.955031 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߞ queue_depth=2015 I0720 21:30:04.955034 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߟ queue_depth=2016 I0720 21:30:04.955037 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߠ queue_depth=2017 I0720 21:30:04.955040 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߡ queue_depth=2018 I0720 21:30:04.955043 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߢ queue_depth=2019 I0720 21:30:04.955046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߣ queue_depth=2020 I0720 21:30:04.955049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߤ queue_depth=2021 I0720 21:30:04.955052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߥ queue_depth=2022 I0720 21:30:04.955056 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߦ queue_depth=2023 I0720 21:30:04.955059 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߧ queue_depth=2024 I0720 21:30:04.955061 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߨ queue_depth=2025 I0720 21:30:04.955064 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߩ queue_depth=2026 I0720 21:30:04.955068 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߪ queue_depth=2027 I0720 21:30:04.955071 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߫ queue_depth=2028 I0720 21:30:04.955074 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߬ queue_depth=2029 I0720 21:30:04.955083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߭ queue_depth=2030 I0720 21:30:04.955086 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߮ queue_depth=2031 I0720 21:30:04.955089 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߯ queue_depth=2032 I0720 21:30:04.955092 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߰ queue_depth=2033 I0720 21:30:04.955095 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߱ queue_depth=2034 I0720 21:30:04.955098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߲ queue_depth=2035 I0720 21:30:04.955101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߳ queue_depth=2036 I0720 21:30:04.955104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߴ queue_depth=2037 I0720 21:30:04.955107 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߵ queue_depth=2038 I0720 21:30:04.955110 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߶ queue_depth=2039 I0720 21:30:04.955113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߷ queue_depth=2040 I0720 21:30:04.955116 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߸ queue_depth=2041 I0720 21:30:04.955119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߹ queue_depth=2042 I0720 21:30:04.955122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ߺ queue_depth=2043 I0720 21:30:04.955125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߻ queue_depth=2044 I0720 21:30:04.955128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߼ queue_depth=2045 I0720 21:30:04.955131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߽ queue_depth=2046 I0720 21:30:04.955134 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߾ queue_depth=2047 I0720 21:30:04.955137 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=߿ queue_depth=2048 I0720 21:30:04.955140 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠀ queue_depth=2049 I0720 21:30:04.955143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠁ queue_depth=2050 I0720 21:30:04.955148 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠂ queue_depth=2051 I0720 21:30:04.955151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠃ queue_depth=2052 I0720 21:30:04.955154 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠄ queue_depth=2053 I0720 21:30:04.955157 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠅ queue_depth=2054 I0720 21:30:04.955160 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠆ queue_depth=2055 I0720 21:30:04.955165 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠇ queue_depth=2056 I0720 21:30:04.955168 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠈ queue_depth=2057 I0720 21:30:04.955171 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠉ queue_depth=2058 I0720 21:30:04.955174 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠊ queue_depth=2059 I0720 21:30:04.955176 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠋ queue_depth=2060 I0720 21:30:04.955179 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠌ queue_depth=2061 I0720 21:30:04.955182 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠍ queue_depth=2062 I0720 21:30:04.955185 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠎ queue_depth=2063 I0720 21:30:04.955188 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠏ queue_depth=2064 I0720 21:30:04.955191 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠐ queue_depth=2065 I0720 21:30:04.955194 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠑ queue_depth=2066 I0720 21:30:04.955197 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠒ queue_depth=2067 I0720 21:30:04.955200 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠓ queue_depth=2068 I0720 21:30:04.955203 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠔ queue_depth=2069 I0720 21:30:04.955206 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠕ queue_depth=2070 I0720 21:30:04.955209 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠖ queue_depth=2071 I0720 21:30:04.955212 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠗ queue_depth=2072 I0720 21:30:04.955215 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠘ queue_depth=2073 I0720 21:30:04.955218 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠙ queue_depth=2074 I0720 21:30:04.955221 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠚ queue_depth=2075 I0720 21:30:04.955223 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠛ queue_depth=2076 I0720 21:30:04.955226 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠜ queue_depth=2077 I0720 21:30:04.955229 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠝ queue_depth=2078 I0720 21:30:04.955232 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠞ queue_depth=2079 I0720 21:30:04.955235 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠟ queue_depth=2080 I0720 21:30:04.955238 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠠ queue_depth=2081 I0720 21:30:04.955241 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠡ queue_depth=2082 I0720 21:30:04.955244 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠢ queue_depth=2083 I0720 21:30:04.955247 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠣ queue_depth=2084 I0720 21:30:04.955249 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠤ queue_depth=2085 I0720 21:30:04.955253 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠥ queue_depth=2086 I0720 21:30:04.955255 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠦ queue_depth=2087 I0720 21:30:04.955258 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠧ queue_depth=2088 I0720 21:30:04.955262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠨ queue_depth=2089 I0720 21:30:04.955267 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠩ queue_depth=2090 I0720 21:30:04.955270 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠪ queue_depth=2091 I0720 21:30:04.955273 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠫ queue_depth=2092 I0720 21:30:04.955275 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࠬ queue_depth=2093 I0720 21:30:04.955278 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠭ queue_depth=2094 I0720 21:30:04.955281 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠮ queue_depth=2095 I0720 21:30:04.955284 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠯ queue_depth=2096 I0720 21:30:04.955287 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠰ queue_depth=2097 I0720 21:30:04.955290 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠱ queue_depth=2098 I0720 21:30:04.955293 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠲ queue_depth=2099 I0720 21:30:04.955296 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠳ queue_depth=2100 I0720 21:30:04.955299 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠴ queue_depth=2101 I0720 21:30:04.955302 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠵ queue_depth=2102 I0720 21:30:04.955305 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠶ queue_depth=2103 I0720 21:30:04.955308 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠷ queue_depth=2104 I0720 21:30:04.955311 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠸ queue_depth=2105 I0720 21:30:04.955314 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠹ queue_depth=2106 I0720 21:30:04.955317 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠺ queue_depth=2107 I0720 21:30:04.955320 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠻ queue_depth=2108 I0720 21:30:04.955324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠼ queue_depth=2109 I0720 21:30:04.955327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠽ queue_depth=2110 I0720 21:30:04.955330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠾ queue_depth=2111 I0720 21:30:04.955333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࠿ queue_depth=2112 I0720 21:30:04.955336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡀ queue_depth=2113 I0720 21:30:04.955339 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡁ queue_depth=2114 I0720 21:30:04.955342 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡂ queue_depth=2115 I0720 21:30:04.955345 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡃ queue_depth=2116 I0720 21:30:04.955348 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡄ queue_depth=2117 I0720 21:30:04.955351 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡅ queue_depth=2118 I0720 21:30:04.955354 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡆ queue_depth=2119 I0720 21:30:04.955357 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡇ queue_depth=2120 I0720 21:30:04.955363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡈ queue_depth=2121 I0720 21:30:04.955367 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡉ queue_depth=2122 I0720 21:30:04.955370 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡊ queue_depth=2123 I0720 21:30:04.955373 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡋ queue_depth=2124 I0720 21:30:04.955376 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡌ queue_depth=2125 I0720 21:30:04.955379 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡍ queue_depth=2126 I0720 21:30:04.955381 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡎ queue_depth=2127 I0720 21:30:04.955384 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡏ queue_depth=2128 I0720 21:30:04.955387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡐ queue_depth=2129 I0720 21:30:04.955390 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡑ queue_depth=2130 I0720 21:30:04.955393 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡒ queue_depth=2131 I0720 21:30:04.955396 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡓ queue_depth=2132 I0720 21:30:04.955399 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡔ queue_depth=2133 I0720 21:30:04.955402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡕ queue_depth=2134 I0720 21:30:04.955405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡖ queue_depth=2135 I0720 21:30:04.955409 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡗ queue_depth=2136 I0720 21:30:04.955412 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡘ queue_depth=2137 I0720 21:30:04.955415 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡙ queue_depth=2138 I0720 21:30:04.955418 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡚ queue_depth=2139 I0720 21:30:04.955421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡛ queue_depth=2140 I0720 21:30:04.955424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡜ queue_depth=2141 I0720 21:30:04.955427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡝ queue_depth=2142 I0720 21:30:04.955430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡞ queue_depth=2143 I0720 21:30:04.955433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡟ queue_depth=2144 I0720 21:30:04.955436 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡠ queue_depth=2145 I0720 21:30:04.955439 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡡ queue_depth=2146 I0720 21:30:04.955442 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡢ queue_depth=2147 I0720 21:30:04.955445 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡣ queue_depth=2148 I0720 21:30:04.955447 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡤ queue_depth=2149 I0720 21:30:04.955450 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡥ queue_depth=2150 I0720 21:30:04.955453 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡦ queue_depth=2151 I0720 21:30:04.955456 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡧ queue_depth=2152 I0720 21:30:04.955459 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡨ queue_depth=2153 I0720 21:30:04.955463 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡩ queue_depth=2154 I0720 21:30:04.955466 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡪ queue_depth=2155 I0720 21:30:04.955469 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡫ queue_depth=2156 I0720 21:30:04.955472 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡬ queue_depth=2157 I0720 21:30:04.955475 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡭ queue_depth=2158 I0720 21:30:04.955478 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡮ queue_depth=2159 I0720 21:30:04.955481 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࡯ queue_depth=2160 I0720 21:30:04.955484 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡰ queue_depth=2161 I0720 21:30:04.955487 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡱ queue_depth=2162 I0720 21:30:04.955490 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡲ queue_depth=2163 I0720 21:30:04.955493 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡳ queue_depth=2164 I0720 21:30:04.955496 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡴ queue_depth=2165 I0720 21:30:04.955499 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡵ queue_depth=2166 I0720 21:30:04.955502 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡶ queue_depth=2167 I0720 21:30:04.955505 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡷ queue_depth=2168 I0720 21:30:04.955508 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡸ queue_depth=2169 I0720 21:30:04.955511 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡹ queue_depth=2170 I0720 21:30:04.955514 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡺ queue_depth=2171 I0720 21:30:04.955517 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡻ queue_depth=2172 I0720 21:30:04.955520 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡼ queue_depth=2173 I0720 21:30:04.955523 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡽ queue_depth=2174 I0720 21:30:04.955525 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡾ queue_depth=2175 I0720 21:30:04.955528 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࡿ queue_depth=2176 I0720 21:30:04.955531 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢀ queue_depth=2177 I0720 21:30:04.955534 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢁ queue_depth=2178 I0720 21:30:04.955537 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢂ queue_depth=2179 I0720 21:30:04.955540 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢃ queue_depth=2180 I0720 21:30:04.955543 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢄ queue_depth=2181 I0720 21:30:04.955546 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢅ queue_depth=2182 I0720 21:30:04.955549 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢆ queue_depth=2183 I0720 21:30:04.955552 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢇ queue_depth=2184 I0720 21:30:04.955555 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢈ queue_depth=2185 I0720 21:30:04.955558 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢉ queue_depth=2186 I0720 21:30:04.955561 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢊ queue_depth=2187 I0720 21:30:04.955564 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢋ queue_depth=2188 I0720 21:30:04.955567 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢌ queue_depth=2189 I0720 21:30:04.955571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢍ queue_depth=2190 I0720 21:30:04.955574 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢎ queue_depth=2191 I0720 21:30:04.955577 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢏ queue_depth=2192 I0720 21:30:04.955580 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢐ queue_depth=2193 I0720 21:30:04.955583 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢑ queue_depth=2194 I0720 21:30:04.955586 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢒ queue_depth=2195 I0720 21:30:04.955589 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢓ queue_depth=2196 I0720 21:30:04.955592 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢔ queue_depth=2197 I0720 21:30:04.955595 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢕ queue_depth=2198 I0720 21:30:04.955598 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢖ queue_depth=2199 I0720 21:30:04.955601 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢗ queue_depth=2200 I0720 21:30:04.955604 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢘ queue_depth=2201 I0720 21:30:04.955607 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢙ queue_depth=2202 I0720 21:30:04.955610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢚ queue_depth=2203 I0720 21:30:04.955613 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢛ queue_depth=2204 I0720 21:30:04.955615 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢜ queue_depth=2205 I0720 21:30:04.955619 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢝ queue_depth=2206 I0720 21:30:04.955622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢞ queue_depth=2207 I0720 21:30:04.955625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࢟ queue_depth=2208 I0720 21:30:04.955627 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢠ queue_depth=2209 I0720 21:30:04.955630 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢡ queue_depth=2210 I0720 21:30:04.955633 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢢ queue_depth=2211 I0720 21:30:04.955636 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢣ queue_depth=2212 I0720 21:30:04.955639 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢤ queue_depth=2213 I0720 21:30:04.955642 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢥ queue_depth=2214 I0720 21:30:04.955645 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢦ queue_depth=2215 I0720 21:30:04.955648 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢧ queue_depth=2216 I0720 21:30:04.955651 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢨ queue_depth=2217 I0720 21:30:04.955654 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢩ queue_depth=2218 I0720 21:30:04.955657 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢪ queue_depth=2219 I0720 21:30:04.955660 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢫ queue_depth=2220 I0720 21:30:04.955664 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢬ queue_depth=2221 I0720 21:30:04.955667 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢭ queue_depth=2222 I0720 21:30:04.955670 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢮ queue_depth=2223 I0720 21:30:04.955673 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢯ queue_depth=2224 I0720 21:30:04.955676 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢰ queue_depth=2225 I0720 21:30:04.955679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢱ queue_depth=2226 I0720 21:30:04.955682 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢲ queue_depth=2227 I0720 21:30:04.955685 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢳ queue_depth=2228 I0720 21:30:04.955688 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢴ queue_depth=2229 I0720 21:30:04.955691 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢵ queue_depth=2230 I0720 21:30:04.955694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢶ queue_depth=2231 I0720 21:30:04.955697 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢷ queue_depth=2232 I0720 21:30:04.955700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢸ queue_depth=2233 I0720 21:30:04.955702 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢹ queue_depth=2234 I0720 21:30:04.955706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢺ queue_depth=2235 I0720 21:30:04.955709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢻ queue_depth=2236 I0720 21:30:04.955712 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢼ queue_depth=2237 I0720 21:30:04.955715 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢽ queue_depth=2238 I0720 21:30:04.955718 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢾ queue_depth=2239 I0720 21:30:04.955720 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࢿ queue_depth=2240 I0720 21:30:04.955723 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣀ queue_depth=2241 I0720 21:30:04.955726 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣁ queue_depth=2242 I0720 21:30:04.955729 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣂ queue_depth=2243 I0720 21:30:04.955732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣃ queue_depth=2244 I0720 21:30:04.955735 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣄ queue_depth=2245 I0720 21:30:04.955738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣅ queue_depth=2246 I0720 21:30:04.955741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣆ queue_depth=2247 I0720 21:30:04.955744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣇ queue_depth=2248 I0720 21:30:04.955748 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣈ queue_depth=2249 I0720 21:30:04.955751 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣉ queue_depth=2250 I0720 21:30:04.955754 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣊ queue_depth=2251 I0720 21:30:04.955757 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣋ queue_depth=2252 I0720 21:30:04.955759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣌ queue_depth=2253 I0720 21:30:04.955763 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣍ queue_depth=2254 I0720 21:30:04.955766 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣎ queue_depth=2255 I0720 21:30:04.955769 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣏ queue_depth=2256 I0720 21:30:04.955772 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣐ queue_depth=2257 I0720 21:30:04.955774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣑ queue_depth=2258 I0720 21:30:04.955777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣒ queue_depth=2259 I0720 21:30:04.955780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣓ queue_depth=2260 I0720 21:30:04.955783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣔ queue_depth=2261 I0720 21:30:04.955786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣕ queue_depth=2262 I0720 21:30:04.955789 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣖ queue_depth=2263 I0720 21:30:04.955792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣗ queue_depth=2264 I0720 21:30:04.955795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣘ queue_depth=2265 I0720 21:30:04.955798 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣙ queue_depth=2266 I0720 21:30:04.955801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣚ queue_depth=2267 I0720 21:30:04.955804 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣛ queue_depth=2268 I0720 21:30:04.955806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣜ queue_depth=2269 I0720 21:30:04.955809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣝ queue_depth=2270 I0720 21:30:04.955812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣞ queue_depth=2271 I0720 21:30:04.955815 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣟ queue_depth=2272 I0720 21:30:04.955819 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣠ queue_depth=2273 I0720 21:30:04.955822 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣡ queue_depth=2274 I0720 21:30:04.955825 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣢ queue_depth=2275 I0720 21:30:04.955828 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣣ queue_depth=2276 I0720 21:30:04.955831 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣤ queue_depth=2277 I0720 21:30:04.955834 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣥ queue_depth=2278 I0720 21:30:04.955837 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣦ queue_depth=2279 I0720 21:30:04.955840 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣧ queue_depth=2280 I0720 21:30:04.955843 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣨ queue_depth=2281 I0720 21:30:04.955846 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣩ queue_depth=2282 I0720 21:30:04.955849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣪ queue_depth=2283 I0720 21:30:04.955852 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣫ queue_depth=2284 I0720 21:30:04.955855 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣬ queue_depth=2285 I0720 21:30:04.955858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣭ queue_depth=2286 I0720 21:30:04.955861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣮ queue_depth=2287 I0720 21:30:04.955867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࣯ queue_depth=2288 I0720 21:30:04.955870 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣰ queue_depth=2289 I0720 21:30:04.955873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣱ queue_depth=2290 I0720 21:30:04.955876 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣲ queue_depth=2291 I0720 21:30:04.955879 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣳ queue_depth=2292 I0720 21:30:04.955882 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣴ queue_depth=2293 I0720 21:30:04.955885 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣵ queue_depth=2294 I0720 21:30:04.955888 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣶ queue_depth=2295 I0720 21:30:04.955891 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣷ queue_depth=2296 I0720 21:30:04.955894 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣸ queue_depth=2297 I0720 21:30:04.955897 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣹ queue_depth=2298 I0720 21:30:04.955900 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣺ queue_depth=2299 I0720 21:30:04.955903 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣻ queue_depth=2300 I0720 21:30:04.955906 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣼ queue_depth=2301 I0720 21:30:04.955909 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣽ queue_depth=2302 I0720 21:30:04.955911 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣾ queue_depth=2303 I0720 21:30:04.955914 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ࣿ queue_depth=2304 I0720 21:30:04.955917 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऀ queue_depth=2305 I0720 21:30:04.955920 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ँ queue_depth=2306 I0720 21:30:04.955923 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ं queue_depth=2307 I0720 21:30:04.955926 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ः queue_depth=2308 I0720 21:30:04.955929 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऄ queue_depth=2309 I0720 21:30:04.955932 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=अ queue_depth=2310 I0720 21:30:04.955935 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=आ queue_depth=2311 I0720 21:30:04.955937 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=इ queue_depth=2312 I0720 21:30:04.955940 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ई queue_depth=2313 I0720 21:30:04.955943 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=उ queue_depth=2314 I0720 21:30:04.955946 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऊ queue_depth=2315 I0720 21:30:04.955949 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऋ queue_depth=2316 I0720 21:30:04.955952 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऌ queue_depth=2317 I0720 21:30:04.955955 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऍ queue_depth=2318 I0720 21:30:04.955958 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऎ queue_depth=2319 I0720 21:30:04.955962 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ए queue_depth=2320 I0720 21:30:04.955965 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऐ queue_depth=2321 I0720 21:30:04.955968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऑ queue_depth=2322 I0720 21:30:04.955970 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऒ queue_depth=2323 I0720 21:30:04.955973 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ओ queue_depth=2324 I0720 21:30:04.955976 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=औ queue_depth=2325 I0720 21:30:04.955980 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=क queue_depth=2326 I0720 21:30:04.955982 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ख queue_depth=2327 I0720 21:30:04.955985 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ग queue_depth=2328 I0720 21:30:04.955988 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=घ queue_depth=2329 I0720 21:30:04.955991 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ङ queue_depth=2330 I0720 21:30:04.955994 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=च queue_depth=2331 I0720 21:30:04.955997 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=छ queue_depth=2332 I0720 21:30:04.956000 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ज queue_depth=2333 I0720 21:30:04.956003 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=झ queue_depth=2334 I0720 21:30:04.956006 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ञ queue_depth=2335 I0720 21:30:04.956009 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ट queue_depth=2336 I0720 21:30:04.956012 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ठ queue_depth=2337 I0720 21:30:04.956015 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ड queue_depth=2338 I0720 21:30:04.956018 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ढ queue_depth=2339 I0720 21:30:04.956021 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ण queue_depth=2340 I0720 21:30:04.956024 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=त queue_depth=2341 I0720 21:30:04.956026 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=थ queue_depth=2342 I0720 21:30:04.956029 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=द queue_depth=2343 I0720 21:30:04.956032 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ध queue_depth=2344 I0720 21:30:04.956035 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=न queue_depth=2345 I0720 21:30:04.956039 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऩ queue_depth=2346 I0720 21:30:04.956042 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=प queue_depth=2347 I0720 21:30:04.956045 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=फ queue_depth=2348 I0720 21:30:04.956048 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ब queue_depth=2349 I0720 21:30:04.956051 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=भ queue_depth=2350 I0720 21:30:04.956054 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=म queue_depth=2351 I0720 21:30:04.956057 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=य queue_depth=2352 I0720 21:30:04.956060 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=र queue_depth=2353 I0720 21:30:04.956063 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऱ queue_depth=2354 I0720 21:30:04.956066 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ल queue_depth=2355 I0720 21:30:04.956069 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ळ queue_depth=2356 I0720 21:30:04.956072 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऴ queue_depth=2357 I0720 21:30:04.956076 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=व queue_depth=2358 I0720 21:30:04.956079 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=श queue_depth=2359 I0720 21:30:04.956082 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ष queue_depth=2360 I0720 21:30:04.956084 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=स queue_depth=2361 I0720 21:30:04.956087 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ह queue_depth=2362 I0720 21:30:04.956090 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऺ queue_depth=2363 I0720 21:30:04.956093 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऻ queue_depth=2364 I0720 21:30:04.956096 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=़ queue_depth=2365 I0720 21:30:04.956099 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ऽ queue_depth=2366 I0720 21:30:04.956102 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ा queue_depth=2367 I0720 21:30:04.956105 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ि queue_depth=2368 I0720 21:30:04.956108 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ी queue_depth=2369 I0720 21:30:04.956111 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ु queue_depth=2370 I0720 21:30:04.956114 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ू queue_depth=2371 I0720 21:30:04.956117 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ृ queue_depth=2372 I0720 21:30:04.956120 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॄ queue_depth=2373 I0720 21:30:04.956122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॅ queue_depth=2374 I0720 21:30:04.956125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॆ queue_depth=2375 I0720 21:30:04.956128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=े queue_depth=2376 I0720 21:30:04.956131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ै queue_depth=2377 I0720 21:30:04.956133 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॉ queue_depth=2378 I0720 21:30:04.956136 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॊ queue_depth=2379 I0720 21:30:04.956139 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ो queue_depth=2380 I0720 21:30:04.956143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ौ queue_depth=2381 I0720 21:30:04.956149 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=् queue_depth=2382 I0720 21:30:04.956155 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॎ queue_depth=2383 I0720 21:30:04.956158 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॏ queue_depth=2384 I0720 21:30:04.956161 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॐ queue_depth=2385 I0720 21:30:04.956164 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=॑ queue_depth=2386 I0720 21:30:04.956167 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=॒ queue_depth=2387 I0720 21:30:04.956170 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=॓ queue_depth=2388 I0720 21:30:04.956173 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=॔ queue_depth=2389 I0720 21:30:04.956176 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॕ queue_depth=2390 I0720 21:30:04.956179 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॖ queue_depth=2391 I0720 21:30:04.956182 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॗ queue_depth=2392 I0720 21:30:04.956185 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=क़ queue_depth=2393 I0720 21:30:04.956188 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ख़ queue_depth=2394 I0720 21:30:04.956191 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ग़ queue_depth=2395 I0720 21:30:04.956194 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ज़ queue_depth=2396 I0720 21:30:04.956197 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ड़ queue_depth=2397 I0720 21:30:04.956200 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ढ़ queue_depth=2398 I0720 21:30:04.956203 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=फ़ queue_depth=2399 I0720 21:30:04.956205 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=य़ queue_depth=2400 I0720 21:30:04.956208 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॠ queue_depth=2401 I0720 21:30:04.956211 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॡ queue_depth=2402 I0720 21:30:04.956214 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॢ queue_depth=2403 I0720 21:30:04.956217 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॣ queue_depth=2404 I0720 21:30:04.956220 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=। queue_depth=2405 I0720 21:30:04.956224 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=॥ queue_depth=2406 I0720 21:30:04.956234 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=० queue_depth=2407 I0720 21:30:04.956237 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=१ queue_depth=2408 I0720 21:30:04.956240 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=२ queue_depth=2409 I0720 21:30:04.956243 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=३ queue_depth=2410 I0720 21:30:04.956246 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=४ queue_depth=2411 I0720 21:30:04.956249 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=५ queue_depth=2412 I0720 21:30:04.956252 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=६ queue_depth=2413 I0720 21:30:04.956255 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=७ queue_depth=2414 I0720 21:30:04.956258 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=८ queue_depth=2415 I0720 21:30:04.956260 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=९ queue_depth=2416 I0720 21:30:04.956263 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=॰ queue_depth=2417 I0720 21:30:04.956266 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॱ queue_depth=2418 I0720 21:30:04.956270 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॲ queue_depth=2419 I0720 21:30:04.956273 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॳ queue_depth=2420 I0720 21:30:04.956276 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॴ queue_depth=2421 I0720 21:30:04.956279 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॵ queue_depth=2422 I0720 21:30:04.956282 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॶ queue_depth=2423 I0720 21:30:04.956285 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॷ queue_depth=2424 I0720 21:30:04.956288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॸ queue_depth=2425 I0720 21:30:04.956291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॹ queue_depth=2426 I0720 21:30:04.956293 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॺ queue_depth=2427 I0720 21:30:04.956296 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॻ queue_depth=2428 I0720 21:30:04.956299 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॼ queue_depth=2429 I0720 21:30:04.956302 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॽ queue_depth=2430 I0720 21:30:04.956305 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॾ queue_depth=2431 I0720 21:30:04.956308 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ॿ queue_depth=2432 I0720 21:30:04.956310 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঀ queue_depth=2433 I0720 21:30:04.956313 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঁ queue_depth=2434 I0720 21:30:04.956316 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ং queue_depth=2435 I0720 21:30:04.956319 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঃ queue_depth=2436 I0720 21:30:04.956322 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঄ queue_depth=2437 I0720 21:30:04.956326 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=অ queue_depth=2438 I0720 21:30:04.956329 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=আ queue_depth=2439 I0720 21:30:04.956332 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ই queue_depth=2440 I0720 21:30:04.956335 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঈ queue_depth=2441 I0720 21:30:04.956338 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=উ queue_depth=2442 I0720 21:30:04.956341 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঊ queue_depth=2443 I0720 21:30:04.956344 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঋ queue_depth=2444 I0720 21:30:04.956346 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঌ queue_depth=2445 I0720 21:30:04.956349 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঍ queue_depth=2446 I0720 21:30:04.956352 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঎ queue_depth=2447 I0720 21:30:04.956355 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=এ queue_depth=2448 I0720 21:30:04.956358 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঐ queue_depth=2449 I0720 21:30:04.956361 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঑ queue_depth=2450 I0720 21:30:04.956364 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঒ queue_depth=2451 I0720 21:30:04.956367 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ও queue_depth=2452 I0720 21:30:04.956370 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঔ queue_depth=2453 I0720 21:30:04.956373 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ক queue_depth=2454 I0720 21:30:04.956376 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=খ queue_depth=2455 I0720 21:30:04.956379 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=গ queue_depth=2456 I0720 21:30:04.956381 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঘ queue_depth=2457 I0720 21:30:04.956395 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঙ queue_depth=2458 I0720 21:30:04.956398 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=চ queue_depth=2459 I0720 21:30:04.956401 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ছ queue_depth=2460 I0720 21:30:04.956404 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=জ queue_depth=2461 I0720 21:30:04.956407 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঝ queue_depth=2462 I0720 21:30:04.956410 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঞ queue_depth=2463 I0720 21:30:04.956413 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ট queue_depth=2464 I0720 21:30:04.956416 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঠ queue_depth=2465 I0720 21:30:04.956419 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ড queue_depth=2466 I0720 21:30:04.956422 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঢ queue_depth=2467 I0720 21:30:04.956425 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ণ queue_depth=2468 I0720 21:30:04.956428 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ত queue_depth=2469 I0720 21:30:04.956431 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=থ queue_depth=2470 I0720 21:30:04.956434 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=দ queue_depth=2471 I0720 21:30:04.956437 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ধ queue_depth=2472 I0720 21:30:04.956441 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ন queue_depth=2473 I0720 21:30:04.956443 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঩ queue_depth=2474 I0720 21:30:04.956446 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=প queue_depth=2475 I0720 21:30:04.956449 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ফ queue_depth=2476 I0720 21:30:04.956452 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ব queue_depth=2477 I0720 21:30:04.956455 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ভ queue_depth=2478 I0720 21:30:04.956458 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ম queue_depth=2479 I0720 21:30:04.956461 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=য queue_depth=2480 I0720 21:30:04.956464 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=র queue_depth=2481 I0720 21:30:04.956467 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঱ queue_depth=2482 I0720 21:30:04.956470 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ল queue_depth=2483 I0720 21:30:04.956473 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঳ queue_depth=2484 I0720 21:30:04.956477 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঴ queue_depth=2485 I0720 21:30:04.956480 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঵ queue_depth=2486 I0720 21:30:04.956483 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=শ queue_depth=2487 I0720 21:30:04.956485 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ষ queue_depth=2488 I0720 21:30:04.956488 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=স queue_depth=2489 I0720 21:30:04.956491 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=হ queue_depth=2490 I0720 21:30:04.956494 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঺ queue_depth=2491 I0720 21:30:04.956498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=঻ queue_depth=2492 I0720 21:30:04.956501 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=় queue_depth=2493 I0720 21:30:04.956504 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঽ queue_depth=2494 I0720 21:30:04.956507 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=া queue_depth=2495 I0720 21:30:04.956509 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ি queue_depth=2496 I0720 21:30:04.956512 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ী queue_depth=2497 I0720 21:30:04.956515 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ু queue_depth=2498 I0720 21:30:04.956518 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ূ queue_depth=2499 I0720 21:30:04.956522 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৃ queue_depth=2500 I0720 21:30:04.956525 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৄ queue_depth=2501 I0720 21:30:04.956528 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৅ queue_depth=2502 I0720 21:30:04.956531 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৆ queue_depth=2503 I0720 21:30:04.956534 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ে queue_depth=2504 I0720 21:30:04.956537 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৈ queue_depth=2505 I0720 21:30:04.956541 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৉ queue_depth=2506 I0720 21:30:04.956544 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৊ queue_depth=2507 I0720 21:30:04.956548 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ো queue_depth=2508 I0720 21:30:04.956551 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৌ queue_depth=2509 I0720 21:30:04.956554 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=্ queue_depth=2510 I0720 21:30:04.956557 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৎ queue_depth=2511 I0720 21:30:04.956559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৏ queue_depth=2512 I0720 21:30:04.956562 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৐ queue_depth=2513 I0720 21:30:04.956565 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৑ queue_depth=2514 I0720 21:30:04.956568 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৒ queue_depth=2515 I0720 21:30:04.956571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৓ queue_depth=2516 I0720 21:30:04.956574 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৔ queue_depth=2517 I0720 21:30:04.956577 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৕ queue_depth=2518 I0720 21:30:04.956581 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৖ queue_depth=2519 I0720 21:30:04.956584 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৗ queue_depth=2520 I0720 21:30:04.956587 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৘ queue_depth=2521 I0720 21:30:04.956589 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৙ queue_depth=2522 I0720 21:30:04.956593 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৚ queue_depth=2523 I0720 21:30:04.956596 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৛ queue_depth=2524 I0720 21:30:04.956599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ড় queue_depth=2525 I0720 21:30:04.956602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ঢ় queue_depth=2526 I0720 21:30:04.956605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৞ queue_depth=2527 I0720 21:30:04.956607 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=য় queue_depth=2528 I0720 21:30:04.956610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৠ queue_depth=2529 I0720 21:30:04.956613 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৡ queue_depth=2530 I0720 21:30:04.956616 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৢ queue_depth=2531 I0720 21:30:04.956619 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৣ queue_depth=2532 I0720 21:30:04.956621 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৤ queue_depth=2533 I0720 21:30:04.956625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৥ queue_depth=2534 I0720 21:30:04.956627 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=০ queue_depth=2535 I0720 21:30:04.956630 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=১ queue_depth=2536 I0720 21:30:04.956633 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=২ queue_depth=2537 I0720 21:30:04.956636 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৩ queue_depth=2538 I0720 21:30:04.956640 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৪ queue_depth=2539 I0720 21:30:04.956643 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৫ queue_depth=2540 I0720 21:30:04.956646 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৬ queue_depth=2541 I0720 21:30:04.956649 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৭ queue_depth=2542 I0720 21:30:04.956652 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৮ queue_depth=2543 I0720 21:30:04.956654 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৯ queue_depth=2544 I0720 21:30:04.956657 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৰ queue_depth=2545 I0720 21:30:04.956660 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৱ queue_depth=2546 I0720 21:30:04.956663 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৲ queue_depth=2547 I0720 21:30:04.956666 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৳ queue_depth=2548 I0720 21:30:04.956669 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৴ queue_depth=2549 I0720 21:30:04.956671 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৵ queue_depth=2550 I0720 21:30:04.956675 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৶ queue_depth=2551 I0720 21:30:04.956678 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৷ queue_depth=2552 I0720 21:30:04.956681 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৸ queue_depth=2553 I0720 21:30:04.956684 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৹ queue_depth=2554 I0720 21:30:04.956686 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৺ queue_depth=2555 I0720 21:30:04.956689 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৻ queue_depth=2556 I0720 21:30:04.956692 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ৼ queue_depth=2557 I0720 21:30:04.956695 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৽ queue_depth=2558 I0720 21:30:04.956698 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৾ queue_depth=2559 I0720 21:30:04.956701 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=৿ queue_depth=2560 I0720 21:30:04.956704 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਀ queue_depth=2561 I0720 21:30:04.956707 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਁ queue_depth=2562 I0720 21:30:04.956710 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਂ queue_depth=2563 I0720 21:30:04.956713 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਃ queue_depth=2564 I0720 21:30:04.956716 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਄ queue_depth=2565 I0720 21:30:04.956719 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਅ queue_depth=2566 I0720 21:30:04.956722 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਆ queue_depth=2567 I0720 21:30:04.956725 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਇ queue_depth=2568 I0720 21:30:04.956728 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਈ queue_depth=2569 I0720 21:30:04.956731 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਉ queue_depth=2570 I0720 21:30:04.956734 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਊ queue_depth=2571 I0720 21:30:04.956737 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਋ queue_depth=2572 I0720 21:30:04.956740 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਌ queue_depth=2573 I0720 21:30:04.956743 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਍ queue_depth=2574 I0720 21:30:04.956746 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਎ queue_depth=2575 I0720 21:30:04.956749 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਏ queue_depth=2576 I0720 21:30:04.956752 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਐ queue_depth=2577 I0720 21:30:04.956755 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਑ queue_depth=2578 I0720 21:30:04.956758 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਒ queue_depth=2579 I0720 21:30:04.956760 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਓ queue_depth=2580 I0720 21:30:04.956763 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਔ queue_depth=2581 I0720 21:30:04.956766 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਕ queue_depth=2582 I0720 21:30:04.956769 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਖ queue_depth=2583 I0720 21:30:04.956773 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਗ queue_depth=2584 I0720 21:30:04.956776 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਘ queue_depth=2585 I0720 21:30:04.956779 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਙ queue_depth=2586 I0720 21:30:04.956782 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਚ queue_depth=2587 I0720 21:30:04.956785 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਛ queue_depth=2588 I0720 21:30:04.956788 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਜ queue_depth=2589 I0720 21:30:04.956791 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਝ queue_depth=2590 I0720 21:30:04.956794 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਞ queue_depth=2591 I0720 21:30:04.956797 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਟ queue_depth=2592 I0720 21:30:04.956799 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਠ queue_depth=2593 I0720 21:30:04.956802 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਡ queue_depth=2594 I0720 21:30:04.956806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਢ queue_depth=2595 I0720 21:30:04.956809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਣ queue_depth=2596 I0720 21:30:04.956812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਤ queue_depth=2597 I0720 21:30:04.956815 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਥ queue_depth=2598 I0720 21:30:04.956818 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਦ queue_depth=2599 I0720 21:30:04.956821 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਧ queue_depth=2600 I0720 21:30:04.956824 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਨ queue_depth=2601 I0720 21:30:04.956827 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਩ queue_depth=2602 I0720 21:30:04.956830 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਪ queue_depth=2603 I0720 21:30:04.956833 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਫ queue_depth=2604 I0720 21:30:04.956837 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਬ queue_depth=2605 I0720 21:30:04.956840 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਭ queue_depth=2606 I0720 21:30:04.956843 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਮ queue_depth=2607 I0720 21:30:04.956846 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਯ queue_depth=2608 I0720 21:30:04.956849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਰ queue_depth=2609 I0720 21:30:04.956852 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਱ queue_depth=2610 I0720 21:30:04.956855 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਲ queue_depth=2611 I0720 21:30:04.956858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਲ਼ queue_depth=2612 I0720 21:30:04.956861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਴ queue_depth=2613 I0720 21:30:04.956864 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਵ queue_depth=2614 I0720 21:30:04.956867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਸ਼ queue_depth=2615 I0720 21:30:04.956870 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਷ queue_depth=2616 I0720 21:30:04.956873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਸ queue_depth=2617 I0720 21:30:04.956876 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਹ queue_depth=2618 I0720 21:30:04.956879 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਺ queue_depth=2619 I0720 21:30:04.956882 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਻ queue_depth=2620 I0720 21:30:04.956885 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਼ queue_depth=2621 I0720 21:30:04.956888 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=਽ queue_depth=2622 I0720 21:30:04.956891 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਾ queue_depth=2623 I0720 21:30:04.956894 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਿ queue_depth=2624 I0720 21:30:04.956897 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੀ queue_depth=2625 I0720 21:30:04.956900 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੁ queue_depth=2626 I0720 21:30:04.956903 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੂ queue_depth=2627 I0720 21:30:04.956905 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੃ queue_depth=2628 I0720 21:30:04.956908 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੄ queue_depth=2629 I0720 21:30:04.956911 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੅ queue_depth=2630 I0720 21:30:04.956914 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੆ queue_depth=2631 I0720 21:30:04.956917 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੇ queue_depth=2632 I0720 21:30:04.956920 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੈ queue_depth=2633 I0720 21:30:04.956923 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੉ queue_depth=2634 I0720 21:30:04.956926 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੊ queue_depth=2635 I0720 21:30:04.956929 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੋ queue_depth=2636 I0720 21:30:04.956931 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੌ queue_depth=2637 I0720 21:30:04.956935 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੍ queue_depth=2638 I0720 21:30:04.956938 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੎ queue_depth=2639 I0720 21:30:04.956941 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੏ queue_depth=2640 I0720 21:30:04.956944 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੐ queue_depth=2641 I0720 21:30:04.956947 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੑ queue_depth=2642 I0720 21:30:04.956950 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੒ queue_depth=2643 I0720 21:30:04.956953 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੓ queue_depth=2644 I0720 21:30:04.956956 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੔ queue_depth=2645 I0720 21:30:04.956958 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੕ queue_depth=2646 I0720 21:30:04.956961 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੖ queue_depth=2647 I0720 21:30:04.956964 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੗ queue_depth=2648 I0720 21:30:04.956967 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੘ queue_depth=2649 I0720 21:30:04.956972 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਖ਼ queue_depth=2650 I0720 21:30:04.956975 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਗ਼ queue_depth=2651 I0720 21:30:04.956977 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਜ਼ queue_depth=2652 I0720 21:30:04.956980 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੜ queue_depth=2653 I0720 21:30:04.956983 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੝ queue_depth=2654 I0720 21:30:04.956986 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ਫ਼ queue_depth=2655 I0720 21:30:04.956989 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੟ queue_depth=2656 I0720 21:30:04.956992 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੠ queue_depth=2657 I0720 21:30:04.956996 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੡ queue_depth=2658 I0720 21:30:04.956999 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੢ queue_depth=2659 I0720 21:30:04.957002 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੣ queue_depth=2660 I0720 21:30:04.957005 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੤ queue_depth=2661 I0720 21:30:04.957008 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੥ queue_depth=2662 I0720 21:30:04.957011 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੦ queue_depth=2663 I0720 21:30:04.957014 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੧ queue_depth=2664 I0720 21:30:04.957018 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੨ queue_depth=2665 I0720 21:30:04.957021 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੩ queue_depth=2666 I0720 21:30:04.957024 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੪ queue_depth=2667 I0720 21:30:04.957027 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੫ queue_depth=2668 I0720 21:30:04.957030 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੬ queue_depth=2669 I0720 21:30:04.957033 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੭ queue_depth=2670 I0720 21:30:04.957037 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੮ queue_depth=2671 I0720 21:30:04.957040 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੯ queue_depth=2672 I0720 21:30:04.957043 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੰ queue_depth=2673 I0720 21:30:04.957046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੱ queue_depth=2674 I0720 21:30:04.957049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੲ queue_depth=2675 I0720 21:30:04.957052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੳ queue_depth=2676 I0720 21:30:04.957055 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੴ queue_depth=2677 I0720 21:30:04.957058 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ੵ queue_depth=2678 I0720 21:30:04.957061 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੶ queue_depth=2679 I0720 21:30:04.957064 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੷ queue_depth=2680 I0720 21:30:04.957067 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੸ queue_depth=2681 I0720 21:30:04.957069 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੹ queue_depth=2682 I0720 21:30:04.957074 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੺ queue_depth=2683 I0720 21:30:04.957077 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੻ queue_depth=2684 I0720 21:30:04.957080 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੼ queue_depth=2685 I0720 21:30:04.957083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੽ queue_depth=2686 I0720 21:30:04.957086 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੾ queue_depth=2687 I0720 21:30:04.957089 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=੿ queue_depth=2688 I0720 21:30:04.957092 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઀ queue_depth=2689 I0720 21:30:04.957095 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઁ queue_depth=2690 I0720 21:30:04.957098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ં queue_depth=2691 I0720 21:30:04.957101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઃ queue_depth=2692 I0720 21:30:04.957104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઄ queue_depth=2693 I0720 21:30:04.957107 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=અ queue_depth=2694 I0720 21:30:04.957110 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=આ queue_depth=2695 I0720 21:30:04.957113 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઇ queue_depth=2696 I0720 21:30:04.957116 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઈ queue_depth=2697 I0720 21:30:04.957119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઉ queue_depth=2698 I0720 21:30:04.957122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઊ queue_depth=2699 I0720 21:30:04.957125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઋ queue_depth=2700 I0720 21:30:04.957128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઌ queue_depth=2701 I0720 21:30:04.957131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઍ queue_depth=2702 I0720 21:30:04.957133 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઎ queue_depth=2703 I0720 21:30:04.957137 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=એ queue_depth=2704 I0720 21:30:04.957140 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઐ queue_depth=2705 I0720 21:30:04.957143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઑ queue_depth=2706 I0720 21:30:04.957146 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઒ queue_depth=2707 I0720 21:30:04.957148 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઓ queue_depth=2708 I0720 21:30:04.957151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઔ queue_depth=2709 I0720 21:30:04.957154 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ક queue_depth=2710 I0720 21:30:04.957157 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ખ queue_depth=2711 I0720 21:30:04.957160 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ગ queue_depth=2712 I0720 21:30:04.957163 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઘ queue_depth=2713 I0720 21:30:04.957166 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઙ queue_depth=2714 I0720 21:30:04.957169 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ચ queue_depth=2715 I0720 21:30:04.957172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=છ queue_depth=2716 I0720 21:30:04.957175 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=જ queue_depth=2717 I0720 21:30:04.957178 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઝ queue_depth=2718 I0720 21:30:04.957181 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઞ queue_depth=2719 I0720 21:30:04.957184 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ટ queue_depth=2720 I0720 21:30:04.957187 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઠ queue_depth=2721 I0720 21:30:04.957200 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ડ queue_depth=2722 I0720 21:30:04.957203 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઢ queue_depth=2723 I0720 21:30:04.957206 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ણ queue_depth=2724 I0720 21:30:04.957209 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ત queue_depth=2725 I0720 21:30:04.957212 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=થ queue_depth=2726 I0720 21:30:04.957215 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=દ queue_depth=2727 I0720 21:30:04.957218 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ધ queue_depth=2728 I0720 21:30:04.957220 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ન queue_depth=2729 I0720 21:30:04.957223 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઩ queue_depth=2730 I0720 21:30:04.957258 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=પ queue_depth=2731 I0720 21:30:04.957263 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ફ queue_depth=2732 I0720 21:30:04.957266 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=બ queue_depth=2733 I0720 21:30:04.957270 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ભ queue_depth=2734 I0720 21:30:04.957273 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=મ queue_depth=2735 I0720 21:30:04.957276 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ય queue_depth=2736 I0720 21:30:04.957279 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ર queue_depth=2737 I0720 21:30:04.957282 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઱ queue_depth=2738 I0720 21:30:04.957285 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=લ queue_depth=2739 I0720 21:30:04.957288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ળ queue_depth=2740 I0720 21:30:04.957291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઴ queue_depth=2741 I0720 21:30:04.957294 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=વ queue_depth=2742 I0720 21:30:04.957297 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=શ queue_depth=2743 I0720 21:30:04.957300 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ષ queue_depth=2744 I0720 21:30:04.957303 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=સ queue_depth=2745 I0720 21:30:04.957306 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=હ queue_depth=2746 I0720 21:30:04.957309 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઺ queue_depth=2747 I0720 21:30:04.957311 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઻ queue_depth=2748 I0720 21:30:04.957315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=઼ queue_depth=2749 I0720 21:30:04.957318 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ઽ queue_depth=2750 I0720 21:30:04.957321 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ા queue_depth=2751 I0720 21:30:04.957324 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=િ queue_depth=2752 I0720 21:30:04.957327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ી queue_depth=2753 I0720 21:30:04.957330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ુ queue_depth=2754 I0720 21:30:04.957333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૂ queue_depth=2755 I0720 21:30:04.957336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૃ queue_depth=2756 I0720 21:30:04.957339 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૄ queue_depth=2757 I0720 21:30:04.957341 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૅ queue_depth=2758 I0720 21:30:04.957344 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૆ queue_depth=2759 I0720 21:30:04.957348 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ે queue_depth=2760 I0720 21:30:04.957350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૈ queue_depth=2761 I0720 21:30:04.957354 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૉ queue_depth=2762 I0720 21:30:04.957356 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૊ queue_depth=2763 I0720 21:30:04.957359 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ો queue_depth=2764 I0720 21:30:04.957363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૌ queue_depth=2765 I0720 21:30:04.957366 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=્ queue_depth=2766 I0720 21:30:04.957369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૎ queue_depth=2767 I0720 21:30:04.957372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૏ queue_depth=2768 I0720 21:30:04.957375 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૐ queue_depth=2769 I0720 21:30:04.957378 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૑ queue_depth=2770 I0720 21:30:04.957381 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૒ queue_depth=2771 I0720 21:30:04.957384 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૓ queue_depth=2772 I0720 21:30:04.957387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૔ queue_depth=2773 I0720 21:30:04.957390 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૕ queue_depth=2774 I0720 21:30:04.957393 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૖ queue_depth=2775 I0720 21:30:04.957396 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૗ queue_depth=2776 I0720 21:30:04.957399 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૘ queue_depth=2777 I0720 21:30:04.957402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૙ queue_depth=2778 I0720 21:30:04.957405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૚ queue_depth=2779 I0720 21:30:04.957408 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૛ queue_depth=2780 I0720 21:30:04.957411 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૜ queue_depth=2781 I0720 21:30:04.957415 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૝ queue_depth=2782 I0720 21:30:04.957417 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૞ queue_depth=2783 I0720 21:30:04.957421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૟ queue_depth=2784 I0720 21:30:04.957424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૠ queue_depth=2785 I0720 21:30:04.957427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૡ queue_depth=2786 I0720 21:30:04.957430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૢ queue_depth=2787 I0720 21:30:04.957433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૣ queue_depth=2788 I0720 21:30:04.957436 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૤ queue_depth=2789 I0720 21:30:04.957439 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૥ queue_depth=2790 I0720 21:30:04.957442 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૦ queue_depth=2791 I0720 21:30:04.957444 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૧ queue_depth=2792 I0720 21:30:04.957447 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૨ queue_depth=2793 I0720 21:30:04.957450 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૩ queue_depth=2794 I0720 21:30:04.957453 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૪ queue_depth=2795 I0720 21:30:04.957456 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૫ queue_depth=2796 I0720 21:30:04.957459 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૬ queue_depth=2797 I0720 21:30:04.957462 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૭ queue_depth=2798 I0720 21:30:04.957465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૮ queue_depth=2799 I0720 21:30:04.957468 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૯ queue_depth=2800 I0720 21:30:04.957471 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૰ queue_depth=2801 I0720 21:30:04.957474 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૱ queue_depth=2802 I0720 21:30:04.957477 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૲ queue_depth=2803 I0720 21:30:04.957479 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૳ queue_depth=2804 I0720 21:30:04.957482 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૴ queue_depth=2805 I0720 21:30:04.957485 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૵ queue_depth=2806 I0720 21:30:04.957488 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૶ queue_depth=2807 I0720 21:30:04.957491 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૷ queue_depth=2808 I0720 21:30:04.957494 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૸ queue_depth=2809 I0720 21:30:04.957497 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૹ queue_depth=2810 I0720 21:30:04.957500 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૺ queue_depth=2811 I0720 21:30:04.957503 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૻ queue_depth=2812 I0720 21:30:04.957506 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ૼ queue_depth=2813 I0720 21:30:04.957509 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૽ queue_depth=2814 I0720 21:30:04.957513 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૾ queue_depth=2815 I0720 21:30:04.957517 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=૿ queue_depth=2816 I0720 21:30:04.957520 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଀ queue_depth=2817 I0720 21:30:04.957523 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଁ queue_depth=2818 I0720 21:30:04.957526 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଂ queue_depth=2819 I0720 21:30:04.957529 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଃ queue_depth=2820 I0720 21:30:04.957532 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଄ queue_depth=2821 I0720 21:30:04.957535 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଅ queue_depth=2822 I0720 21:30:04.957538 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଆ queue_depth=2823 I0720 21:30:04.957541 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଇ queue_depth=2824 I0720 21:30:04.957544 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଈ queue_depth=2825 I0720 21:30:04.957547 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଉ queue_depth=2826 I0720 21:30:04.957550 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଊ queue_depth=2827 I0720 21:30:04.957552 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଋ queue_depth=2828 I0720 21:30:04.957555 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଌ queue_depth=2829 I0720 21:30:04.957559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଍ queue_depth=2830 I0720 21:30:04.957563 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଎ queue_depth=2831 I0720 21:30:04.957566 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଏ queue_depth=2832 I0720 21:30:04.957569 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଐ queue_depth=2833 I0720 21:30:04.957571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଑ queue_depth=2834 I0720 21:30:04.957574 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଒ queue_depth=2835 I0720 21:30:04.957577 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଓ queue_depth=2836 I0720 21:30:04.957580 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଔ queue_depth=2837 I0720 21:30:04.957583 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=କ queue_depth=2838 I0720 21:30:04.957586 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଖ queue_depth=2839 I0720 21:30:04.957589 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଗ queue_depth=2840 I0720 21:30:04.957592 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଘ queue_depth=2841 I0720 21:30:04.957595 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଙ queue_depth=2842 I0720 21:30:04.957597 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଚ queue_depth=2843 I0720 21:30:04.957600 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଛ queue_depth=2844 I0720 21:30:04.957603 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଜ queue_depth=2845 I0720 21:30:04.957607 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଝ queue_depth=2846 I0720 21:30:04.957610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଞ queue_depth=2847 I0720 21:30:04.957614 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଟ queue_depth=2848 I0720 21:30:04.957617 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଠ queue_depth=2849 I0720 21:30:04.957620 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଡ queue_depth=2850 I0720 21:30:04.957623 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଢ queue_depth=2851 I0720 21:30:04.957626 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଣ queue_depth=2852 I0720 21:30:04.957629 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ତ queue_depth=2853 I0720 21:30:04.957632 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଥ queue_depth=2854 I0720 21:30:04.957635 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଦ queue_depth=2855 I0720 21:30:04.957638 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଧ queue_depth=2856 I0720 21:30:04.957641 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ନ queue_depth=2857 I0720 21:30:04.957644 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଩ queue_depth=2858 I0720 21:30:04.957648 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ପ queue_depth=2859 I0720 21:30:04.957651 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଫ queue_depth=2860 I0720 21:30:04.957654 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ବ queue_depth=2861 I0720 21:30:04.957656 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଭ queue_depth=2862 I0720 21:30:04.957659 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ମ queue_depth=2863 I0720 21:30:04.957663 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଯ queue_depth=2864 I0720 21:30:04.957666 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ର queue_depth=2865 I0720 21:30:04.957669 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଱ queue_depth=2866 I0720 21:30:04.957672 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଲ queue_depth=2867 I0720 21:30:04.957675 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଳ queue_depth=2868 I0720 21:30:04.957678 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଴ queue_depth=2869 I0720 21:30:04.957680 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଵ queue_depth=2870 I0720 21:30:04.957683 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଶ queue_depth=2871 I0720 21:30:04.957686 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଷ queue_depth=2872 I0720 21:30:04.957690 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ସ queue_depth=2873 I0720 21:30:04.957693 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ହ queue_depth=2874 I0720 21:30:04.957696 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଺ queue_depth=2875 I0720 21:30:04.957698 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଻ queue_depth=2876 I0720 21:30:04.957701 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=଼ queue_depth=2877 I0720 21:30:04.957704 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଽ queue_depth=2878 I0720 21:30:04.957707 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ା queue_depth=2879 I0720 21:30:04.957710 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ି queue_depth=2880 I0720 21:30:04.957720 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୀ queue_depth=2881 I0720 21:30:04.957723 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୁ queue_depth=2882 I0720 21:30:04.957726 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୂ queue_depth=2883 I0720 21:30:04.957729 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୃ queue_depth=2884 I0720 21:30:04.957732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୄ queue_depth=2885 I0720 21:30:04.957735 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୅ queue_depth=2886 I0720 21:30:04.957739 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୆ queue_depth=2887 I0720 21:30:04.957741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=େ queue_depth=2888 I0720 21:30:04.957745 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୈ queue_depth=2889 I0720 21:30:04.957748 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୉ queue_depth=2890 I0720 21:30:04.957751 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୊ queue_depth=2891 I0720 21:30:04.957754 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୋ queue_depth=2892 I0720 21:30:04.957757 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୌ queue_depth=2893 I0720 21:30:04.957760 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୍ queue_depth=2894 I0720 21:30:04.957763 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୎ queue_depth=2895 I0720 21:30:04.957765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୏ queue_depth=2896 I0720 21:30:04.957769 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୐ queue_depth=2897 I0720 21:30:04.957772 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୑ queue_depth=2898 I0720 21:30:04.957775 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୒ queue_depth=2899 I0720 21:30:04.957778 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୓ queue_depth=2900 I0720 21:30:04.957781 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୔ queue_depth=2901 I0720 21:30:04.957784 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୕ queue_depth=2902 I0720 21:30:04.957787 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୖ queue_depth=2903 I0720 21:30:04.957790 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୗ queue_depth=2904 I0720 21:30:04.957793 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୘ queue_depth=2905 I0720 21:30:04.957795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୙ queue_depth=2906 I0720 21:30:04.957798 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୚ queue_depth=2907 I0720 21:30:04.957801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୛ queue_depth=2908 I0720 21:30:04.957804 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଡ଼ queue_depth=2909 I0720 21:30:04.957808 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ଢ଼ queue_depth=2910 I0720 21:30:04.957811 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୞ queue_depth=2911 I0720 21:30:04.957814 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୟ queue_depth=2912 I0720 21:30:04.957817 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୠ queue_depth=2913 I0720 21:30:04.957821 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୡ queue_depth=2914 I0720 21:30:04.957824 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୢ queue_depth=2915 I0720 21:30:04.957827 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୣ queue_depth=2916 I0720 21:30:04.957830 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୤ queue_depth=2917 I0720 21:30:04.957833 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୥ queue_depth=2918 I0720 21:30:04.957836 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୦ queue_depth=2919 I0720 21:30:04.957839 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୧ queue_depth=2920 I0720 21:30:04.957842 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୨ queue_depth=2921 I0720 21:30:04.957846 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୩ queue_depth=2922 I0720 21:30:04.957849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୪ queue_depth=2923 I0720 21:30:04.957852 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୫ queue_depth=2924 I0720 21:30:04.957855 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୬ queue_depth=2925 I0720 21:30:04.957858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୭ queue_depth=2926 I0720 21:30:04.957860 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୮ queue_depth=2927 I0720 21:30:04.957863 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୯ queue_depth=2928 I0720 21:30:04.957867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୰ queue_depth=2929 I0720 21:30:04.957870 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ୱ queue_depth=2930 I0720 21:30:04.957874 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୲ queue_depth=2931 I0720 21:30:04.957877 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୳ queue_depth=2932 I0720 21:30:04.957879 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୴ queue_depth=2933 I0720 21:30:04.957882 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୵ queue_depth=2934 I0720 21:30:04.957885 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୶ queue_depth=2935 I0720 21:30:04.957888 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୷ queue_depth=2936 I0720 21:30:04.957892 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୸ queue_depth=2937 I0720 21:30:04.957895 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୹ queue_depth=2938 I0720 21:30:04.957897 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୺ queue_depth=2939 I0720 21:30:04.957900 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୻ queue_depth=2940 I0720 21:30:04.957903 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୼ queue_depth=2941 I0720 21:30:04.957906 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୽ queue_depth=2942 I0720 21:30:04.957909 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୾ queue_depth=2943 I0720 21:30:04.957912 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=୿ queue_depth=2944 I0720 21:30:04.957914 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஀ queue_depth=2945 I0720 21:30:04.957917 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஁ queue_depth=2946 I0720 21:30:04.957921 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஂ queue_depth=2947 I0720 21:30:04.957923 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஃ queue_depth=2948 I0720 21:30:04.957926 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஄ queue_depth=2949 I0720 21:30:04.957929 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=அ queue_depth=2950 I0720 21:30:04.957932 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஆ queue_depth=2951 I0720 21:30:04.957935 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=இ queue_depth=2952 I0720 21:30:04.957938 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஈ queue_depth=2953 I0720 21:30:04.957941 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=உ queue_depth=2954 I0720 21:30:04.957944 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஊ queue_depth=2955 I0720 21:30:04.957947 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஋ queue_depth=2956 I0720 21:30:04.957950 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஌ queue_depth=2957 I0720 21:30:04.957953 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஍ queue_depth=2958 I0720 21:30:04.957956 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=எ queue_depth=2959 I0720 21:30:04.957959 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஏ queue_depth=2960 I0720 21:30:04.957962 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஐ queue_depth=2961 I0720 21:30:04.957965 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஑ queue_depth=2962 I0720 21:30:04.957968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஒ queue_depth=2963 I0720 21:30:04.957971 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஓ queue_depth=2964 I0720 21:30:04.957974 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஔ queue_depth=2965 I0720 21:30:04.957977 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=க queue_depth=2966 I0720 21:30:04.957980 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஖ queue_depth=2967 I0720 21:30:04.957983 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஗ queue_depth=2968 I0720 21:30:04.957986 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஘ queue_depth=2969 I0720 21:30:04.957988 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ங queue_depth=2970 I0720 21:30:04.957991 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ச queue_depth=2971 I0720 21:30:04.957994 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஛ queue_depth=2972 I0720 21:30:04.957997 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஜ queue_depth=2973 I0720 21:30:04.958000 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஝ queue_depth=2974 I0720 21:30:04.958003 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஞ queue_depth=2975 I0720 21:30:04.958006 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ட queue_depth=2976 I0720 21:30:04.958009 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஠ queue_depth=2977 I0720 21:30:04.958012 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஡ queue_depth=2978 I0720 21:30:04.958015 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஢ queue_depth=2979 I0720 21:30:04.958018 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ண queue_depth=2980 I0720 21:30:04.958022 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=த queue_depth=2981 I0720 21:30:04.958025 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஥ queue_depth=2982 I0720 21:30:04.958028 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஦ queue_depth=2983 I0720 21:30:04.958030 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஧ queue_depth=2984 I0720 21:30:04.958033 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ந queue_depth=2985 I0720 21:30:04.958037 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ன queue_depth=2986 I0720 21:30:04.958039 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ப queue_depth=2987 I0720 21:30:04.958042 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஫ queue_depth=2988 I0720 21:30:04.958046 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஬ queue_depth=2989 I0720 21:30:04.958049 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஭ queue_depth=2990 I0720 21:30:04.958052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ம queue_depth=2991 I0720 21:30:04.958055 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ய queue_depth=2992 I0720 21:30:04.958058 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ர queue_depth=2993 I0720 21:30:04.958061 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ற queue_depth=2994 I0720 21:30:04.958064 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ல queue_depth=2995 I0720 21:30:04.958067 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ள queue_depth=2996 I0720 21:30:04.958070 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ழ queue_depth=2997 I0720 21:30:04.958073 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=வ queue_depth=2998 I0720 21:30:04.958076 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஶ queue_depth=2999 I0720 21:30:04.958079 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஷ queue_depth=3000 I0720 21:30:04.958082 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஸ queue_depth=3001 I0720 21:30:04.958085 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ஹ queue_depth=3002 I0720 21:30:04.958088 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஺ queue_depth=3003 I0720 21:30:04.958091 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஻ queue_depth=3004 I0720 21:30:04.958094 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஼ queue_depth=3005 I0720 21:30:04.958097 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=஽ queue_depth=3006 I0720 21:30:04.958100 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ா queue_depth=3007 I0720 21:30:04.958102 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ி queue_depth=3008 I0720 21:30:04.958105 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ீ queue_depth=3009 I0720 21:30:04.958108 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ு queue_depth=3010 I0720 21:30:04.958112 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ூ queue_depth=3011 I0720 21:30:04.958114 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௃ queue_depth=3012 I0720 21:30:04.958118 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௄ queue_depth=3013 I0720 21:30:04.958121 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௅ queue_depth=3014 I0720 21:30:04.958124 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ெ queue_depth=3015 I0720 21:30:04.958127 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ே queue_depth=3016 I0720 21:30:04.958129 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ை queue_depth=3017 I0720 21:30:04.958132 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௉ queue_depth=3018 I0720 21:30:04.958135 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ொ queue_depth=3019 I0720 21:30:04.958138 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ோ queue_depth=3020 I0720 21:30:04.958141 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ௌ queue_depth=3021 I0720 21:30:04.958144 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=் queue_depth=3022 I0720 21:30:04.958147 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௎ queue_depth=3023 I0720 21:30:04.958150 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௏ queue_depth=3024 I0720 21:30:04.958153 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ௐ queue_depth=3025 I0720 21:30:04.958156 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௑ queue_depth=3026 I0720 21:30:04.958159 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௒ queue_depth=3027 I0720 21:30:04.958161 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௓ queue_depth=3028 I0720 21:30:04.958165 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௔ queue_depth=3029 I0720 21:30:04.958169 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௕ queue_depth=3030 I0720 21:30:04.958175 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௖ queue_depth=3031 I0720 21:30:04.958178 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ௗ queue_depth=3032 I0720 21:30:04.958181 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௘ queue_depth=3033 I0720 21:30:04.958183 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௙ queue_depth=3034 I0720 21:30:04.958186 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௚ queue_depth=3035 I0720 21:30:04.958189 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௛ queue_depth=3036 I0720 21:30:04.958192 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௜ queue_depth=3037 I0720 21:30:04.958195 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௝ queue_depth=3038 I0720 21:30:04.958198 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௞ queue_depth=3039 I0720 21:30:04.958201 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௟ queue_depth=3040 I0720 21:30:04.958204 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௠ queue_depth=3041 I0720 21:30:04.958207 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௡ queue_depth=3042 I0720 21:30:04.958210 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௢ queue_depth=3043 I0720 21:30:04.958213 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௣ queue_depth=3044 I0720 21:30:04.958216 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௤ queue_depth=3045 I0720 21:30:04.958219 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௥ queue_depth=3046 I0720 21:30:04.958222 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௦ queue_depth=3047 I0720 21:30:04.958225 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௧ queue_depth=3048 I0720 21:30:04.958227 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௨ queue_depth=3049 I0720 21:30:04.958230 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௩ queue_depth=3050 I0720 21:30:04.958233 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௪ queue_depth=3051 I0720 21:30:04.958236 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௫ queue_depth=3052 I0720 21:30:04.958239 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௬ queue_depth=3053 I0720 21:30:04.958242 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௭ queue_depth=3054 I0720 21:30:04.958244 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௮ queue_depth=3055 I0720 21:30:04.958247 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௯ queue_depth=3056 I0720 21:30:04.958250 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௰ queue_depth=3057 I0720 21:30:04.958253 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௱ queue_depth=3058 I0720 21:30:04.958256 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௲ queue_depth=3059 I0720 21:30:04.958259 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௳ queue_depth=3060 I0720 21:30:04.958262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௴ queue_depth=3061 I0720 21:30:04.958265 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௵ queue_depth=3062 I0720 21:30:04.958268 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௶ queue_depth=3063 I0720 21:30:04.958271 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௷ queue_depth=3064 I0720 21:30:04.958274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௸ queue_depth=3065 I0720 21:30:04.958277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௹ queue_depth=3066 I0720 21:30:04.958280 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௺ queue_depth=3067 I0720 21:30:04.958283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௻ queue_depth=3068 I0720 21:30:04.958286 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௼ queue_depth=3069 I0720 21:30:04.958289 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௽ queue_depth=3070 I0720 21:30:04.958292 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௾ queue_depth=3071 I0720 21:30:04.958295 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=௿ queue_depth=3072 I0720 21:30:04.958298 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఀ queue_depth=3073 I0720 21:30:04.958301 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఁ queue_depth=3074 I0720 21:30:04.958303 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ం queue_depth=3075 I0720 21:30:04.958306 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ః queue_depth=3076 I0720 21:30:04.958309 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఄ queue_depth=3077 I0720 21:30:04.958312 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=అ queue_depth=3078 I0720 21:30:04.958315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఆ queue_depth=3079 I0720 21:30:04.958318 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఇ queue_depth=3080 I0720 21:30:04.958322 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఈ queue_depth=3081 I0720 21:30:04.958325 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఉ queue_depth=3082 I0720 21:30:04.958328 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఊ queue_depth=3083 I0720 21:30:04.958330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఋ queue_depth=3084 I0720 21:30:04.958333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఌ queue_depth=3085 I0720 21:30:04.958336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=఍ queue_depth=3086 I0720 21:30:04.958339 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఎ queue_depth=3087 I0720 21:30:04.958342 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఏ queue_depth=3088 I0720 21:30:04.958345 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఐ queue_depth=3089 I0720 21:30:04.958347 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=఑ queue_depth=3090 I0720 21:30:04.958350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఒ queue_depth=3091 I0720 21:30:04.958353 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఓ queue_depth=3092 I0720 21:30:04.958357 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఔ queue_depth=3093 I0720 21:30:04.958360 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=క queue_depth=3094 I0720 21:30:04.958363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఖ queue_depth=3095 I0720 21:30:04.958366 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=గ queue_depth=3096 I0720 21:30:04.958369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఘ queue_depth=3097 I0720 21:30:04.958372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఙ queue_depth=3098 I0720 21:30:04.958375 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=చ queue_depth=3099 I0720 21:30:04.958378 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఛ queue_depth=3100 I0720 21:30:04.958381 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=జ queue_depth=3101 I0720 21:30:04.958384 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఝ queue_depth=3102 I0720 21:30:04.958387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఞ queue_depth=3103 I0720 21:30:04.958390 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ట queue_depth=3104 I0720 21:30:04.958393 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఠ queue_depth=3105 I0720 21:30:04.958396 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=డ queue_depth=3106 I0720 21:30:04.958399 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఢ queue_depth=3107 I0720 21:30:04.958402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ణ queue_depth=3108 I0720 21:30:04.958405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=త queue_depth=3109 I0720 21:30:04.958407 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=థ queue_depth=3110 I0720 21:30:04.958410 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ద queue_depth=3111 I0720 21:30:04.958414 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ధ queue_depth=3112 I0720 21:30:04.958417 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=న queue_depth=3113 I0720 21:30:04.958420 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=఩ queue_depth=3114 I0720 21:30:04.958423 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ప queue_depth=3115 I0720 21:30:04.958425 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఫ queue_depth=3116 I0720 21:30:04.958428 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=బ queue_depth=3117 I0720 21:30:04.958431 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=భ queue_depth=3118 I0720 21:30:04.958434 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=మ queue_depth=3119 I0720 21:30:04.958437 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=య queue_depth=3120 I0720 21:30:04.958440 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ర queue_depth=3121 I0720 21:30:04.958443 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఱ queue_depth=3122 I0720 21:30:04.958446 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ల queue_depth=3123 I0720 21:30:04.958448 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ళ queue_depth=3124 I0720 21:30:04.958451 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఴ queue_depth=3125 I0720 21:30:04.958454 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=వ queue_depth=3126 I0720 21:30:04.958457 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=శ queue_depth=3127 I0720 21:30:04.958461 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ష queue_depth=3128 I0720 21:30:04.958463 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=స queue_depth=3129 I0720 21:30:04.958466 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=హ queue_depth=3130 I0720 21:30:04.958469 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=఺ queue_depth=3131 I0720 21:30:04.958472 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=఻ queue_depth=3132 I0720 21:30:04.958475 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=఼ queue_depth=3133 I0720 21:30:04.958478 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ఽ queue_depth=3134 I0720 21:30:04.958481 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ా queue_depth=3135 I0720 21:30:04.958483 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ి queue_depth=3136 I0720 21:30:04.958486 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ీ queue_depth=3137 I0720 21:30:04.958489 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ు queue_depth=3138 I0720 21:30:04.958492 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ూ queue_depth=3139 I0720 21:30:04.958495 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ృ queue_depth=3140 I0720 21:30:04.958498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౄ queue_depth=3141 I0720 21:30:04.958501 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౅ queue_depth=3142 I0720 21:30:04.958504 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ె queue_depth=3143 I0720 21:30:04.958507 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ే queue_depth=3144 I0720 21:30:04.958510 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ై queue_depth=3145 I0720 21:30:04.958513 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౉ queue_depth=3146 I0720 21:30:04.958516 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ొ queue_depth=3147 I0720 21:30:04.958519 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ో queue_depth=3148 I0720 21:30:04.958522 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౌ queue_depth=3149 I0720 21:30:04.958525 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=్ queue_depth=3150 I0720 21:30:04.958528 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౎ queue_depth=3151 I0720 21:30:04.958531 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౏ queue_depth=3152 I0720 21:30:04.958533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౐ queue_depth=3153 I0720 21:30:04.958536 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౑ queue_depth=3154 I0720 21:30:04.958539 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౒ queue_depth=3155 I0720 21:30:04.958542 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౓ queue_depth=3156 I0720 21:30:04.958545 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౔ queue_depth=3157 I0720 21:30:04.958548 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౕ queue_depth=3158 I0720 21:30:04.958550 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౖ queue_depth=3159 I0720 21:30:04.958553 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౗ queue_depth=3160 I0720 21:30:04.958557 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౘ queue_depth=3161 I0720 21:30:04.958560 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౙ queue_depth=3162 I0720 21:30:04.958563 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౚ queue_depth=3163 I0720 21:30:04.958566 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౛ queue_depth=3164 I0720 21:30:04.958569 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౜ queue_depth=3165 I0720 21:30:04.958573 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౝ queue_depth=3166 I0720 21:30:04.958575 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౞ queue_depth=3167 I0720 21:30:04.958578 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౟ queue_depth=3168 I0720 21:30:04.958581 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౠ queue_depth=3169 I0720 21:30:04.958584 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౡ queue_depth=3170 I0720 21:30:04.958587 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౢ queue_depth=3171 I0720 21:30:04.958590 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ౣ queue_depth=3172 I0720 21:30:04.958592 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౤ queue_depth=3173 I0720 21:30:04.958595 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౥ queue_depth=3174 I0720 21:30:04.958599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౦ queue_depth=3175 I0720 21:30:04.958602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౧ queue_depth=3176 I0720 21:30:04.958605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౨ queue_depth=3177 I0720 21:30:04.958608 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౩ queue_depth=3178 I0720 21:30:04.958611 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౪ queue_depth=3179 I0720 21:30:04.958614 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౫ queue_depth=3180 I0720 21:30:04.958617 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౬ queue_depth=3181 I0720 21:30:04.958620 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౭ queue_depth=3182 I0720 21:30:04.958622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౮ queue_depth=3183 I0720 21:30:04.958625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౯ queue_depth=3184 I0720 21:30:04.958629 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౰ queue_depth=3185 I0720 21:30:04.958632 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౱ queue_depth=3186 I0720 21:30:04.958635 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౲ queue_depth=3187 I0720 21:30:04.958638 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౳ queue_depth=3188 I0720 21:30:04.958641 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౴ queue_depth=3189 I0720 21:30:04.958644 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౵ queue_depth=3190 I0720 21:30:04.958646 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౶ queue_depth=3191 I0720 21:30:04.958649 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౷ queue_depth=3192 I0720 21:30:04.958652 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౸ queue_depth=3193 I0720 21:30:04.958656 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౹ queue_depth=3194 I0720 21:30:04.958659 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౺ queue_depth=3195 I0720 21:30:04.958662 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౻ queue_depth=3196 I0720 21:30:04.958665 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౼ queue_depth=3197 I0720 21:30:04.958668 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౽ queue_depth=3198 I0720 21:30:04.958671 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౾ queue_depth=3199 I0720 21:30:04.958674 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=౿ queue_depth=3200 I0720 21:30:04.958677 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಀ queue_depth=3201 I0720 21:30:04.958680 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಁ queue_depth=3202 I0720 21:30:04.958683 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಂ queue_depth=3203 I0720 21:30:04.958686 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಃ queue_depth=3204 I0720 21:30:04.958689 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಄ queue_depth=3205 I0720 21:30:04.958692 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಅ queue_depth=3206 I0720 21:30:04.958694 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಆ queue_depth=3207 I0720 21:30:04.958697 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಇ queue_depth=3208 I0720 21:30:04.958700 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಈ queue_depth=3209 I0720 21:30:04.958703 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಉ queue_depth=3210 I0720 21:30:04.958706 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಊ queue_depth=3211 I0720 21:30:04.958709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಋ queue_depth=3212 I0720 21:30:04.958712 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಌ queue_depth=3213 I0720 21:30:04.958715 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಍ queue_depth=3214 I0720 21:30:04.958718 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಎ queue_depth=3215 I0720 21:30:04.958721 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಏ queue_depth=3216 I0720 21:30:04.958724 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಐ queue_depth=3217 I0720 21:30:04.958727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಑ queue_depth=3218 I0720 21:30:04.958730 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಒ queue_depth=3219 I0720 21:30:04.958732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಓ queue_depth=3220 I0720 21:30:04.958735 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಔ queue_depth=3221 I0720 21:30:04.958738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಕ queue_depth=3222 I0720 21:30:04.958741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಖ queue_depth=3223 I0720 21:30:04.958744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಗ queue_depth=3224 I0720 21:30:04.958756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಘ queue_depth=3225 I0720 21:30:04.958759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಙ queue_depth=3226 I0720 21:30:04.958762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಚ queue_depth=3227 I0720 21:30:04.958765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಛ queue_depth=3228 I0720 21:30:04.958768 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಜ queue_depth=3229 I0720 21:30:04.958771 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಝ queue_depth=3230 I0720 21:30:04.958774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಞ queue_depth=3231 I0720 21:30:04.958777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಟ queue_depth=3232 I0720 21:30:04.958780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಠ queue_depth=3233 I0720 21:30:04.958783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಡ queue_depth=3234 I0720 21:30:04.958786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಢ queue_depth=3235 I0720 21:30:04.958788 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಣ queue_depth=3236 I0720 21:30:04.958792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ತ queue_depth=3237 I0720 21:30:04.958794 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಥ queue_depth=3238 I0720 21:30:04.958797 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ದ queue_depth=3239 I0720 21:30:04.958800 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಧ queue_depth=3240 I0720 21:30:04.958803 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ನ queue_depth=3241 I0720 21:30:04.958806 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಩ queue_depth=3242 I0720 21:30:04.958809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಪ queue_depth=3243 I0720 21:30:04.958812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಫ queue_depth=3244 I0720 21:30:04.958815 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಬ queue_depth=3245 I0720 21:30:04.958818 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಭ queue_depth=3246 I0720 21:30:04.958821 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಮ queue_depth=3247 I0720 21:30:04.958824 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಯ queue_depth=3248 I0720 21:30:04.958827 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ರ queue_depth=3249 I0720 21:30:04.958830 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಱ queue_depth=3250 I0720 21:30:04.958833 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಲ queue_depth=3251 I0720 21:30:04.958835 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಳ queue_depth=3252 I0720 21:30:04.958838 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಴ queue_depth=3253 I0720 21:30:04.958841 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ವ queue_depth=3254 I0720 21:30:04.958844 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಶ queue_depth=3255 I0720 21:30:04.958847 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಷ queue_depth=3256 I0720 21:30:04.958851 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಸ queue_depth=3257 I0720 21:30:04.958854 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಹ queue_depth=3258 I0720 21:30:04.958857 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಺ queue_depth=3259 I0720 21:30:04.958859 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಻ queue_depth=3260 I0720 21:30:04.958862 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=಼ queue_depth=3261 I0720 21:30:04.958866 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಽ queue_depth=3262 I0720 21:30:04.958869 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಾ queue_depth=3263 I0720 21:30:04.958872 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ಿ queue_depth=3264 I0720 21:30:04.958875 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೀ queue_depth=3265 I0720 21:30:04.958877 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ು queue_depth=3266 I0720 21:30:04.958881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೂ queue_depth=3267 I0720 21:30:04.958884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೃ queue_depth=3268 I0720 21:30:04.958887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೄ queue_depth=3269 I0720 21:30:04.958890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೅ queue_depth=3270 I0720 21:30:04.958893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೆ queue_depth=3271 I0720 21:30:04.958896 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೇ queue_depth=3272 I0720 21:30:04.958901 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೈ queue_depth=3273 I0720 21:30:04.958904 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೉ queue_depth=3274 I0720 21:30:04.958907 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೊ queue_depth=3275 I0720 21:30:04.958910 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೋ queue_depth=3276 I0720 21:30:04.958913 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೌ queue_depth=3277 I0720 21:30:04.958916 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=್ queue_depth=3278 I0720 21:30:04.958919 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೎ queue_depth=3279 I0720 21:30:04.958922 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೏ queue_depth=3280 I0720 21:30:04.958925 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೐ queue_depth=3281 I0720 21:30:04.958928 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೑ queue_depth=3282 I0720 21:30:04.958931 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೒ queue_depth=3283 I0720 21:30:04.958934 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೓ queue_depth=3284 I0720 21:30:04.958937 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೔ queue_depth=3285 I0720 21:30:04.958940 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೕ queue_depth=3286 I0720 21:30:04.958943 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೖ queue_depth=3287 I0720 21:30:04.958946 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೗ queue_depth=3288 I0720 21:30:04.958949 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೘ queue_depth=3289 I0720 21:30:04.958952 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೙ queue_depth=3290 I0720 21:30:04.958955 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೚ queue_depth=3291 I0720 21:30:04.958958 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೛ queue_depth=3292 I0720 21:30:04.958961 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೜ queue_depth=3293 I0720 21:30:04.958964 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೝ queue_depth=3294 I0720 21:30:04.958968 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೞ queue_depth=3295 I0720 21:30:04.958971 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೟ queue_depth=3296 I0720 21:30:04.958974 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೠ queue_depth=3297 I0720 21:30:04.958977 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೡ queue_depth=3298 I0720 21:30:04.958980 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೢ queue_depth=3299 I0720 21:30:04.958984 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೣ queue_depth=3300 I0720 21:30:04.958987 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೤ queue_depth=3301 I0720 21:30:04.958990 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೥ queue_depth=3302 I0720 21:30:04.958992 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೦ queue_depth=3303 I0720 21:30:04.958995 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೧ queue_depth=3304 I0720 21:30:04.958999 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೨ queue_depth=3305 I0720 21:30:04.959001 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೩ queue_depth=3306 I0720 21:30:04.959004 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೪ queue_depth=3307 I0720 21:30:04.959007 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೫ queue_depth=3308 I0720 21:30:04.959010 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೬ queue_depth=3309 I0720 21:30:04.959013 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೭ queue_depth=3310 I0720 21:30:04.959017 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೮ queue_depth=3311 I0720 21:30:04.959020 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೯ queue_depth=3312 I0720 21:30:04.959023 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೰ queue_depth=3313 I0720 21:30:04.959026 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೱ queue_depth=3314 I0720 21:30:04.959029 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೲ queue_depth=3315 I0720 21:30:04.959032 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ೳ queue_depth=3316 I0720 21:30:04.959035 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೴ queue_depth=3317 I0720 21:30:04.959038 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೵ queue_depth=3318 I0720 21:30:04.959041 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೶ queue_depth=3319 I0720 21:30:04.959044 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೷ queue_depth=3320 I0720 21:30:04.959047 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೸ queue_depth=3321 I0720 21:30:04.959050 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೹ queue_depth=3322 I0720 21:30:04.959053 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೺ queue_depth=3323 I0720 21:30:04.959056 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೻ queue_depth=3324 I0720 21:30:04.959059 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೼ queue_depth=3325 I0720 21:30:04.959062 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೽ queue_depth=3326 I0720 21:30:04.959065 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೾ queue_depth=3327 I0720 21:30:04.959068 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=೿ queue_depth=3328 I0720 21:30:04.959071 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഀ queue_depth=3329 I0720 21:30:04.959074 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഁ queue_depth=3330 I0720 21:30:04.959077 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ം queue_depth=3331 I0720 21:30:04.959080 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഃ queue_depth=3332 I0720 21:30:04.959083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഄ queue_depth=3333 I0720 21:30:04.959086 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=അ queue_depth=3334 I0720 21:30:04.959088 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ആ queue_depth=3335 I0720 21:30:04.959091 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഇ queue_depth=3336 I0720 21:30:04.959094 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഈ queue_depth=3337 I0720 21:30:04.959099 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഉ queue_depth=3338 I0720 21:30:04.959102 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഊ queue_depth=3339 I0720 21:30:04.959105 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഋ queue_depth=3340 I0720 21:30:04.959108 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഌ queue_depth=3341 I0720 21:30:04.959111 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=഍ queue_depth=3342 I0720 21:30:04.959114 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=എ queue_depth=3343 I0720 21:30:04.959117 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഏ queue_depth=3344 I0720 21:30:04.959120 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഐ queue_depth=3345 I0720 21:30:04.959123 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=഑ queue_depth=3346 I0720 21:30:04.959126 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഒ queue_depth=3347 I0720 21:30:04.959129 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഓ queue_depth=3348 I0720 21:30:04.959132 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഔ queue_depth=3349 I0720 21:30:04.959135 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ക queue_depth=3350 I0720 21:30:04.959138 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഖ queue_depth=3351 I0720 21:30:04.959141 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഗ queue_depth=3352 I0720 21:30:04.959144 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഘ queue_depth=3353 I0720 21:30:04.959147 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ങ queue_depth=3354 I0720 21:30:04.959150 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ച queue_depth=3355 I0720 21:30:04.959153 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഛ queue_depth=3356 I0720 21:30:04.959156 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ജ queue_depth=3357 I0720 21:30:04.959159 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഝ queue_depth=3358 I0720 21:30:04.959163 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഞ queue_depth=3359 I0720 21:30:04.959166 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ട queue_depth=3360 I0720 21:30:04.959169 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഠ queue_depth=3361 I0720 21:30:04.959172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഡ queue_depth=3362 I0720 21:30:04.959175 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഢ queue_depth=3363 I0720 21:30:04.959178 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ണ queue_depth=3364 I0720 21:30:04.959181 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ത queue_depth=3365 I0720 21:30:04.959195 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഥ queue_depth=3366 I0720 21:30:04.959201 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ദ queue_depth=3367 I0720 21:30:04.959204 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ധ queue_depth=3368 I0720 21:30:04.959207 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ന queue_depth=3369 I0720 21:30:04.959210 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഩ queue_depth=3370 I0720 21:30:04.959213 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=പ queue_depth=3371 I0720 21:30:04.959216 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഫ queue_depth=3372 I0720 21:30:04.959219 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ബ queue_depth=3373 I0720 21:30:04.959222 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഭ queue_depth=3374 I0720 21:30:04.959225 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=മ queue_depth=3375 I0720 21:30:04.959229 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=യ queue_depth=3376 I0720 21:30:04.959232 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ര queue_depth=3377 I0720 21:30:04.959235 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=റ queue_depth=3378 I0720 21:30:04.959237 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ല queue_depth=3379 I0720 21:30:04.959240 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ള queue_depth=3380 I0720 21:30:04.959244 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഴ queue_depth=3381 I0720 21:30:04.959247 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=വ queue_depth=3382 I0720 21:30:04.959250 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ശ queue_depth=3383 I0720 21:30:04.959253 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഷ queue_depth=3384 I0720 21:30:04.959256 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=സ queue_depth=3385 I0720 21:30:04.959259 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഹ queue_depth=3386 I0720 21:30:04.959262 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഺ queue_depth=3387 I0720 21:30:04.959265 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=഻ queue_depth=3388 I0720 21:30:04.959268 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=഼ queue_depth=3389 I0720 21:30:04.959271 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ഽ queue_depth=3390 I0720 21:30:04.959274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ാ queue_depth=3391 I0720 21:30:04.959276 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ി queue_depth=3392 I0720 21:30:04.959279 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ീ queue_depth=3393 I0720 21:30:04.959283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ു queue_depth=3394 I0720 21:30:04.959286 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൂ queue_depth=3395 I0720 21:30:04.959288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൃ queue_depth=3396 I0720 21:30:04.959291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൄ queue_depth=3397 I0720 21:30:04.959294 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൅ queue_depth=3398 I0720 21:30:04.959297 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=െ queue_depth=3399 I0720 21:30:04.959300 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=േ queue_depth=3400 I0720 21:30:04.959303 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൈ queue_depth=3401 I0720 21:30:04.959306 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൉ queue_depth=3402 I0720 21:30:04.959309 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൊ queue_depth=3403 I0720 21:30:04.959312 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ോ queue_depth=3404 I0720 21:30:04.959315 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൌ queue_depth=3405 I0720 21:30:04.959318 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=് queue_depth=3406 I0720 21:30:04.959321 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൎ queue_depth=3407 I0720 21:30:04.959323 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൏ queue_depth=3408 I0720 21:30:04.959327 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൐ queue_depth=3409 I0720 21:30:04.959330 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൑ queue_depth=3410 I0720 21:30:04.959333 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൒ queue_depth=3411 I0720 21:30:04.959336 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൓ queue_depth=3412 I0720 21:30:04.959338 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൔ queue_depth=3413 I0720 21:30:04.959342 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൕ queue_depth=3414 I0720 21:30:04.959345 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൖ queue_depth=3415 I0720 21:30:04.959348 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൗ queue_depth=3416 I0720 21:30:04.959351 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൘ queue_depth=3417 I0720 21:30:04.959354 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൙ queue_depth=3418 I0720 21:30:04.959357 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൚ queue_depth=3419 I0720 21:30:04.959360 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൛ queue_depth=3420 I0720 21:30:04.959363 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൜ queue_depth=3421 I0720 21:30:04.959366 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൝ queue_depth=3422 I0720 21:30:04.959369 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൞ queue_depth=3423 I0720 21:30:04.959372 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൟ queue_depth=3424 I0720 21:30:04.959375 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൠ queue_depth=3425 I0720 21:30:04.959378 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൡ queue_depth=3426 I0720 21:30:04.959381 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൢ queue_depth=3427 I0720 21:30:04.959384 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൣ queue_depth=3428 I0720 21:30:04.959387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൤ queue_depth=3429 I0720 21:30:04.959389 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൥ queue_depth=3430 I0720 21:30:04.959392 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൦ queue_depth=3431 I0720 21:30:04.959395 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൧ queue_depth=3432 I0720 21:30:04.959399 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൨ queue_depth=3433 I0720 21:30:04.959402 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൩ queue_depth=3434 I0720 21:30:04.959405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൪ queue_depth=3435 I0720 21:30:04.959408 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൫ queue_depth=3436 I0720 21:30:04.959411 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൬ queue_depth=3437 I0720 21:30:04.959414 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൭ queue_depth=3438 I0720 21:30:04.959418 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൮ queue_depth=3439 I0720 21:30:04.959421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൯ queue_depth=3440 I0720 21:30:04.959424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൰ queue_depth=3441 I0720 21:30:04.959427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൱ queue_depth=3442 I0720 21:30:04.959435 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൲ queue_depth=3443 I0720 21:30:04.959438 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൳ queue_depth=3444 I0720 21:30:04.959440 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൴ queue_depth=3445 I0720 21:30:04.959443 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൵ queue_depth=3446 I0720 21:30:04.959446 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൶ queue_depth=3447 I0720 21:30:04.959449 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൷ queue_depth=3448 I0720 21:30:04.959452 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൸ queue_depth=3449 I0720 21:30:04.959455 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=൹ queue_depth=3450 I0720 21:30:04.959458 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൺ queue_depth=3451 I0720 21:30:04.959461 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൻ queue_depth=3452 I0720 21:30:04.959464 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ർ queue_depth=3453 I0720 21:30:04.959466 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൽ queue_depth=3454 I0720 21:30:04.959469 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൾ queue_depth=3455 I0720 21:30:04.959472 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ൿ queue_depth=3456 I0720 21:30:04.959475 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඀ queue_depth=3457 I0720 21:30:04.959478 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඁ queue_depth=3458 I0720 21:30:04.959482 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ං queue_depth=3459 I0720 21:30:04.959484 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඃ queue_depth=3460 I0720 21:30:04.959563 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඄ queue_depth=3461 I0720 21:30:04.959566 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=අ queue_depth=3462 I0720 21:30:04.959569 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ආ queue_depth=3463 I0720 21:30:04.959573 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඇ queue_depth=3464 I0720 21:30:04.959576 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඈ queue_depth=3465 I0720 21:30:04.959579 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඉ queue_depth=3466 I0720 21:30:04.959582 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඊ queue_depth=3467 I0720 21:30:04.959585 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=උ queue_depth=3468 I0720 21:30:04.959588 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඌ queue_depth=3469 I0720 21:30:04.959591 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඍ queue_depth=3470 I0720 21:30:04.959594 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඎ queue_depth=3471 I0720 21:30:04.959596 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඏ queue_depth=3472 I0720 21:30:04.959599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඐ queue_depth=3473 I0720 21:30:04.959602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=එ queue_depth=3474 I0720 21:30:04.959606 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඒ queue_depth=3475 I0720 21:30:04.959610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඓ queue_depth=3476 I0720 21:30:04.959612 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඔ queue_depth=3477 I0720 21:30:04.959615 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඕ queue_depth=3478 I0720 21:30:04.959618 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඖ queue_depth=3479 I0720 21:30:04.959621 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඗ queue_depth=3480 I0720 21:30:04.959624 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඘ queue_depth=3481 I0720 21:30:04.959627 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඙ queue_depth=3482 I0720 21:30:04.959630 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ක queue_depth=3483 I0720 21:30:04.959633 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඛ queue_depth=3484 I0720 21:30:04.959636 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ග queue_depth=3485 I0720 21:30:04.959639 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඝ queue_depth=3486 I0720 21:30:04.959642 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඞ queue_depth=3487 I0720 21:30:04.959645 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඟ queue_depth=3488 I0720 21:30:04.959648 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ච queue_depth=3489 I0720 21:30:04.959651 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඡ queue_depth=3490 I0720 21:30:04.959654 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ජ queue_depth=3491 I0720 21:30:04.959657 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඣ queue_depth=3492 I0720 21:30:04.959659 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඤ queue_depth=3493 I0720 21:30:04.959662 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඥ queue_depth=3494 I0720 21:30:04.959665 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඦ queue_depth=3495 I0720 21:30:04.959668 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ට queue_depth=3496 I0720 21:30:04.959671 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඨ queue_depth=3497 I0720 21:30:04.959674 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඩ queue_depth=3498 I0720 21:30:04.959677 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඪ queue_depth=3499 I0720 21:30:04.959679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ණ queue_depth=3500 I0720 21:30:04.959682 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඬ queue_depth=3501 I0720 21:30:04.959685 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ත queue_depth=3502 I0720 21:30:04.959690 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ථ queue_depth=3503 I0720 21:30:04.959693 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ද queue_depth=3504 I0720 21:30:04.959696 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ධ queue_depth=3505 I0720 21:30:04.959699 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=න queue_depth=3506 I0720 21:30:04.959701 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඲ queue_depth=3507 I0720 21:30:04.959705 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඳ queue_depth=3508 I0720 21:30:04.959709 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ප queue_depth=3509 I0720 21:30:04.959711 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඵ queue_depth=3510 I0720 21:30:04.959714 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=බ queue_depth=3511 I0720 21:30:04.959717 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=භ queue_depth=3512 I0720 21:30:04.959720 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ම queue_depth=3513 I0720 21:30:04.959723 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ඹ queue_depth=3514 I0720 21:30:04.959727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ය queue_depth=3515 I0720 21:30:04.959729 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ර queue_depth=3516 I0720 21:30:04.959732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඼ queue_depth=3517 I0720 21:30:04.959735 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ල queue_depth=3518 I0720 21:30:04.959738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඾ queue_depth=3519 I0720 21:30:04.959741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=඿ queue_depth=3520 I0720 21:30:04.959744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ව queue_depth=3521 I0720 21:30:04.959747 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ශ queue_depth=3522 I0720 21:30:04.959749 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෂ queue_depth=3523 I0720 21:30:04.959753 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ස queue_depth=3524 I0720 21:30:04.959756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=හ queue_depth=3525 I0720 21:30:04.959759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ළ queue_depth=3526 I0720 21:30:04.959762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෆ queue_depth=3527 I0720 21:30:04.959765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෇ queue_depth=3528 I0720 21:30:04.959768 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෈ queue_depth=3529 I0720 21:30:04.959771 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෉ queue_depth=3530 I0720 21:30:04.959774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=් queue_depth=3531 I0720 21:30:04.959777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෋ queue_depth=3532 I0720 21:30:04.959780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෌ queue_depth=3533 I0720 21:30:04.959783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෍ queue_depth=3534 I0720 21:30:04.959786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෎ queue_depth=3535 I0720 21:30:04.959788 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ා queue_depth=3536 I0720 21:30:04.959837 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ැ queue_depth=3537 I0720 21:30:04.959840 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෑ queue_depth=3538 I0720 21:30:04.959843 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ි queue_depth=3539 I0720 21:30:04.959846 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ී queue_depth=3540 I0720 21:30:04.959849 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ු queue_depth=3541 I0720 21:30:04.959853 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෕ queue_depth=3542 I0720 21:30:04.959856 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ූ queue_depth=3543 I0720 21:30:04.959859 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෗ queue_depth=3544 I0720 21:30:04.959862 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෘ queue_depth=3545 I0720 21:30:04.959866 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෙ queue_depth=3546 I0720 21:30:04.959869 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ේ queue_depth=3547 I0720 21:30:04.959871 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෛ queue_depth=3548 I0720 21:30:04.959875 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ො queue_depth=3549 I0720 21:30:04.959878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෝ queue_depth=3550 I0720 21:30:04.959881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෞ queue_depth=3551 I0720 21:30:04.959884 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෟ queue_depth=3552 I0720 21:30:04.959887 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෠ queue_depth=3553 I0720 21:30:04.959890 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෡ queue_depth=3554 I0720 21:30:04.959893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෢ queue_depth=3555 I0720 21:30:04.959896 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෣ queue_depth=3556 I0720 21:30:04.959899 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෤ queue_depth=3557 I0720 21:30:04.959902 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෥ queue_depth=3558 I0720 21:30:04.959905 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෦ queue_depth=3559 I0720 21:30:04.959907 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෧ queue_depth=3560 I0720 21:30:04.959910 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෨ queue_depth=3561 I0720 21:30:04.959913 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෩ queue_depth=3562 I0720 21:30:04.959916 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෪ queue_depth=3563 I0720 21:30:04.959919 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෫ queue_depth=3564 I0720 21:30:04.959922 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෬ queue_depth=3565 I0720 21:30:04.959925 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෭ queue_depth=3566 I0720 21:30:04.959927 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෮ queue_depth=3567 I0720 21:30:04.959930 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෯ queue_depth=3568 I0720 21:30:04.959933 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෰ queue_depth=3569 I0720 21:30:04.959936 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෱ queue_depth=3570 I0720 21:30:04.959939 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෲ queue_depth=3571 I0720 21:30:04.959942 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ෳ queue_depth=3572 I0720 21:30:04.959945 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෴ queue_depth=3573 I0720 21:30:04.959948 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෵ queue_depth=3574 I0720 21:30:04.959952 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෶ queue_depth=3575 I0720 21:30:04.959954 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෷ queue_depth=3576 I0720 21:30:04.959957 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෸ queue_depth=3577 I0720 21:30:04.959960 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෹ queue_depth=3578 I0720 21:30:04.959963 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෺ queue_depth=3579 I0720 21:30:04.959966 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෻ queue_depth=3580 I0720 21:30:04.959969 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෼ queue_depth=3581 I0720 21:30:04.959972 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෽ queue_depth=3582 I0720 21:30:04.959975 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෾ queue_depth=3583 I0720 21:30:04.959977 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=෿ queue_depth=3584 I0720 21:30:04.959981 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=฀ queue_depth=3585 I0720 21:30:04.959984 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ก queue_depth=3586 I0720 21:30:04.959986 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ข queue_depth=3587 I0720 21:30:04.959989 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฃ queue_depth=3588 I0720 21:30:04.959992 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ค queue_depth=3589 I0720 21:30:04.959995 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฅ queue_depth=3590 I0720 21:30:04.959998 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฆ queue_depth=3591 I0720 21:30:04.960001 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ง queue_depth=3592 I0720 21:30:04.960004 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=จ queue_depth=3593 I0720 21:30:04.960007 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฉ queue_depth=3594 I0720 21:30:04.960010 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ช queue_depth=3595 I0720 21:30:04.960012 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ซ queue_depth=3596 I0720 21:30:04.960015 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฌ queue_depth=3597 I0720 21:30:04.960019 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ญ queue_depth=3598 I0720 21:30:04.960021 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฎ queue_depth=3599 I0720 21:30:04.960024 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฏ queue_depth=3600 I0720 21:30:04.960027 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฐ queue_depth=3601 I0720 21:30:04.960030 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฑ queue_depth=3602 I0720 21:30:04.960036 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฒ queue_depth=3603 I0720 21:30:04.960039 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ณ queue_depth=3604 I0720 21:30:04.960042 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ด queue_depth=3605 I0720 21:30:04.960045 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ต queue_depth=3606 I0720 21:30:04.960048 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ถ queue_depth=3607 I0720 21:30:04.960052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ท queue_depth=3608 I0720 21:30:04.960055 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ธ queue_depth=3609 I0720 21:30:04.960058 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=น queue_depth=3610 I0720 21:30:04.960061 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=บ queue_depth=3611 I0720 21:30:04.960064 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ป queue_depth=3612 I0720 21:30:04.960067 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ผ queue_depth=3613 I0720 21:30:04.960070 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฝ queue_depth=3614 I0720 21:30:04.960073 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=พ queue_depth=3615 I0720 21:30:04.960075 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฟ queue_depth=3616 I0720 21:30:04.960078 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ภ queue_depth=3617 I0720 21:30:04.960081 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ม queue_depth=3618 I0720 21:30:04.960084 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ย queue_depth=3619 I0720 21:30:04.960087 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ร queue_depth=3620 I0720 21:30:04.960090 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฤ queue_depth=3621 I0720 21:30:04.960093 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ล queue_depth=3622 I0720 21:30:04.960096 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฦ queue_depth=3623 I0720 21:30:04.960099 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ว queue_depth=3624 I0720 21:30:04.960102 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ศ queue_depth=3625 I0720 21:30:04.960104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ษ queue_depth=3626 I0720 21:30:04.960108 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ส queue_depth=3627 I0720 21:30:04.960111 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ห queue_depth=3628 I0720 21:30:04.960114 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฬ queue_depth=3629 I0720 21:30:04.960117 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=อ queue_depth=3630 I0720 21:30:04.960119 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฮ queue_depth=3631 I0720 21:30:04.960122 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฯ queue_depth=3632 I0720 21:30:04.960125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ะ queue_depth=3633 I0720 21:30:04.960128 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ั queue_depth=3634 I0720 21:30:04.960131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=า queue_depth=3635 I0720 21:30:04.960134 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ำ queue_depth=3636 I0720 21:30:04.960137 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ิ queue_depth=3637 I0720 21:30:04.960140 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ี queue_depth=3638 I0720 21:30:04.960143 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ึ queue_depth=3639 I0720 21:30:04.960146 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ื queue_depth=3640 I0720 21:30:04.960151 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ุ queue_depth=3641 I0720 21:30:04.960157 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ู queue_depth=3642 I0720 21:30:04.960160 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ฺ queue_depth=3643 I0720 21:30:04.960163 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=฻ queue_depth=3644 I0720 21:30:04.960166 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=฼ queue_depth=3645 I0720 21:30:04.960169 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=฽ queue_depth=3646 I0720 21:30:04.960172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=฾ queue_depth=3647 I0720 21:30:04.960175 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=฿ queue_depth=3648 I0720 21:30:04.960178 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=เ queue_depth=3649 I0720 21:30:04.960181 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=แ queue_depth=3650 I0720 21:30:04.960187 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=โ queue_depth=3651 I0720 21:30:04.960190 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ใ queue_depth=3652 I0720 21:30:04.960193 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ไ queue_depth=3653 I0720 21:30:04.960197 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ๅ queue_depth=3654 I0720 21:30:04.960200 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ๆ queue_depth=3655 I0720 21:30:04.960203 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=็ queue_depth=3656 I0720 21:30:04.960206 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=่ queue_depth=3657 I0720 21:30:04.960208 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=้ queue_depth=3658 I0720 21:30:04.960211 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๊ queue_depth=3659 I0720 21:30:04.960214 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๋ queue_depth=3660 I0720 21:30:04.960217 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=์ queue_depth=3661 I0720 21:30:04.960220 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ํ queue_depth=3662 I0720 21:30:04.960223 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๎ queue_depth=3663 I0720 21:30:04.960226 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๏ queue_depth=3664 I0720 21:30:04.960229 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๐ queue_depth=3665 I0720 21:30:04.960232 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๑ queue_depth=3666 I0720 21:30:04.960236 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๒ queue_depth=3667 I0720 21:30:04.960238 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๓ queue_depth=3668 I0720 21:30:04.960242 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๔ queue_depth=3669 I0720 21:30:04.960245 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๕ queue_depth=3670 I0720 21:30:04.960248 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๖ queue_depth=3671 I0720 21:30:04.960251 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๗ queue_depth=3672 I0720 21:30:04.960254 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๘ queue_depth=3673 I0720 21:30:04.960257 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๙ queue_depth=3674 I0720 21:30:04.960260 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๚ queue_depth=3675 I0720 21:30:04.960310 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๛ queue_depth=3676 I0720 21:30:04.960316 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๜ queue_depth=3677 I0720 21:30:04.960319 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๝ queue_depth=3678 I0720 21:30:04.960322 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๞ queue_depth=3679 I0720 21:30:04.960325 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๟ queue_depth=3680 I0720 21:30:04.960328 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๠ queue_depth=3681 I0720 21:30:04.960332 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๡ queue_depth=3682 I0720 21:30:04.960335 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๢ queue_depth=3683 I0720 21:30:04.960338 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๣ queue_depth=3684 I0720 21:30:04.960341 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๤ queue_depth=3685 I0720 21:30:04.960344 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๥ queue_depth=3686 I0720 21:30:04.960347 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๦ queue_depth=3687 I0720 21:30:04.960350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๧ queue_depth=3688 I0720 21:30:04.960352 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๨ queue_depth=3689 I0720 21:30:04.960355 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๩ queue_depth=3690 I0720 21:30:04.960395 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๪ queue_depth=3691 I0720 21:30:04.960398 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๫ queue_depth=3692 I0720 21:30:04.960401 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๬ queue_depth=3693 I0720 21:30:04.960404 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๭ queue_depth=3694 I0720 21:30:04.960407 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๮ queue_depth=3695 I0720 21:30:04.960410 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๯ queue_depth=3696 I0720 21:30:04.960413 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๰ queue_depth=3697 I0720 21:30:04.960416 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๱ queue_depth=3698 I0720 21:30:04.960419 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๲ queue_depth=3699 I0720 21:30:04.960421 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๳ queue_depth=3700 I0720 21:30:04.960424 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๴ queue_depth=3701 I0720 21:30:04.960427 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๵ queue_depth=3702 I0720 21:30:04.960430 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๶ queue_depth=3703 I0720 21:30:04.960433 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๷ queue_depth=3704 I0720 21:30:04.960436 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๸ queue_depth=3705 I0720 21:30:04.960440 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๹ queue_depth=3706 I0720 21:30:04.960442 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๺ queue_depth=3707 I0720 21:30:04.960445 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๻ queue_depth=3708 I0720 21:30:04.960449 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๼ queue_depth=3709 I0720 21:30:04.960451 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๽ queue_depth=3710 I0720 21:30:04.960454 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๾ queue_depth=3711 I0720 21:30:04.960457 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=๿ queue_depth=3712 I0720 21:30:04.960460 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຀ queue_depth=3713 I0720 21:30:04.960463 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ກ queue_depth=3714 I0720 21:30:04.960466 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຂ queue_depth=3715 I0720 21:30:04.960469 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຃ queue_depth=3716 I0720 21:30:04.960472 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຄ queue_depth=3717 I0720 21:30:04.960475 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຅ queue_depth=3718 I0720 21:30:04.960478 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຆ queue_depth=3719 I0720 21:30:04.960481 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ງ queue_depth=3720 I0720 21:30:04.960484 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຈ queue_depth=3721 I0720 21:30:04.960487 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຉ queue_depth=3722 I0720 21:30:04.960490 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຊ queue_depth=3723 I0720 21:30:04.960493 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຋ queue_depth=3724 I0720 21:30:04.960496 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຌ queue_depth=3725 I0720 21:30:04.960499 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຍ queue_depth=3726 I0720 21:30:04.960502 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຎ queue_depth=3727 I0720 21:30:04.960505 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຏ queue_depth=3728 I0720 21:30:04.960508 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຐ queue_depth=3729 I0720 21:30:04.960511 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຑ queue_depth=3730 I0720 21:30:04.960513 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຒ queue_depth=3731 I0720 21:30:04.960516 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຓ queue_depth=3732 I0720 21:30:04.960519 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ດ queue_depth=3733 I0720 21:30:04.960522 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຕ queue_depth=3734 I0720 21:30:04.960525 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຖ queue_depth=3735 I0720 21:30:04.960527 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ທ queue_depth=3736 I0720 21:30:04.960530 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຘ queue_depth=3737 I0720 21:30:04.960533 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ນ queue_depth=3738 I0720 21:30:04.960536 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ບ queue_depth=3739 I0720 21:30:04.960539 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ປ queue_depth=3740 I0720 21:30:04.960542 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຜ queue_depth=3741 I0720 21:30:04.960545 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຝ queue_depth=3742 I0720 21:30:04.960549 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ພ queue_depth=3743 I0720 21:30:04.960552 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຟ queue_depth=3744 I0720 21:30:04.960554 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຠ queue_depth=3745 I0720 21:30:04.960557 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ມ queue_depth=3746 I0720 21:30:04.960571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຢ queue_depth=3747 I0720 21:30:04.960574 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຣ queue_depth=3748 I0720 21:30:04.960578 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຤ queue_depth=3749 I0720 21:30:04.960583 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ລ queue_depth=3750 I0720 21:30:04.960586 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຦ queue_depth=3751 I0720 21:30:04.960589 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ວ queue_depth=3752 I0720 21:30:04.960592 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຨ queue_depth=3753 I0720 21:30:04.960595 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຩ queue_depth=3754 I0720 21:30:04.960598 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ສ queue_depth=3755 I0720 21:30:04.960601 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຫ queue_depth=3756 I0720 21:30:04.960604 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຬ queue_depth=3757 I0720 21:30:04.960607 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ອ queue_depth=3758 I0720 21:30:04.960610 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຮ queue_depth=3759 I0720 21:30:04.960612 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຯ queue_depth=3760 I0720 21:30:04.960615 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ະ queue_depth=3761 I0720 21:30:04.960618 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ັ queue_depth=3762 I0720 21:30:04.960622 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=າ queue_depth=3763 I0720 21:30:04.960625 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຳ queue_depth=3764 I0720 21:30:04.960629 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ິ queue_depth=3765 I0720 21:30:04.960632 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ີ queue_depth=3766 I0720 21:30:04.960636 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຶ queue_depth=3767 I0720 21:30:04.960638 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ື queue_depth=3768 I0720 21:30:04.960641 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຸ queue_depth=3769 I0720 21:30:04.960645 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ູ queue_depth=3770 I0720 21:30:04.960648 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຺ queue_depth=3771 I0720 21:30:04.960651 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ົ queue_depth=3772 I0720 21:30:04.960654 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຼ queue_depth=3773 I0720 21:30:04.960657 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ຽ queue_depth=3774 I0720 21:30:04.960660 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຾ queue_depth=3775 I0720 21:30:04.960663 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=຿ queue_depth=3776 I0720 21:30:04.960666 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ເ queue_depth=3777 I0720 21:30:04.960669 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ແ queue_depth=3778 I0720 21:30:04.960677 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໂ queue_depth=3779 I0720 21:30:04.960681 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໃ queue_depth=3780 I0720 21:30:04.960684 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໄ queue_depth=3781 I0720 21:30:04.960687 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໅ queue_depth=3782 I0720 21:30:04.960690 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໆ queue_depth=3783 I0720 21:30:04.960693 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໇ queue_depth=3784 I0720 21:30:04.960696 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=່ queue_depth=3785 I0720 21:30:04.960699 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=້ queue_depth=3786 I0720 21:30:04.960702 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໊ queue_depth=3787 I0720 21:30:04.960705 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໋ queue_depth=3788 I0720 21:30:04.960708 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໌ queue_depth=3789 I0720 21:30:04.960711 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໍ queue_depth=3790 I0720 21:30:04.960714 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໎ queue_depth=3791 I0720 21:30:04.960717 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໏ queue_depth=3792 I0720 21:30:04.960720 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໐ queue_depth=3793 I0720 21:30:04.960723 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໑ queue_depth=3794 I0720 21:30:04.960726 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໒ queue_depth=3795 I0720 21:30:04.960729 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໓ queue_depth=3796 I0720 21:30:04.960732 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໔ queue_depth=3797 I0720 21:30:04.960735 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໕ queue_depth=3798 I0720 21:30:04.960738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໖ queue_depth=3799 I0720 21:30:04.960740 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໗ queue_depth=3800 I0720 21:30:04.960744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໘ queue_depth=3801 I0720 21:30:04.960746 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໙ queue_depth=3802 I0720 21:30:04.960749 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໚ queue_depth=3803 I0720 21:30:04.960752 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໛ queue_depth=3804 I0720 21:30:04.960756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໜ queue_depth=3805 I0720 21:30:04.960759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໝ queue_depth=3806 I0720 21:30:04.960762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໞ queue_depth=3807 I0720 21:30:04.960765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ໟ queue_depth=3808 I0720 21:30:04.960768 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໠ queue_depth=3809 I0720 21:30:04.960771 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໡ queue_depth=3810 I0720 21:30:04.960774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໢ queue_depth=3811 I0720 21:30:04.960777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໣ queue_depth=3812 I0720 21:30:04.960780 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໤ queue_depth=3813 I0720 21:30:04.960783 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໥ queue_depth=3814 I0720 21:30:04.960786 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໦ queue_depth=3815 I0720 21:30:04.960789 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໧ queue_depth=3816 I0720 21:30:04.960792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໨ queue_depth=3817 I0720 21:30:04.960795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໩ queue_depth=3818 I0720 21:30:04.960798 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໪ queue_depth=3819 I0720 21:30:04.960801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໫ queue_depth=3820 I0720 21:30:04.960804 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໬ queue_depth=3821 I0720 21:30:04.960807 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໭ queue_depth=3822 I0720 21:30:04.960810 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໮ queue_depth=3823 I0720 21:30:04.960813 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໯ queue_depth=3824 I0720 21:30:04.960816 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໰ queue_depth=3825 I0720 21:30:04.960819 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໱ queue_depth=3826 I0720 21:30:04.960822 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໲ queue_depth=3827 I0720 21:30:04.960825 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໳ queue_depth=3828 I0720 21:30:04.960827 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໴ queue_depth=3829 I0720 21:30:04.960830 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໵ queue_depth=3830 I0720 21:30:04.961065 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໶ queue_depth=3831 I0720 21:30:04.961073 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໷ queue_depth=3832 I0720 21:30:04.961077 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໸ queue_depth=3833 I0720 21:30:04.961080 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໹ queue_depth=3834 I0720 21:30:04.961083 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໺ queue_depth=3835 I0720 21:30:04.961090 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໻ queue_depth=3836 I0720 21:30:04.961094 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໼ queue_depth=3837 I0720 21:30:04.961098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໽ queue_depth=3838 I0720 21:30:04.961101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໾ queue_depth=3839 I0720 21:30:04.961666 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=໿ queue_depth=3840 I0720 21:30:04.961673 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ༀ queue_depth=3841 I0720 21:30:04.961676 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༁ queue_depth=3842 I0720 21:30:04.961679 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༂ queue_depth=3843 I0720 21:30:04.961683 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༃ queue_depth=3844 I0720 21:30:04.961689 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༄ queue_depth=3845 I0720 21:30:04.961693 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༅ queue_depth=3846 I0720 21:30:04.961696 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༆ queue_depth=3847 I0720 21:30:04.961699 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༇ queue_depth=3848 I0720 21:30:04.961702 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༈ queue_depth=3849 I0720 21:30:04.961707 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༉ queue_depth=3850 I0720 21:30:04.961710 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༊ queue_depth=3851 I0720 21:30:04.961713 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=་ queue_depth=3852 I0720 21:30:04.961716 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༌ queue_depth=3853 I0720 21:30:04.961719 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=། queue_depth=3854 I0720 21:30:04.961724 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༎ queue_depth=3855 I0720 21:30:04.961727 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༏ queue_depth=3856 I0720 21:30:04.961730 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༐ queue_depth=3857 I0720 21:30:04.961733 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༑ queue_depth=3858 I0720 21:30:04.961738 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༒ queue_depth=3859 I0720 21:30:04.961741 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༓ queue_depth=3860 I0720 21:30:04.961744 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༔ queue_depth=3861 I0720 21:30:04.961747 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༕ queue_depth=3862 I0720 21:30:04.961750 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༖ queue_depth=3863 I0720 21:30:04.961756 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༗ queue_depth=3864 I0720 21:30:04.961759 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༘ queue_depth=3865 I0720 21:30:04.961762 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༙ queue_depth=3866 I0720 21:30:04.961765 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༚ queue_depth=3867 I0720 21:30:04.961769 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༛ queue_depth=3868 I0720 21:30:04.961774 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༜ queue_depth=3869 I0720 21:30:04.961777 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༝ queue_depth=3870 I0720 21:30:04.961781 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༞ queue_depth=3871 I0720 21:30:04.961784 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༟ queue_depth=3872 I0720 21:30:04.961787 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༠ queue_depth=3873 I0720 21:30:04.961792 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༡ queue_depth=3874 I0720 21:30:04.961795 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༢ queue_depth=3875 I0720 21:30:04.961798 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༣ queue_depth=3876 I0720 21:30:04.961801 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༤ queue_depth=3877 I0720 21:30:04.961804 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༥ queue_depth=3878 I0720 21:30:04.961809 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༦ queue_depth=3879 I0720 21:30:04.961812 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༧ queue_depth=3880 I0720 21:30:04.961815 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༨ queue_depth=3881 I0720 21:30:04.961817 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༩ queue_depth=3882 I0720 21:30:04.961822 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༪ queue_depth=3883 I0720 21:30:04.961825 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༫ queue_depth=3884 I0720 21:30:04.961828 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༬ queue_depth=3885 I0720 21:30:04.961831 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༭ queue_depth=3886 I0720 21:30:04.961834 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༮ queue_depth=3887 I0720 21:30:04.961839 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༯ queue_depth=3888 I0720 21:30:04.961842 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༰ queue_depth=3889 I0720 21:30:04.961844 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༱ queue_depth=3890 I0720 21:30:04.961847 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༲ queue_depth=3891 I0720 21:30:04.961850 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༳ queue_depth=3892 I0720 21:30:04.961855 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༴ queue_depth=3893 I0720 21:30:04.961858 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༵ queue_depth=3894 I0720 21:30:04.961861 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༶ queue_depth=3895 I0720 21:30:04.961864 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༷ queue_depth=3896 I0720 21:30:04.961867 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༸ queue_depth=3897 I0720 21:30:04.961873 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༹ queue_depth=3898 I0720 21:30:04.961875 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༺ queue_depth=3899 I0720 21:30:04.961878 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༻ queue_depth=3900 I0720 21:30:04.961881 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༼ queue_depth=3901 I0720 21:30:04.961886 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༽ queue_depth=3902 I0720 21:30:04.961889 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༾ queue_depth=3903 I0720 21:30:04.961893 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=༿ queue_depth=3904 I0720 21:30:04.961896 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཀ queue_depth=3905 I0720 21:30:04.961899 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཁ queue_depth=3906 I0720 21:30:04.961905 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ག queue_depth=3907 I0720 21:30:04.961908 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=གྷ queue_depth=3908 I0720 21:30:04.961910 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ང queue_depth=3909 I0720 21:30:04.961913 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཅ queue_depth=3910 I0720 21:30:04.961916 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཆ queue_depth=3911 I0720 21:30:04.961921 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཇ queue_depth=3912 I0720 21:30:04.961925 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=཈ queue_depth=3913 I0720 21:30:04.961928 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཉ queue_depth=3914 I0720 21:30:04.961930 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཊ queue_depth=3915 I0720 21:30:04.961933 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཋ queue_depth=3916 I0720 21:30:04.961938 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཌ queue_depth=3917 I0720 21:30:04.961947 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཌྷ queue_depth=3918 I0720 21:30:04.961950 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཎ queue_depth=3919 I0720 21:30:04.961953 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཏ queue_depth=3920 I0720 21:30:04.961956 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཐ queue_depth=3921 I0720 21:30:04.961961 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ད queue_depth=3922 I0720 21:30:04.961964 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=དྷ queue_depth=3923 I0720 21:30:04.961967 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ན queue_depth=3924 I0720 21:30:04.961970 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=པ queue_depth=3925 I0720 21:30:04.961974 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཕ queue_depth=3926 I0720 21:30:04.961977 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=བ queue_depth=3927 I0720 21:30:04.961980 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=བྷ queue_depth=3928 I0720 21:30:04.961986 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=མ queue_depth=3929 I0720 21:30:04.961989 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཙ queue_depth=3930 I0720 21:30:04.961994 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཚ queue_depth=3931 I0720 21:30:04.961997 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཛ queue_depth=3932 I0720 21:30:04.962000 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཛྷ queue_depth=3933 I0720 21:30:04.962003 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཝ queue_depth=3934 I0720 21:30:04.962005 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཞ queue_depth=3935 I0720 21:30:04.962012 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཟ queue_depth=3936 I0720 21:30:04.962018 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=འ queue_depth=3937 I0720 21:30:04.962021 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཡ queue_depth=3938 I0720 21:30:04.962024 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ར queue_depth=3939 I0720 21:30:04.962028 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ལ queue_depth=3940 I0720 21:30:04.962033 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཤ queue_depth=3941 I0720 21:30:04.962036 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཥ queue_depth=3942 I0720 21:30:04.962039 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ས queue_depth=3943 I0720 21:30:04.962042 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཧ queue_depth=3944 I0720 21:30:04.962047 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཨ queue_depth=3945 I0720 21:30:04.962050 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཀྵ queue_depth=3946 I0720 21:30:04.962052 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཪ queue_depth=3947 I0720 21:30:04.962055 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཫ queue_depth=3948 I0720 21:30:04.962058 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཬ queue_depth=3949 I0720 21:30:04.962063 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=཭ queue_depth=3950 I0720 21:30:04.962066 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=཮ queue_depth=3951 I0720 21:30:04.962068 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=཯ queue_depth=3952 I0720 21:30:04.962071 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=཰ queue_depth=3953 I0720 21:30:04.962074 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཱ queue_depth=3954 I0720 21:30:04.962079 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ི queue_depth=3955 I0720 21:30:04.962082 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཱི queue_depth=3956 I0720 21:30:04.962085 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ུ queue_depth=3957 I0720 21:30:04.962088 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཱུ queue_depth=3958 I0720 21:30:04.962090 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྲྀ queue_depth=3959 I0720 21:30:04.962095 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཷ queue_depth=3960 I0720 21:30:04.962098 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ླྀ queue_depth=3961 I0720 21:30:04.962101 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཹ queue_depth=3962 I0720 21:30:04.962104 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ེ queue_depth=3963 I0720 21:30:04.962106 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཻ queue_depth=3964 I0720 21:30:04.962111 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ོ queue_depth=3965 I0720 21:30:04.962114 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཽ queue_depth=3966 I0720 21:30:04.962117 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཾ queue_depth=3967 I0720 21:30:04.962120 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཿ queue_depth=3968 I0720 21:30:04.962125 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྀ queue_depth=3969 I0720 21:30:04.962131 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ཱྀ queue_depth=3970 I0720 21:30:04.962139 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྂ queue_depth=3971 I0720 21:30:04.962142 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྃ queue_depth=3972 I0720 21:30:04.962145 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྄ queue_depth=3973 I0720 21:30:04.962150 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྅ queue_depth=3974 I0720 21:30:04.962153 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྆ queue_depth=3975 I0720 21:30:04.962156 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྇ queue_depth=3976 I0720 21:30:04.962159 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྈ queue_depth=3977 I0720 21:30:04.962162 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྉ queue_depth=3978 I0720 21:30:04.962167 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྊ queue_depth=3979 I0720 21:30:04.962169 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྋ queue_depth=3980 I0720 21:30:04.962172 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྌ queue_depth=3981 I0720 21:30:04.962175 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྍ queue_depth=3982 I0720 21:30:04.962178 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྎ queue_depth=3983 I0720 21:30:04.962183 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྏ queue_depth=3984 I0720 21:30:04.962186 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྐ queue_depth=3985 I0720 21:30:04.962188 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྑ queue_depth=3986 I0720 21:30:04.962191 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྒ queue_depth=3987 I0720 21:30:04.962196 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྒྷ queue_depth=3988 I0720 21:30:04.962199 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྔ queue_depth=3989 I0720 21:30:04.962202 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྕ queue_depth=3990 I0720 21:30:04.962204 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྖ queue_depth=3991 I0720 21:30:04.962207 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྗ queue_depth=3992 I0720 21:30:04.962212 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྘ queue_depth=3993 I0720 21:30:04.962215 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྙ queue_depth=3994 I0720 21:30:04.962221 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྚ queue_depth=3995 I0720 21:30:04.962224 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྛ queue_depth=3996 I0720 21:30:04.962227 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྜ queue_depth=3997 I0720 21:30:04.962232 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྜྷ queue_depth=3998 I0720 21:30:04.962239 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྞ queue_depth=3999 I0720 21:30:04.962241 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྟ queue_depth=4000 I0720 21:30:04.962244 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྠ queue_depth=4001 I0720 21:30:04.962247 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྡ queue_depth=4002 I0720 21:30:04.962255 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྡྷ queue_depth=4003 I0720 21:30:04.962258 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྣ queue_depth=4004 I0720 21:30:04.962261 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྤ queue_depth=4005 I0720 21:30:04.962264 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྥ queue_depth=4006 I0720 21:30:04.962267 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྦ queue_depth=4007 I0720 21:30:04.962271 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྦྷ queue_depth=4008 I0720 21:30:04.962274 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྨ queue_depth=4009 I0720 21:30:04.962277 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྩ queue_depth=4010 I0720 21:30:04.962283 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྪ queue_depth=4011 I0720 21:30:04.962288 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྫ queue_depth=4012 I0720 21:30:04.962291 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྫྷ queue_depth=4013 I0720 21:30:04.962294 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྭ queue_depth=4014 I0720 21:30:04.962297 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྮ queue_depth=4015 I0720 21:30:04.962300 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྯ queue_depth=4016 I0720 21:30:04.962347 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྰ queue_depth=4017 I0720 21:30:04.962350 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྱ queue_depth=4018 I0720 21:30:04.962353 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྲ queue_depth=4019 I0720 21:30:04.962356 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ླ queue_depth=4020 I0720 21:30:04.962359 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྴ queue_depth=4021 I0720 21:30:04.962371 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྵ queue_depth=4022 I0720 21:30:04.962374 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྶ queue_depth=4023 I0720 21:30:04.962377 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྷ queue_depth=4024 I0720 21:30:04.962379 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྸ queue_depth=4025 I0720 21:30:04.962382 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྐྵ queue_depth=4026 I0720 21:30:04.962387 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྺ queue_depth=4027 I0720 21:30:04.962390 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྻ queue_depth=4028 I0720 21:30:04.962393 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=ྼ queue_depth=4029 I0720 21:30:04.962395 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྽ queue_depth=4030 I0720 21:30:04.962400 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྾ queue_depth=4031 I0720 21:30:04.962403 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=྿ queue_depth=4032 I0720 21:30:04.962405 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿀ queue_depth=4033 I0720 21:30:04.962408 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿁ queue_depth=4034 I0720 21:30:04.962411 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿂ queue_depth=4035 I0720 21:30:04.962423 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿃ queue_depth=4036 I0720 21:30:04.962426 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿄ queue_depth=4037 I0720 21:30:04.962429 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿅ queue_depth=4038 I0720 21:30:04.962432 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿆ queue_depth=4039 I0720 21:30:04.962435 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿇ queue_depth=4040 I0720 21:30:04.962440 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿈ queue_depth=4041 I0720 21:30:04.962443 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿉ queue_depth=4042 I0720 21:30:04.962445 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿊ queue_depth=4043 I0720 21:30:04.962448 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿋ queue_depth=4044 I0720 21:30:04.962451 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿌ queue_depth=4045 I0720 21:30:04.962457 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿍ queue_depth=4046 I0720 21:30:04.962460 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿎ queue_depth=4047 I0720 21:30:04.962463 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿏ queue_depth=4048 I0720 21:30:04.962465 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿐ queue_depth=4049 I0720 21:30:04.962470 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿑ queue_depth=4050 I0720 21:30:04.962473 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿒ queue_depth=4051 I0720 21:30:04.962476 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿓ queue_depth=4052 I0720 21:30:04.962479 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿔ queue_depth=4053 I0720 21:30:04.962481 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿕ queue_depth=4054 I0720 21:30:04.962486 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿖ queue_depth=4055 I0720 21:30:04.962489 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿗ queue_depth=4056 I0720 21:30:04.962492 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿘ queue_depth=4057 I0720 21:30:04.962495 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿙ queue_depth=4058 I0720 21:30:04.962498 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿚ queue_depth=4059 I0720 21:30:04.962503 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿛ queue_depth=4060 I0720 21:30:04.962506 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿜ queue_depth=4061 I0720 21:30:04.962509 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿝ queue_depth=4062 I0720 21:30:04.962512 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿞ queue_depth=4063 I0720 21:30:04.962514 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿟ queue_depth=4064 I0720 21:30:04.962520 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿠ queue_depth=4065 I0720 21:30:04.962523 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿡ queue_depth=4066 I0720 21:30:04.962526 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿢ queue_depth=4067 I0720 21:30:04.962529 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿣ queue_depth=4068 I0720 21:30:04.962541 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿤ queue_depth=4069 I0720 21:30:04.962544 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿥ queue_depth=4070 I0720 21:30:04.962547 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿦ queue_depth=4071 I0720 21:30:04.962551 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿧ queue_depth=4072 I0720 21:30:04.962554 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿨ queue_depth=4073 I0720 21:30:04.962559 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿩ queue_depth=4074 I0720 21:30:04.962562 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿪ queue_depth=4075 I0720 21:30:04.962565 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿫ queue_depth=4076 I0720 21:30:04.962571 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿬ queue_depth=4077 I0720 21:30:04.962574 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿭ queue_depth=4078 I0720 21:30:04.962579 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿮ queue_depth=4079 I0720 21:30:04.962582 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿯ queue_depth=4080 I0720 21:30:04.962585 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿰ queue_depth=4081 I0720 21:30:04.962588 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿱ queue_depth=4082 I0720 21:30:04.962591 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿲ queue_depth=4083 I0720 21:30:04.962597 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿳ queue_depth=4084 I0720 21:30:04.962599 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿴ queue_depth=4085 I0720 21:30:04.962602 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿵ queue_depth=4086 I0720 21:30:04.962605 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿶ queue_depth=4087 I0720 21:30:04.962608 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿷ queue_depth=4088 I0720 21:30:04.962613 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿸ queue_depth=4089 I0720 21:30:04.962617 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿹ queue_depth=4090 I0720 21:30:04.962620 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿺ queue_depth=4091 I0720 21:30:04.962623 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿻ queue_depth=4092 I0720 21:30:04.962628 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿼ queue_depth=4093 I0720 21:30:04.962636 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿽ queue_depth=4094 I0720 21:30:04.962639 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿾ queue_depth=4095 I0720 21:30:04.962642 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=࿿ queue_depth=4096 W0720 21:30:04.962645 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=က capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962652 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ခ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962655 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဂ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962658 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဃ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962661 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=င capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962668 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=စ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962674 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဆ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962677 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဇ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962680 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဈ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962683 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဉ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962685 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ည capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962691 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဋ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962694 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဌ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962697 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဍ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962699 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဎ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962702 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဏ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962708 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=တ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962711 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ထ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962714 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဒ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962716 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဓ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962719 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=န capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962722 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ပ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962727 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဖ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962730 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဗ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962733 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဘ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962736 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=မ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962739 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ယ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962744 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ရ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962747 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=လ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962750 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဝ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962753 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=သ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962756 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဟ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962761 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဠ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962763 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=အ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962766 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဢ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962770 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဣ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962773 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဤ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962779 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဥ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962785 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဦ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962791 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဧ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962794 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဨ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962797 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဩ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962802 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ဪ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962894 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ါ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962898 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ာ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962900 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ိ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962903 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ီ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962908 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ု capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962911 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ူ capacity=4096 (candidate dropped; read-path heal still covers it) W0720 21:30:04.962913 s3api_versioning_reconciler.go:50 [versioning-heal] event=queue_full bucket=b key=ေ capacity=4096 (candidate dropped; read-path heal still covers it) --- PASS: TestVersionsHealQueue_CapacityCap (0.01s) === RUN TestVersionsHealQueue_PopReadyOnlyDueItems I0720 21:30:04.962961 s3api_versioning_reconciler.go:46 [versioning-heal] event=enqueue bucket=b key=due queue_depth=1 --- PASS: TestVersionsHealQueue_PopReadyOnlyDueItems (0.00s) === RUN TestVersionsHealQueue_RequeueWithBackoff --- PASS: TestVersionsHealQueue_RequeueWithBackoff (0.00s) === RUN TestVersionsHealQueue_GiveUpAfterMaxAttempts W0720 21:30:04.963007 s3api_versioning_reconciler.go:50 [versioning-heal] event=gave_up bucket=b key=obj attempts=6 (read-path heal will still recover) --- PASS: TestVersionsHealQueue_GiveUpAfterMaxAttempts (0.00s) === RUN TestRetryFilerOp_SucceedsBeforeExhaustion --- PASS: TestRetryFilerOp_SucceedsBeforeExhaustion (0.30s) === RUN TestRetryFilerOp_PropagatesAfterExhaustion --- PASS: TestRetryFilerOp_PropagatesAfterExhaustion (3.11s) === RUN TestRetryFilerOp_ContextCancelInterruptsBackoff --- PASS: TestRetryFilerOp_ContextCancelInterruptsBackoff (0.03s) === RUN TestRetryFilerOp_TerminalErrorsShortCircuit === RUN TestRetryFilerOp_TerminalErrorsShortCircuit/filer_pb.ErrNotFound === RUN TestRetryFilerOp_TerminalErrorsShortCircuit/wrapped_filer_pb.ErrNotFound === RUN TestRetryFilerOp_TerminalErrorsShortCircuit/grpc_NotFound_status === RUN TestRetryFilerOp_TerminalErrorsShortCircuit/context_canceled === RUN TestRetryFilerOp_TerminalErrorsShortCircuit/context_deadline_exceeded --- PASS: TestRetryFilerOp_TerminalErrorsShortCircuit (0.00s) --- PASS: TestRetryFilerOp_TerminalErrorsShortCircuit/filer_pb.ErrNotFound (0.00s) --- PASS: TestRetryFilerOp_TerminalErrorsShortCircuit/wrapped_filer_pb.ErrNotFound (0.00s) --- PASS: TestRetryFilerOp_TerminalErrorsShortCircuit/grpc_NotFound_status (0.00s) --- PASS: TestRetryFilerOp_TerminalErrorsShortCircuit/context_canceled (0.00s) --- PASS: TestRetryFilerOp_TerminalErrorsShortCircuit/context_deadline_exceeded (0.00s) === RUN TestMultipartSSES3RealisticEndToEnd --- PASS: TestMultipartSSES3RealisticEndToEnd (0.09s) === RUN TestBuildMultipartSSES3Reader_LazyChunkFetch --- PASS: TestBuildMultipartSSES3Reader_LazyChunkFetch (0.00s) === RUN TestTrackAuditFallbackForDirectWriteHeader --- PASS: TestTrackAuditFallbackForDirectWriteHeader (0.00s) === RUN TestTrackAuditSkipsFallbackWhenHandlerEmits --- PASS: TestTrackAuditSkipsFallbackWhenHandlerEmits (0.00s) === RUN TestComputePackedPolicySize === RUN TestComputePackedPolicySize/empty_->_nil === RUN TestComputePackedPolicySize/tiny_policy_->_0% === RUN TestComputePackedPolicySize/half_budget_->_50% === RUN TestComputePackedPolicySize/full_budget_->_100% === RUN TestComputePackedPolicySize/oversized_->_capped_at_100 --- PASS: TestComputePackedPolicySize (0.00s) --- PASS: TestComputePackedPolicySize/empty_->_nil (0.00s) --- PASS: TestComputePackedPolicySize/tiny_policy_->_0% (0.00s) --- PASS: TestComputePackedPolicySize/half_budget_->_50% (0.00s) --- PASS: TestComputePackedPolicySize/full_budget_->_100% (0.00s) --- PASS: TestComputePackedPolicySize/oversized_->_capped_at_100 (0.00s) === RUN TestSTSAssumeRolePostBody === RUN TestSTSAssumeRolePostBody/ActionInQuery === RUN TestSTSAssumeRolePostBody/ActionInBody === RUN TestSTSAssumeRolePostBody/ActionInBodyWithSigV4Style --- PASS: TestSTSAssumeRolePostBody (0.00s) --- PASS: TestSTSAssumeRolePostBody/ActionInQuery (0.00s) --- PASS: TestSTSAssumeRolePostBody/ActionInBody (0.00s) --- PASS: TestSTSAssumeRolePostBody/ActionInBodyWithSigV4Style (0.00s) === RUN TestValidateRoleSessionName === RUN TestValidateRoleSessionName/empty_rejected === RUN TestValidateRoleSessionName/single_char_rejected_(below_min_len_2) === RUN TestValidateRoleSessionName/min_length_2_accepted === RUN TestValidateRoleSessionName/plain_ascii_accepted === RUN TestValidateRoleSessionName/all_special_chars_allowed === RUN TestValidateRoleSessionName/email-style_accepted === RUN TestValidateRoleSessionName/max_length_64_accepted === RUN TestValidateRoleSessionName/max_length_64_valid_charset_accepted === RUN TestValidateRoleSessionName/length_65_rejected === RUN TestValidateRoleSessionName/space_rejected === RUN TestValidateRoleSessionName/slash_rejected === RUN TestValidateRoleSessionName/colon_rejected === RUN TestValidateRoleSessionName/unicode_rejected --- PASS: TestValidateRoleSessionName (0.00s) --- PASS: TestValidateRoleSessionName/empty_rejected (0.00s) --- PASS: TestValidateRoleSessionName/single_char_rejected_(below_min_len_2) (0.00s) --- PASS: TestValidateRoleSessionName/min_length_2_accepted (0.00s) --- PASS: TestValidateRoleSessionName/plain_ascii_accepted (0.00s) --- PASS: TestValidateRoleSessionName/all_special_chars_allowed (0.00s) --- PASS: TestValidateRoleSessionName/email-style_accepted (0.00s) --- PASS: TestValidateRoleSessionName/max_length_64_accepted (0.00s) --- PASS: TestValidateRoleSessionName/max_length_64_valid_charset_accepted (0.00s) --- PASS: TestValidateRoleSessionName/length_65_rejected (0.00s) --- PASS: TestValidateRoleSessionName/space_rejected (0.00s) --- PASS: TestValidateRoleSessionName/slash_rejected (0.00s) --- PASS: TestValidateRoleSessionName/colon_rejected (0.00s) --- PASS: TestValidateRoleSessionName/unicode_rejected (0.00s) === RUN TestParseTagsHeader === RUN TestParseTagsHeader/simple_tags === RUN TestParseTagsHeader/URL_encoded_timestamp_-_issue_#7040_scenario === RUN TestParseTagsHeader/URL_encoded_key_and_value === RUN TestParseTagsHeader/empty_value === RUN TestParseTagsHeader/special_characters_encoded === RUN TestParseTagsHeader/invalid_URL_encoding === RUN TestParseTagsHeader/plus_signs_and_equals_in_values === RUN TestParseTagsHeader/unescaped_equals_in_value --- PASS: TestParseTagsHeader (0.00s) --- PASS: TestParseTagsHeader/simple_tags (0.00s) --- PASS: TestParseTagsHeader/URL_encoded_timestamp_-_issue_#7040_scenario (0.00s) --- PASS: TestParseTagsHeader/URL_encoded_key_and_value (0.00s) --- PASS: TestParseTagsHeader/empty_value (0.00s) --- PASS: TestParseTagsHeader/special_characters_encoded (0.00s) --- PASS: TestParseTagsHeader/invalid_URL_encoding (0.00s) --- PASS: TestParseTagsHeader/plus_signs_and_equals_in_values (0.00s) --- PASS: TestParseTagsHeader/unescaped_equals_in_value (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api 6.297s === RUN TestValidateConfiguration === RUN TestValidateConfiguration/nil_config === RUN TestValidateConfiguration/empty_rules === RUN TestValidateConfiguration/valid_single_rule === RUN TestValidateConfiguration/too_many_rules === RUN TestValidateConfiguration/invalid_method === RUN TestValidateConfiguration/empty_origins === RUN TestValidateConfiguration/invalid_origin_with_multiple_wildcards === RUN TestValidateConfiguration/negative_MaxAgeSeconds --- PASS: TestValidateConfiguration (0.00s) --- PASS: TestValidateConfiguration/nil_config (0.00s) --- PASS: TestValidateConfiguration/empty_rules (0.00s) --- PASS: TestValidateConfiguration/valid_single_rule (0.00s) --- PASS: TestValidateConfiguration/too_many_rules (0.00s) --- PASS: TestValidateConfiguration/invalid_method (0.00s) --- PASS: TestValidateConfiguration/empty_origins (0.00s) --- PASS: TestValidateConfiguration/invalid_origin_with_multiple_wildcards (0.00s) --- PASS: TestValidateConfiguration/negative_MaxAgeSeconds (0.00s) === RUN TestValidateOrigin === RUN TestValidateOrigin/empty_origin === RUN TestValidateOrigin/valid_origin === RUN TestValidateOrigin/wildcard_origin === RUN TestValidateOrigin/valid_wildcard_origin === RUN TestValidateOrigin/https_wildcard_origin === RUN TestValidateOrigin/invalid_wildcard_origin === RUN TestValidateOrigin/multiple_wildcards --- PASS: TestValidateOrigin (0.00s) --- PASS: TestValidateOrigin/empty_origin (0.00s) --- PASS: TestValidateOrigin/valid_origin (0.00s) --- PASS: TestValidateOrigin/wildcard_origin (0.00s) --- PASS: TestValidateOrigin/valid_wildcard_origin (0.00s) --- PASS: TestValidateOrigin/https_wildcard_origin (0.00s) --- PASS: TestValidateOrigin/invalid_wildcard_origin (0.00s) --- PASS: TestValidateOrigin/multiple_wildcards (0.00s) === RUN TestParseRequest === RUN TestParseRequest/simple_GET_request === RUN TestParseRequest/OPTIONS_preflight_request === RUN TestParseRequest/request_without_origin --- PASS: TestParseRequest (0.00s) --- PASS: TestParseRequest/simple_GET_request (0.00s) --- PASS: TestParseRequest/OPTIONS_preflight_request (0.00s) --- PASS: TestParseRequest/request_without_origin (0.00s) === RUN TestMatchesOrigin === RUN TestMatchesOrigin/wildcard_match === RUN TestMatchesOrigin/exact_match === RUN TestMatchesOrigin/no_match === RUN TestMatchesOrigin/wildcard_subdomain_match === RUN TestMatchesOrigin/wildcard_subdomain_no_match === RUN TestMatchesOrigin/multiple_origins_with_match === RUN TestMatchesOrigin/https_exact_match === RUN TestMatchesOrigin/https_no_match === RUN TestMatchesOrigin/https_wildcard_subdomain_match === RUN TestMatchesOrigin/https_wildcard_subdomain_no_match_-_base_domain === RUN TestMatchesOrigin/https_wildcard_subdomain_no_match_-_different_domain === RUN TestMatchesOrigin/protocol_mismatch_-_http_pattern_https_origin === RUN TestMatchesOrigin/protocol_mismatch_-_https_pattern_http_origin --- PASS: TestMatchesOrigin (0.00s) --- PASS: TestMatchesOrigin/wildcard_match (0.00s) --- PASS: TestMatchesOrigin/exact_match (0.00s) --- PASS: TestMatchesOrigin/no_match (0.00s) --- PASS: TestMatchesOrigin/wildcard_subdomain_match (0.00s) --- PASS: TestMatchesOrigin/wildcard_subdomain_no_match (0.00s) --- PASS: TestMatchesOrigin/multiple_origins_with_match (0.00s) --- PASS: TestMatchesOrigin/https_exact_match (0.00s) --- PASS: TestMatchesOrigin/https_no_match (0.00s) --- PASS: TestMatchesOrigin/https_wildcard_subdomain_match (0.00s) --- PASS: TestMatchesOrigin/https_wildcard_subdomain_no_match_-_base_domain (0.00s) --- PASS: TestMatchesOrigin/https_wildcard_subdomain_no_match_-_different_domain (0.00s) --- PASS: TestMatchesOrigin/protocol_mismatch_-_http_pattern_https_origin (0.00s) --- PASS: TestMatchesOrigin/protocol_mismatch_-_https_pattern_http_origin (0.00s) === RUN TestMatchesHeader === RUN TestMatchesHeader/empty_allowed_headers === RUN TestMatchesHeader/wildcard_match === RUN TestMatchesHeader/exact_match === RUN TestMatchesHeader/case_insensitive_match === RUN TestMatchesHeader/no_match === RUN TestMatchesHeader/wildcard_prefix_match --- PASS: TestMatchesHeader (0.00s) --- PASS: TestMatchesHeader/empty_allowed_headers (0.00s) --- PASS: TestMatchesHeader/wildcard_match (0.00s) --- PASS: TestMatchesHeader/exact_match (0.00s) --- PASS: TestMatchesHeader/case_insensitive_match (0.00s) --- PASS: TestMatchesHeader/no_match (0.00s) --- PASS: TestMatchesHeader/wildcard_prefix_match (0.00s) === RUN TestEvaluateRequest === RUN TestEvaluateRequest/matching_first_rule === RUN TestEvaluateRequest/matching_second_rule === RUN TestEvaluateRequest/no_matching_rule === RUN TestEvaluateRequest/preflight_request === RUN TestEvaluateRequest/preflight_request_with_forbidden_header === RUN TestEvaluateRequest/request_without_origin --- PASS: TestEvaluateRequest (0.00s) --- PASS: TestEvaluateRequest/matching_first_rule (0.00s) --- PASS: TestEvaluateRequest/matching_second_rule (0.00s) --- PASS: TestEvaluateRequest/no_matching_rule (0.00s) --- PASS: TestEvaluateRequest/preflight_request (0.00s) --- PASS: TestEvaluateRequest/preflight_request_with_forbidden_header (0.00s) --- PASS: TestEvaluateRequest/request_without_origin (0.00s) === RUN TestApplyHeaders === RUN TestApplyHeaders/nil_response === RUN TestApplyHeaders/complete_response === RUN TestApplyHeaders/with_credentials --- PASS: TestApplyHeaders (0.00s) --- PASS: TestApplyHeaders/nil_response (0.00s) --- PASS: TestApplyHeaders/complete_response (0.00s) --- PASS: TestApplyHeaders/with_credentials (0.00s) === RUN TestMiddlewareNonExistentBucket === RUN TestMiddlewareNonExistentBucket/Preflight_request_to_non-existent_bucket_with_global_CORS_config === RUN TestMiddlewareNonExistentBucket/Actual_request_to_non-existent_bucket_with_global_CORS_config === RUN TestMiddlewareNonExistentBucket/Preflight_to_non-existent_bucket_with_specific_origin === RUN TestMiddlewareNonExistentBucket/Preflight_to_non-existent_bucket_with_non-matching_origin === RUN TestMiddlewareNonExistentBucket/Preflight_to_non-existent_bucket_without_CORS_config === RUN TestMiddlewareNonExistentBucket/Bucket_listing_request_to_non-existent_bucket_with_CORS --- PASS: TestMiddlewareNonExistentBucket (0.00s) --- PASS: TestMiddlewareNonExistentBucket/Preflight_request_to_non-existent_bucket_with_global_CORS_config (0.00s) --- PASS: TestMiddlewareNonExistentBucket/Actual_request_to_non-existent_bucket_with_global_CORS_config (0.00s) --- PASS: TestMiddlewareNonExistentBucket/Preflight_to_non-existent_bucket_with_specific_origin (0.00s) --- PASS: TestMiddlewareNonExistentBucket/Preflight_to_non-existent_bucket_with_non-matching_origin (0.00s) --- PASS: TestMiddlewareNonExistentBucket/Preflight_to_non-existent_bucket_without_CORS_config (0.00s) --- PASS: TestMiddlewareNonExistentBucket/Bucket_listing_request_to_non-existent_bucket_with_CORS (0.00s) === RUN TestMiddlewareConsistentBehavior === RUN TestMiddlewareConsistentBehavior/existing_bucket === RUN TestMiddlewareConsistentBehavior/non-existent_bucket --- PASS: TestMiddlewareConsistentBehavior (0.00s) --- PASS: TestMiddlewareConsistentBehavior/existing_bucket (0.00s) --- PASS: TestMiddlewareConsistentBehavior/non-existent_bucket (0.00s) === RUN TestMiddlewareFallbackConfig === RUN TestMiddlewareFallbackConfig/No_bucket_config,_fallback_to_global_config_with_wildcard === RUN TestMiddlewareFallbackConfig/No_bucket_config,_fallback_to_global_config_with_specific_origin === RUN TestMiddlewareFallbackConfig/No_bucket_config,_fallback_rejects_non-matching_origin === RUN TestMiddlewareFallbackConfig/Bucket_config_takes_precedence_over_fallback === RUN TestMiddlewareFallbackConfig/Bucket_config_rejects,_even_though_fallback_would_allow === RUN TestMiddlewareFallbackConfig/No_config_at_all,_no_CORS_headers === RUN TestMiddlewareFallbackConfig/OPTIONS_preflight_with_fallback_config === RUN TestMiddlewareFallbackConfig/OPTIONS_preflight_without_any_config_should_fail --- PASS: TestMiddlewareFallbackConfig (0.00s) --- PASS: TestMiddlewareFallbackConfig/No_bucket_config,_fallback_to_global_config_with_wildcard (0.00s) --- PASS: TestMiddlewareFallbackConfig/No_bucket_config,_fallback_to_global_config_with_specific_origin (0.00s) --- PASS: TestMiddlewareFallbackConfig/No_bucket_config,_fallback_rejects_non-matching_origin (0.00s) --- PASS: TestMiddlewareFallbackConfig/Bucket_config_takes_precedence_over_fallback (0.00s) --- PASS: TestMiddlewareFallbackConfig/Bucket_config_rejects,_even_though_fallback_would_allow (0.00s) --- PASS: TestMiddlewareFallbackConfig/No_config_at_all,_no_CORS_headers (0.00s) --- PASS: TestMiddlewareFallbackConfig/OPTIONS_preflight_with_fallback_config (0.00s) --- PASS: TestMiddlewareFallbackConfig/OPTIONS_preflight_without_any_config_should_fail (0.00s) === RUN TestMiddlewareFallbackConfigWithMultipleOrigins === RUN TestMiddlewareFallbackConfigWithMultipleOrigins/First_allowed_origin_should_match === RUN TestMiddlewareFallbackConfigWithMultipleOrigins/Second_allowed_origin_should_match === RUN TestMiddlewareFallbackConfigWithMultipleOrigins/Non-allowed_origin_should_not_match --- PASS: TestMiddlewareFallbackConfigWithMultipleOrigins (0.00s) --- PASS: TestMiddlewareFallbackConfigWithMultipleOrigins/First_allowed_origin_should_match (0.00s) --- PASS: TestMiddlewareFallbackConfigWithMultipleOrigins/Second_allowed_origin_should_match (0.00s) --- PASS: TestMiddlewareFallbackConfigWithMultipleOrigins/Non-allowed_origin_should_not_match (0.00s) === RUN TestMiddlewareFallbackWithError === RUN TestMiddlewareFallbackWithError/ErrAccessDenied_should_not_trigger_fallback === RUN TestMiddlewareFallbackWithError/ErrInternalError_should_not_trigger_fallback === RUN TestMiddlewareFallbackWithError/ErrNoSuchBucket_should_trigger_fallback === RUN TestMiddlewareFallbackWithError/ErrNoSuchCORSConfiguration_should_trigger_fallback --- PASS: TestMiddlewareFallbackWithError (0.00s) --- PASS: TestMiddlewareFallbackWithError/ErrAccessDenied_should_not_trigger_fallback (0.00s) --- PASS: TestMiddlewareFallbackWithError/ErrInternalError_should_not_trigger_fallback (0.00s) --- PASS: TestMiddlewareFallbackWithError/ErrNoSuchBucket_should_trigger_fallback (0.00s) --- PASS: TestMiddlewareFallbackWithError/ErrNoSuchCORSConfiguration_should_trigger_fallback (0.00s) === RUN TestMiddlewareVaryHeader === RUN TestMiddlewareVaryHeader/Specific_allowed_origin === RUN TestMiddlewareVaryHeader/Wildcard_allowed_origin === RUN TestMiddlewareVaryHeader/No_CORS_configuration --- PASS: TestMiddlewareVaryHeader (0.00s) --- PASS: TestMiddlewareVaryHeader/Specific_allowed_origin (0.00s) --- PASS: TestMiddlewareVaryHeader/Wildcard_allowed_origin (0.00s) --- PASS: TestMiddlewareVaryHeader/No_CORS_configuration (0.00s) === RUN TestHandleOptionsRequestVaryHeader --- PASS: TestHandleOptionsRequestVaryHeader (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/cors 0.035s === RUN TestHandleOAuthTokens_Success --- PASS: TestHandleOAuthTokens_Success (0.00s) === RUN TestHandleOAuthTokens_InvalidCredentials --- PASS: TestHandleOAuthTokens_InvalidCredentials (0.00s) === RUN TestHandleOAuthTokens_UnsupportedGrantType --- PASS: TestHandleOAuthTokens_UnsupportedGrantType (0.00s) === RUN TestBearerTokenRoundTrip --- PASS: TestBearerTokenRoundTrip (0.00s) === RUN TestBearerTokenInvalid --- PASS: TestBearerTokenInvalid (0.00s) === RUN TestBearerTokenNone --- PASS: TestBearerTokenNone (0.00s) === RUN TestParseCommitUpdatesSeparatesStatistics --- PASS: TestParseCommitUpdatesSeparatesStatistics (0.00s) === RUN TestParseCommitUpdatesRejectsIncompleteSetStatistics --- PASS: TestParseCommitUpdatesRejectsIncompleteSetStatistics (0.00s) === RUN TestApplyStatisticsUpdatesUpsertAndRemove --- PASS: TestApplyStatisticsUpdatesUpsertAndRemove (0.00s) === RUN TestIsS3TablesConflict --- PASS: TestIsS3TablesConflict (0.00s) === RUN TestValidateCreateTableRequestRequiresName --- PASS: TestValidateCreateTableRequestRequiresName (0.00s) === RUN TestValidateCreateTableRequestAcceptsWithName --- PASS: TestValidateCreateTableRequestAcceptsWithName (0.00s) === RUN TestIsStageCreateEnabledDefaultsToTrue --- PASS: TestIsStageCreateEnabledDefaultsToTrue (0.00s) === RUN TestIsStageCreateEnabledFalseValues --- PASS: TestIsStageCreateEnabledFalseValues (0.00s) === RUN TestCreateViewMissingNamespaceReturns404 E0720 21:30:02.437061 handler.go:199 S3Tables: error handling GetView: filer: no entry is found in filer store E0720 21:30:02.437564 handler.go:199 S3Tables: error handling CreateView: filer: no entry is found in filer store --- PASS: TestCreateViewMissingNamespaceReturns404 (0.00s) === RUN TestCreateViewTagsEntryAsView E0720 21:30:02.437723 handler.go:199 S3Tables: error handling GetView: filer: no entry is found in filer store --- PASS: TestCreateViewTagsEntryAsView (0.00s) === RUN TestCreateViewDuplicateDoesNotClobberMetadata E0720 21:30:02.437934 handler.go:199 S3Tables: error handling GetView: filer: no entry is found in filer store --- PASS: TestCreateViewDuplicateDoesNotClobberMetadata (0.00s) === RUN TestCreateViewRollsBackEntryWhenMetadataWriteFails E0720 21:30:02.438219 handler.go:199 S3Tables: error handling GetView: filer: no entry is found in filer store --- PASS: TestCreateViewRollsBackEntryWhenMetadataWriteFails (0.00s) === RUN TestNameValidationError --- PASS: TestNameValidationError (0.00s) === RUN TestWriteManagerError === RUN TestWriteManagerError/invalid_name_is_a_client_error === RUN TestWriteManagerError/everything_else_is_a_server_fault --- PASS: TestWriteManagerError (0.00s) --- PASS: TestWriteManagerError/invalid_name_is_a_client_error (0.00s) --- PASS: TestWriteManagerError/everything_else_is_a_server_fault (0.00s) === RUN TestGetBucketFromPrefix_WarehouseQueryFallback === RUN TestGetBucketFromPrefix_WarehouseQueryFallback/warehouse_query_routes_to_its_bucket_when_no_prefix_in_path === RUN TestGetBucketFromPrefix_WarehouseQueryFallback/warehouse_query_with_sub-path_still_picks_the_bucket === RUN TestGetBucketFromPrefix_WarehouseQueryFallback/malformed_warehouse_value_falls_through_to_default === RUN TestGetBucketFromPrefix_WarehouseQueryFallback/no_warehouse_and_no_prefix_returns_default --- PASS: TestGetBucketFromPrefix_WarehouseQueryFallback (0.00s) --- PASS: TestGetBucketFromPrefix_WarehouseQueryFallback/warehouse_query_routes_to_its_bucket_when_no_prefix_in_path (0.00s) --- PASS: TestGetBucketFromPrefix_WarehouseQueryFallback/warehouse_query_with_sub-path_still_picks_the_bucket (0.00s) --- PASS: TestGetBucketFromPrefix_WarehouseQueryFallback/malformed_warehouse_value_falls_through_to_default (0.00s) --- PASS: TestGetBucketFromPrefix_WarehouseQueryFallback/no_warehouse_and_no_prefix_returns_default (0.00s) === RUN TestBuildFileIOConfig === RUN TestBuildFileIOConfig/no_endpoint_configured_yields_empty_config === RUN TestBuildFileIOConfig/endpoint_is_advertised_with_path-style-access_and_region --- PASS: TestBuildFileIOConfig (0.00s) --- PASS: TestBuildFileIOConfig/no_endpoint_configured_yields_empty_config (0.00s) --- PASS: TestBuildFileIOConfig/endpoint_is_advertised_with_path-style-access_and_region (0.00s) === RUN TestNormalizeNamespacePropertiesNil --- PASS: TestNormalizeNamespacePropertiesNil (0.00s) === RUN TestNormalizeNamespacePropertiesReturnsInputWhenSet --- PASS: TestNormalizeNamespacePropertiesReturnsInputWhenSet (0.00s) === RUN TestParsePaginationDefaultValues --- PASS: TestParsePaginationDefaultValues (0.00s) === RUN TestParsePaginationUsesCamelCaseParameters --- PASS: TestParsePaginationUsesCamelCaseParameters (0.00s) === RUN TestParsePaginationSupportsHyphenatedFallback --- PASS: TestParsePaginationSupportsHyphenatedFallback (0.00s) === RUN TestParsePaginationRejectsInvalidPageSize === RUN TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=0 === RUN TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=-1 === RUN TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=foo === RUN TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=1001 --- PASS: TestParsePaginationRejectsInvalidPageSize (0.00s) --- PASS: TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=0 (0.00s) --- PASS: TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=-1 (0.00s) --- PASS: TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=foo (0.00s) --- PASS: TestParsePaginationRejectsInvalidPageSize//v1/namespaces?pageSize=1001 (0.00s) === RUN TestRegisterTableRequestWireShape --- PASS: TestRegisterTableRequestWireShape (0.00s) === RUN TestRegisterTableRejectsOutOfBoundsLocation === RUN TestRegisterTableRejectsOutOfBoundsLocation/cross_bucket === RUN TestRegisterTableRejectsOutOfBoundsLocation/traversal --- PASS: TestRegisterTableRejectsOutOfBoundsLocation (0.00s) --- PASS: TestRegisterTableRejectsOutOfBoundsLocation/cross_bucket (0.00s) --- PASS: TestRegisterTableRejectsOutOfBoundsLocation/traversal (0.00s) === RUN TestHandleRenameTableRejectsIncompleteIdentifiers === RUN TestHandleRenameTableRejectsIncompleteIdentifiers/missing_source_name === RUN TestHandleRenameTableRejectsIncompleteIdentifiers/missing_source_namespace === RUN TestHandleRenameTableRejectsIncompleteIdentifiers/missing_destination_name === RUN TestHandleRenameTableRejectsIncompleteIdentifiers/missing_destination_namespace === RUN TestHandleRenameTableRejectsIncompleteIdentifiers/empty_body === RUN TestHandleRenameTableRejectsIncompleteIdentifiers/malformed_json --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers (0.00s) --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers/missing_source_name (0.00s) --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers/missing_source_namespace (0.00s) --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers/missing_destination_name (0.00s) --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers/missing_destination_namespace (0.00s) --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers/empty_body (0.00s) --- PASS: TestHandleRenameTableRejectsIncompleteIdentifiers/malformed_json (0.00s) === RUN TestDecodeCommitTransactionRequest --- PASS: TestDecodeCommitTransactionRequest (0.00s) === RUN TestParseTableChangeSeparatesStatistics --- PASS: TestParseTableChangeSeparatesStatistics (0.00s) === RUN TestParseTableChangeRejectsInvalidRequirements --- PASS: TestParseTableChangeRejectsInvalidRequirements (0.00s) === RUN TestCloneTableMetadataIsIndependent --- PASS: TestCloneTableMetadataIsIndependent (0.00s) === RUN TestBuildTableRestoreRequestCarriesPriorState --- PASS: TestBuildTableRestoreRequestCarriesPriorState (0.00s) === RUN TestRollbackRevertsEveryFlippedField --- PASS: TestRollbackRevertsEveryFlippedField (0.00s) === RUN TestCleanupSafeMetadataKeepsFailedRollback --- PASS: TestCleanupSafeMetadataKeepsFailedRollback (0.00s) === RUN TestCleanupSafeMetadataFirstFlipFails --- PASS: TestCleanupSafeMetadataFirstFlipFails (0.00s) === RUN TestCleanupSafeMetadataAllRolledBack --- PASS: TestCleanupSafeMetadataAllRolledBack (0.00s) === RUN TestApplyNamespacePropertyUpdates --- PASS: TestApplyNamespacePropertyUpdates (0.00s) === RUN TestApplyNamespacePropertyUpdatesRemoveExisting --- PASS: TestApplyNamespacePropertyUpdatesRemoveExisting (0.00s) === RUN TestApplyNamespacePropertyUpdatesDuplicateRemovals --- PASS: TestApplyNamespacePropertyUpdatesDuplicateRemovals (0.00s) === RUN TestHandleUpdateNamespacePropertiesConflict --- PASS: TestHandleUpdateNamespacePropertiesConflict (0.00s) === RUN TestIsViewNotFound --- PASS: TestIsViewNotFound (0.00s) === RUN TestIsViewAlreadyExists --- PASS: TestIsViewAlreadyExists (0.00s) === RUN TestViewPropertiesCopiesAndIsNonNil --- PASS: TestViewPropertiesCopiesAndIsNonNil (0.00s) === RUN TestViewResponseRoundTrip --- PASS: TestViewResponseRoundTrip (0.00s) === RUN TestEnsureMetadataSpecCompliance_BackfillsMissingFields --- PASS: TestEnsureMetadataSpecCompliance_BackfillsMissingFields (0.00s) === RUN TestEnsureMetadataSpecCompliance_PreservesExistingFields --- PASS: TestEnsureMetadataSpecCompliance_PreservesExistingFields (0.00s) === RUN TestEnsureMetadataSpecCompliance_ReplacesExplicitNullsWithSentinels --- PASS: TestEnsureMetadataSpecCompliance_ReplacesExplicitNullsWithSentinels (0.00s) === RUN TestEnsureMetadataSpecCompliance_InvalidJSONReturnedUnchanged --- PASS: TestEnsureMetadataSpecCompliance_InvalidJSONReturnedUnchanged (0.00s) === RUN TestEnsureMetadataSpecCompliance_EmptyInputReturnedUnchanged --- PASS: TestEnsureMetadataSpecCompliance_EmptyInputReturnedUnchanged (0.00s) === RUN TestEnsureMetadataSpecCompliance_PreservesOriginalKeyOrder --- PASS: TestEnsureMetadataSpecCompliance_PreservesOriginalKeyOrder (0.00s) === RUN TestEnsureMetadataSpecCompliance_EmptyObjectBackfilled --- PASS: TestEnsureMetadataSpecCompliance_EmptyObjectBackfilled (0.00s) === RUN TestEnsureMetadataSpecCompliance_AllPresentReturnsSameBytes --- PASS: TestEnsureMetadataSpecCompliance_AllPresentReturnsSameBytes (0.00s) === RUN TestIsValidTablePath --- PASS: TestIsValidTablePath (0.00s) === RUN TestValidateRequestPath_RejectsTraversal === RUN TestValidateRequestPath_RejectsTraversal/clean_namespace+table_passes === RUN TestValidateRequestPath_RejectsTraversal/clean_prefixed_passes === RUN TestValidateRequestPath_RejectsTraversal/clean_namespace_only_passes === RUN TestValidateRequestPath_RejectsTraversal/dotdot_as_prefix_var_rejected === RUN TestValidateRequestPath_RejectsTraversal/dotdot_as_namespace_var_rejected === RUN TestValidateRequestPath_RejectsTraversal/dotdot_as_namespace_var_prefixed_rejected === RUN TestValidateRequestPath_RejectsTraversal/dotdot_as_table_var_rejected === RUN TestValidateRequestPath_RejectsTraversal/dot_as_table_var_rejected === RUN TestValidateRequestPath_RejectsTraversal/unit-sep_namespace_with_dotdot_part_rejected === RUN TestValidateRequestPath_RejectsTraversal/leading_unit-sep_namespace_rejected === RUN TestValidateRequestPath_RejectsTraversal/trailing_unit-sep_namespace_rejected === RUN TestValidateRequestPath_RejectsTraversal/consecutive_unit-sep_namespace_rejected --- PASS: TestValidateRequestPath_RejectsTraversal (0.01s) --- PASS: TestValidateRequestPath_RejectsTraversal/clean_namespace+table_passes (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/clean_prefixed_passes (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/clean_namespace_only_passes (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/dotdot_as_prefix_var_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/dotdot_as_namespace_var_rejected (0.01s) --- PASS: TestValidateRequestPath_RejectsTraversal/dotdot_as_namespace_var_prefixed_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/dotdot_as_table_var_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/dot_as_table_var_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/unit-sep_namespace_with_dotdot_part_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/leading_unit-sep_namespace_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/trailing_unit-sep_namespace_rejected (0.00s) --- PASS: TestValidateRequestPath_RejectsTraversal/consecutive_unit-sep_namespace_rejected (0.00s) === RUN TestValidateRequestPath_RejectsEmptyCapturedVars === RUN TestValidateRequestPath_RejectsEmptyCapturedVars/empty_prefix === RUN TestValidateRequestPath_RejectsEmptyCapturedVars/empty_table === RUN TestValidateRequestPath_RejectsEmptyCapturedVars/empty_namespace --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars (0.00s) --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars/empty_prefix (0.00s) --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars/empty_table (0.00s) --- PASS: TestValidateRequestPath_RejectsEmptyCapturedVars/empty_namespace (0.00s) === RUN TestIsValidNameSegment === RUN TestIsValidNameSegment/empty_ok === RUN TestIsValidNameSegment/plain === RUN TestIsValidNameSegment/with_dot_inside === RUN TestIsValidNameSegment/hidden === RUN TestIsValidNameSegment/bare_dot === RUN TestIsValidNameSegment/bare_dotdot === RUN TestIsValidNameSegment/contains_slash === RUN TestIsValidNameSegment/contains_backslash === RUN TestIsValidNameSegment/contains_nul --- PASS: TestIsValidNameSegment (0.00s) --- PASS: TestIsValidNameSegment/empty_ok (0.00s) --- PASS: TestIsValidNameSegment/plain (0.00s) --- PASS: TestIsValidNameSegment/with_dot_inside (0.00s) --- PASS: TestIsValidNameSegment/hidden (0.00s) --- PASS: TestIsValidNameSegment/bare_dot (0.00s) --- PASS: TestIsValidNameSegment/bare_dotdot (0.00s) --- PASS: TestIsValidNameSegment/contains_slash (0.00s) --- PASS: TestIsValidNameSegment/contains_backslash (0.00s) --- PASS: TestIsValidNameSegment/contains_nul (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/iceberg 0.262s === RUN TestLifecycleToCanonical_TopLevelPrefix --- PASS: TestLifecycleToCanonical_TopLevelPrefix (0.00s) === RUN TestLifecycleToCanonical_FilterPrefix --- PASS: TestLifecycleToCanonical_FilterPrefix (0.00s) === RUN TestLifecycleToCanonical_FilterTagAndSize --- PASS: TestLifecycleToCanonical_FilterTagAndSize (0.00s) === RUN TestLifecycleToCanonical_SingleTagFilter --- PASS: TestLifecycleToCanonical_SingleTagFilter (0.00s) === RUN TestLifecycleToCanonical_MultipleActions --- PASS: TestLifecycleToCanonical_MultipleActions (0.00s) === RUN TestLifecycleToCanonical_ExpirationDate --- PASS: TestLifecycleToCanonical_ExpirationDate (0.00s) === RUN TestLifecycleToCanonical_ExpiredObjectDeleteMarker --- PASS: TestLifecycleToCanonical_ExpiredObjectDeleteMarker (0.00s) === RUN TestLifecycleToCanonical_DisabledRulePreserved --- PASS: TestLifecycleToCanonical_DisabledRulePreserved (0.00s) === RUN TestLifecycleToCanonical_NilSafe --- PASS: TestLifecycleToCanonical_NilSafe (0.00s) === RUN TestLifecycleToCanonical_EmptyRules --- PASS: TestLifecycleToCanonical_EmptyRules (0.00s) === RUN TestLifecycleXMLRoundTrip_NoncurrentVersionExpiration --- PASS: TestLifecycleXMLRoundTrip_NoncurrentVersionExpiration (0.00s) === RUN TestLifecycleXMLRoundTrip_AbortIncompleteMultipartUpload --- PASS: TestLifecycleXMLRoundTrip_AbortIncompleteMultipartUpload (0.00s) === RUN TestLifecycleXMLRoundTrip_FilterWithTag --- PASS: TestLifecycleXMLRoundTrip_FilterWithTag (0.00s) === RUN TestLifecycleXMLRoundTrip_FilterWithAnd --- PASS: TestLifecycleXMLRoundTrip_FilterWithAnd (0.00s) === RUN TestLifecycleXMLRoundTrip_FilterWithSizeOnly --- PASS: TestLifecycleXMLRoundTrip_FilterWithSizeOnly (0.00s) === RUN TestLifecycleXML_TransitionSetFlag --- PASS: TestLifecycleXML_TransitionSetFlag (0.00s) === RUN TestLifecycleXML_NoncurrentVersionTransitionSetFlag --- PASS: TestLifecycleXML_NoncurrentVersionTransitionSetFlag (0.00s) === RUN TestLifecycleXMLRoundTrip_CompleteRule --- PASS: TestLifecycleXMLRoundTrip_CompleteRule (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/lifecycle_xml 0.026s === RUN TestCheckPostPolicy_RejectsUnknownConditionKey --- PASS: TestCheckPostPolicy_RejectsUnknownConditionKey (0.00s) === RUN TestCheckPostPolicy_RejectsExtraXAmzFormField --- PASS: TestCheckPostPolicy_RejectsExtraXAmzFormField (0.00s) === RUN TestCheckPostPolicy_AllowsXAmzAuthFields --- PASS: TestCheckPostPolicy_AllowsXAmzAuthFields (0.00s) === RUN TestCheckPostPolicy_AllowsMatchingXAmzField --- PASS: TestCheckPostPolicy_AllowsMatchingXAmzField (0.00s) === RUN TestCheckPostPolicy_ExistingXAmzMetaCheckStillWorks --- PASS: TestCheckPostPolicy_ExistingXAmzMetaCheckStillWorks (0.00s) === RUN TestCheckPostPolicy_AllowsStartsWithPrefixStem --- PASS: TestCheckPostPolicy_AllowsStartsWithPrefixStem (0.00s) === RUN TestCheckPostPolicy_PrefixStemDoesNotCoverOtherPrefixes --- PASS: TestCheckPostPolicy_PrefixStemDoesNotCoverOtherPrefixes (0.00s) === RUN TestCheckPostPolicy_PrefixStemEnforcesValuePrefix --- PASS: TestCheckPostPolicy_PrefixStemEnforcesValuePrefix (0.00s) === RUN TestCheckPostPolicy_ExactAndPrefixBothEnforced --- PASS: TestCheckPostPolicy_ExactAndPrefixBothEnforced (0.00s) === RUN TestCheckPostPolicy_MultiplePrefixStemsAllEnforced --- PASS: TestCheckPostPolicy_MultiplePrefixStemsAllEnforced (0.00s) === RUN TestCheckPostPolicy_UnknownKeyErrorIncludesPolicyValue --- PASS: TestCheckPostPolicy_UnknownKeyErrorIncludesPolicyValue (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/policy 0.026s === RUN TestExtractPrincipalVariables === RUN TestExtractPrincipalVariables/IAM_User_ARN === RUN TestExtractPrincipalVariables/Assumed_Role_ARN === RUN TestExtractPrincipalVariables/IAM_Role_ARN === RUN TestExtractPrincipalVariables/Non-ARN_principal === RUN TestExtractPrincipalVariables/Wildcard_principal --- PASS: TestExtractPrincipalVariables (0.00s) --- PASS: TestExtractPrincipalVariables/IAM_User_ARN (0.00s) --- PASS: TestExtractPrincipalVariables/Assumed_Role_ARN (0.00s) --- PASS: TestExtractPrincipalVariables/IAM_Role_ARN (0.00s) --- PASS: TestExtractPrincipalVariables/Non-ARN_principal (0.00s) --- PASS: TestExtractPrincipalVariables/Wildcard_principal (0.00s) === RUN TestSubstituteVariablesWithClaims === RUN TestSubstituteVariablesWithClaims/Standard_context_variable === RUN TestSubstituteVariablesWithClaims/JWT_claim_substitution === RUN TestSubstituteVariablesWithClaims/Mixed_variables === RUN TestSubstituteVariablesWithClaims/Variable_not_found --- PASS: TestSubstituteVariablesWithClaims (0.00s) --- PASS: TestSubstituteVariablesWithClaims/Standard_context_variable (0.00s) --- PASS: TestSubstituteVariablesWithClaims/JWT_claim_substitution (0.00s) --- PASS: TestSubstituteVariablesWithClaims/Mixed_variables (0.00s) --- PASS: TestSubstituteVariablesWithClaims/Variable_not_found (0.00s) === RUN TestPolicyVariablesWithPrincipalType --- PASS: TestPolicyVariablesWithPrincipalType (0.00s) === RUN TestPolicyVariablesWithJWTClaims --- PASS: TestPolicyVariablesWithJWTClaims (0.00s) === RUN TestExtractPrincipalVariablesWithAccount --- PASS: TestExtractPrincipalVariablesWithAccount (0.00s) === RUN TestSubstituteVariablesWithLDAP --- PASS: TestSubstituteVariablesWithLDAP (0.00s) === RUN TestSubstituteVariablesSpecialChars === RUN TestSubstituteVariablesSpecialChars/Comparison_operators_in_claims/vars === RUN TestSubstituteVariablesSpecialChars/Path_traversal_attempt_(should_just_substitute_text) --- PASS: TestSubstituteVariablesSpecialChars (0.00s) --- PASS: TestSubstituteVariablesSpecialChars/Comparison_operators_in_claims/vars (0.00s) --- PASS: TestSubstituteVariablesSpecialChars/Path_traversal_attempt_(should_just_substitute_text) (0.00s) === RUN TestIsolationPolicy --- PASS: TestIsolationPolicy (0.00s) === RUN TestNotResourceWithVariables --- PASS: TestNotResourceWithVariables (0.00s) === RUN TestExtractPrincipalVariablesWithPaths === RUN TestExtractPrincipalVariablesWithPaths/IAM_User_with_path === RUN TestExtractPrincipalVariablesWithPaths/IAM_Role_with_path === RUN TestExtractPrincipalVariablesWithPaths/Assumed_Role_with_path --- PASS: TestExtractPrincipalVariablesWithPaths (0.00s) --- PASS: TestExtractPrincipalVariablesWithPaths/IAM_User_with_path (0.00s) --- PASS: TestExtractPrincipalVariablesWithPaths/IAM_Role_with_path (0.00s) --- PASS: TestExtractPrincipalVariablesWithPaths/Assumed_Role_with_path (0.00s) === RUN TestPolicyEngine --- PASS: TestPolicyEngine (0.00s) === RUN TestConditionEvaluators === RUN TestConditionEvaluators/StringEquals_-_match === RUN TestConditionEvaluators/StringEquals_-_no_match === RUN TestConditionEvaluators/StringLike_-_wildcard_match === RUN TestConditionEvaluators/StringLike_-_wildcard_no_match === RUN TestConditionEvaluators/NumericEquals_-_match === RUN TestConditionEvaluators/NumericLessThan_-_match === RUN TestConditionEvaluators/NumericLessThan_-_no_match === RUN TestConditionEvaluators/IpAddress_-_CIDR_match === RUN TestConditionEvaluators/IpAddress_-_CIDR_no_match === RUN TestConditionEvaluators/Bool_-_true_match === RUN TestConditionEvaluators/Bool_-_false_match === RUN TestConditionEvaluators/Bool_-_no_match --- PASS: TestConditionEvaluators (0.00s) --- PASS: TestConditionEvaluators/StringEquals_-_match (0.00s) --- PASS: TestConditionEvaluators/StringEquals_-_no_match (0.00s) --- PASS: TestConditionEvaluators/StringLike_-_wildcard_match (0.00s) --- PASS: TestConditionEvaluators/StringLike_-_wildcard_no_match (0.00s) --- PASS: TestConditionEvaluators/NumericEquals_-_match (0.00s) --- PASS: TestConditionEvaluators/NumericLessThan_-_match (0.00s) --- PASS: TestConditionEvaluators/NumericLessThan_-_no_match (0.00s) --- PASS: TestConditionEvaluators/IpAddress_-_CIDR_match (0.00s) --- PASS: TestConditionEvaluators/IpAddress_-_CIDR_no_match (0.00s) --- PASS: TestConditionEvaluators/Bool_-_true_match (0.00s) --- PASS: TestConditionEvaluators/Bool_-_false_match (0.00s) --- PASS: TestConditionEvaluators/Bool_-_no_match (0.00s) === RUN TestPolicyValidation === RUN TestPolicyValidation/Valid_policy === RUN TestPolicyValidation/Valid_policy_with_single_statement_object === RUN TestPolicyValidation/Invalid_version === RUN TestPolicyValidation/Missing_action === RUN TestPolicyValidation/Invalid_JSON --- PASS: TestPolicyValidation (0.00s) --- PASS: TestPolicyValidation/Valid_policy (0.00s) --- PASS: TestPolicyValidation/Valid_policy_with_single_statement_object (0.00s) --- PASS: TestPolicyValidation/Invalid_version (0.00s) --- PASS: TestPolicyValidation/Missing_action (0.00s) --- PASS: TestPolicyValidation/Invalid_JSON (0.00s) === RUN TestPatternMatching === RUN TestPatternMatching/Exact_match === RUN TestPatternMatching/Wildcard_match === RUN TestPatternMatching/Wildcard_no_match === RUN TestPatternMatching/Full_wildcard === RUN TestPatternMatching/Question_mark_wildcard --- PASS: TestPatternMatching (0.00s) --- PASS: TestPatternMatching/Exact_match (0.00s) --- PASS: TestPatternMatching/Wildcard_match (0.00s) --- PASS: TestPatternMatching/Wildcard_no_match (0.00s) --- PASS: TestPatternMatching/Full_wildcard (0.00s) --- PASS: TestPatternMatching/Question_mark_wildcard (0.00s) === RUN TestExtractConditionValuesFromRequest --- PASS: TestExtractConditionValuesFromRequest (0.00s) === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/uses_right-most_public_X-Forwarded-For_entry === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/falls_back_to_X-Real-Ip_when_X-Forwarded-For_has_no_valid_ip === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/uses_RemoteAddr_ip_when_no_forwarding_headers === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/keeps_unix_socket_marker_when_RemoteAddr_is_not_an_ip === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/uses_IPv6_X-Forwarded-For_entry === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/ignores_spoofed_IP_when_real_client_is_public === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/handles_bracketed_IPv6_remote_address === RUN TestExtractConditionValuesFromRequestSourceIPPrecedence/avoids_returning_DNS_host_names --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/uses_right-most_public_X-Forwarded-For_entry (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/falls_back_to_X-Real-Ip_when_X-Forwarded-For_has_no_valid_ip (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/uses_RemoteAddr_ip_when_no_forwarding_headers (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/keeps_unix_socket_marker_when_RemoteAddr_is_not_an_ip (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/uses_IPv6_X-Forwarded-For_entry (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/ignores_spoofed_IP_when_real_client_is_public (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/handles_bracketed_IPv6_remote_address (0.00s) --- PASS: TestExtractConditionValuesFromRequestSourceIPPrecedence/avoids_returning_DNS_host_names (0.00s) === RUN TestPolicyEvaluationWithConditions --- PASS: TestPolicyEvaluationWithConditions (0.00s) === RUN TestResourceArn === RUN TestResourceArn/Bucket_only === RUN TestResourceArn/Bucket_and_object === RUN TestResourceArn/Bucket_and_nested_object --- PASS: TestResourceArn (0.00s) --- PASS: TestResourceArn/Bucket_only (0.00s) --- PASS: TestResourceArn/Bucket_and_object (0.00s) --- PASS: TestResourceArn/Bucket_and_nested_object (0.00s) === RUN TestActionConversion === RUN TestActionConversion/Already_has_s3_prefix === RUN TestActionConversion/Add_s3_prefix --- PASS: TestActionConversion (0.00s) --- PASS: TestActionConversion/Already_has_s3_prefix (0.00s) --- PASS: TestActionConversion/Add_s3_prefix (0.00s) === RUN TestPolicyEngineForRequest --- PASS: TestPolicyEngineForRequest (0.00s) === RUN TestWildcardMatching === RUN TestWildcardMatching/Exact_match === RUN TestWildcardMatching/Single_wildcard === RUN TestWildcardMatching/Prefix_wildcard === RUN TestWildcardMatching/Suffix_wildcard === RUN TestWildcardMatching/Middle_wildcard === RUN TestWildcardMatching/No_match === RUN TestWildcardMatching/Multiple_wildcards --- PASS: TestWildcardMatching (0.00s) --- PASS: TestWildcardMatching/Exact_match (0.00s) --- PASS: TestWildcardMatching/Single_wildcard (0.00s) --- PASS: TestWildcardMatching/Prefix_wildcard (0.00s) --- PASS: TestWildcardMatching/Suffix_wildcard (0.00s) --- PASS: TestWildcardMatching/Middle_wildcard (0.00s) --- PASS: TestWildcardMatching/No_match (0.00s) --- PASS: TestWildcardMatching/Multiple_wildcards (0.00s) === RUN TestCompilePolicy --- PASS: TestCompilePolicy (0.00s) === RUN TestExistingObjectTagCondition === RUN TestExistingObjectTagCondition/Matching_tag_value_-_should_allow === RUN TestExistingObjectTagCondition/Non-matching_tag_value_-_should_be_indeterminate === RUN TestExistingObjectTagCondition/Missing_tag_-_should_be_indeterminate === RUN TestExistingObjectTagCondition/No_tags_-_should_be_indeterminate === RUN TestExistingObjectTagCondition/Empty_tags_-_should_be_indeterminate === RUN TestExistingObjectTagCondition/Multiple_tags_with_matching_one_-_should_allow --- PASS: TestExistingObjectTagCondition (0.00s) --- PASS: TestExistingObjectTagCondition/Matching_tag_value_-_should_allow (0.00s) --- PASS: TestExistingObjectTagCondition/Non-matching_tag_value_-_should_be_indeterminate (0.00s) --- PASS: TestExistingObjectTagCondition/Missing_tag_-_should_be_indeterminate (0.00s) --- PASS: TestExistingObjectTagCondition/No_tags_-_should_be_indeterminate (0.00s) --- PASS: TestExistingObjectTagCondition/Empty_tags_-_should_be_indeterminate (0.00s) --- PASS: TestExistingObjectTagCondition/Multiple_tags_with_matching_one_-_should_allow (0.00s) === RUN TestExistingObjectTagConditionMultipleTags === RUN TestExistingObjectTagConditionMultipleTags/Both_tags_match_-_should_allow === RUN TestExistingObjectTagConditionMultipleTags/Both_tags_match_(premium_tier)_-_should_allow === RUN TestExistingObjectTagConditionMultipleTags/Only_status_matches_-_should_be_indeterminate === RUN TestExistingObjectTagConditionMultipleTags/Only_tier_matches_-_should_be_indeterminate === RUN TestExistingObjectTagConditionMultipleTags/Neither_tag_matches_-_should_be_indeterminate --- PASS: TestExistingObjectTagConditionMultipleTags (0.00s) --- PASS: TestExistingObjectTagConditionMultipleTags/Both_tags_match_-_should_allow (0.00s) --- PASS: TestExistingObjectTagConditionMultipleTags/Both_tags_match_(premium_tier)_-_should_allow (0.00s) --- PASS: TestExistingObjectTagConditionMultipleTags/Only_status_matches_-_should_be_indeterminate (0.00s) --- PASS: TestExistingObjectTagConditionMultipleTags/Only_tier_matches_-_should_be_indeterminate (0.00s) --- PASS: TestExistingObjectTagConditionMultipleTags/Neither_tag_matches_-_should_be_indeterminate (0.00s) === RUN TestExistingObjectTagDenyPolicy === RUN TestExistingObjectTagDenyPolicy/No_tags_-_allow_by_default_statement === RUN TestExistingObjectTagDenyPolicy/Non-confidential_tag_-_allow === RUN TestExistingObjectTagDenyPolicy/Confidential_tag_-_deny --- PASS: TestExistingObjectTagDenyPolicy (0.00s) --- PASS: TestExistingObjectTagDenyPolicy/No_tags_-_allow_by_default_statement (0.00s) --- PASS: TestExistingObjectTagDenyPolicy/Non-confidential_tag_-_allow (0.00s) --- PASS: TestExistingObjectTagDenyPolicy/Confidential_tag_-_deny (0.00s) === RUN TestMultipartUploadInheritsPutObjectPermission === RUN TestMultipartUploadInheritsPutObjectPermission/s3:CreateMultipartUpload === RUN TestMultipartUploadInheritsPutObjectPermission/s3:UploadPart === RUN TestMultipartUploadInheritsPutObjectPermission/s3:UploadPartCopy === RUN TestMultipartUploadInheritsPutObjectPermission/s3:CompleteMultipartUpload === RUN TestMultipartUploadInheritsPutObjectPermission/s3:AbortMultipartUpload === RUN TestMultipartUploadInheritsPutObjectPermission/s3:ListMultipartUploadParts === RUN TestMultipartUploadInheritsPutObjectPermission/s3:ListBucketMultipartUploads === RUN TestMultipartUploadInheritsPutObjectPermission/s3:ListBucketMultipartUploads_bucket_ARN === RUN TestMultipartUploadInheritsPutObjectPermission/s3:DeleteObject_not_inherited --- PASS: TestMultipartUploadInheritsPutObjectPermission (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:CreateMultipartUpload (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:UploadPart (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:UploadPartCopy (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:CompleteMultipartUpload (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:AbortMultipartUpload (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:ListMultipartUploadParts (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:ListBucketMultipartUploads (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:ListBucketMultipartUploads_bucket_ARN (0.00s) --- PASS: TestMultipartUploadInheritsPutObjectPermission/s3:DeleteObject_not_inherited (0.00s) === RUN TestPolicyVariables --- PASS: TestPolicyVariables (0.00s) === RUN TestEvaluatePolicyForRequestVariables --- PASS: TestEvaluatePolicyForRequestVariables (0.00s) === RUN TestPolicyPrincipalUnmarshal === RUN TestPolicyPrincipalUnmarshal/bare_wildcard === RUN TestPolicyPrincipalUnmarshal/bare_arn === RUN TestPolicyPrincipalUnmarshal/bare_array === RUN TestPolicyPrincipalUnmarshal/AWS_single === RUN TestPolicyPrincipalUnmarshal/AWS_array === RUN TestPolicyPrincipalUnmarshal/AWS_wildcard === RUN TestPolicyPrincipalUnmarshal/AWS_array_wildcard_(public_read) === RUN TestPolicyPrincipalUnmarshal/service === RUN TestPolicyPrincipalUnmarshal/canonical_user === RUN TestPolicyPrincipalUnmarshal/mixed_keys --- PASS: TestPolicyPrincipalUnmarshal (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/bare_wildcard (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/bare_arn (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/bare_array (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/AWS_single (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/AWS_array (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/AWS_wildcard (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/AWS_array_wildcard_(public_read) (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/service (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/canonical_user (0.00s) --- PASS: TestPolicyPrincipalUnmarshal/mixed_keys (0.00s) === RUN TestPolicyPrincipalUnmarshalInvalid --- PASS: TestPolicyPrincipalUnmarshalInvalid (0.00s) === RUN TestNotPrincipalDynamicCompiledPath --- PASS: TestNotPrincipalDynamicCompiledPath (0.00s) === RUN TestPolicyPrincipalRoundTrip --- PASS: TestPolicyPrincipalRoundTrip (0.00s) === RUN TestBucketPolicyAWSObjectPrincipalEndToEnd --- PASS: TestBucketPolicyAWSObjectPrincipalEndToEnd (0.00s) === RUN TestBucketPolicySpecificAWSPrincipal --- PASS: TestBucketPolicySpecificAWSPrincipal (0.00s) === RUN TestBucketPolicyNotPrincipalDeny --- PASS: TestBucketPolicyNotPrincipalDeny (0.00s) === RUN TestBucketPolicyNotPrincipalCompiledPath --- PASS: TestBucketPolicyNotPrincipalCompiledPath (0.00s) === RUN TestPolicyBothPrincipalAndNotPrincipalRejected --- PASS: TestPolicyBothPrincipalAndNotPrincipalRejected (0.00s) === RUN TestSSEStringEqualsPresent --- PASS: TestSSEStringEqualsPresent (0.00s) === RUN TestSSEStringEqualsWrongValue --- PASS: TestSSEStringEqualsWrongValue (0.00s) === RUN TestSSENullConditionAbsent --- PASS: TestSSENullConditionAbsent (0.00s) === RUN TestSSENullConditionPresent --- PASS: TestSSENullConditionPresent (0.00s) === RUN TestSSECaseInsensitiveNormalizationAES256 --- PASS: TestSSECaseInsensitiveNormalizationAES256 (0.00s) === RUN TestSSECaseInsensitiveNormalizationKMS --- PASS: TestSSECaseInsensitiveNormalizationKMS (0.00s) === RUN TestSSEMultipartAES256Exempt --- PASS: TestSSEMultipartAES256Exempt (0.00s) === RUN TestSSEMultipartKMSExempt --- PASS: TestSSEMultipartKMSExempt (0.00s) === RUN TestSSEMultipartNoSSEDenied --- PASS: TestSSEMultipartNoSSEDenied (0.00s) === RUN TestSSEUploadPartCopyKMSExempt --- PASS: TestSSEUploadPartCopyKMSExempt (0.00s) === RUN TestSSEPutObjectStillBlockedWithoutHeader --- PASS: TestSSEPutObjectStillBlockedWithoutHeader (0.00s) === RUN TestCloneStringOrStringSliceCopiesBackingSlice --- PASS: TestCloneStringOrStringSliceCopiesBackingSlice (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine 0.045s === RUN TestNormalizeObjectKey === RUN TestNormalizeObjectKey/simple_key === RUN TestNormalizeObjectKey/key_with_leading_slash === RUN TestNormalizeObjectKey/key_with_directory === RUN TestNormalizeObjectKey/key_with_leading_slash_and_directory === RUN TestNormalizeObjectKey/key_with_duplicate_slashes === RUN TestNormalizeObjectKey/Windows_backslash_-_simple === RUN TestNormalizeObjectKey/Windows_backslash_-_nested === RUN TestNormalizeObjectKey/Windows_backslash_-_with_leading_slash === RUN TestNormalizeObjectKey/mixed_slashes === RUN TestNormalizeObjectKey/Windows_full_path_style_(edge_case) === RUN TestNormalizeObjectKey/empty_string === RUN TestNormalizeObjectKey/just_a_slash === RUN TestNormalizeObjectKey/just_a_backslash --- PASS: TestNormalizeObjectKey (0.00s) --- PASS: TestNormalizeObjectKey/simple_key (0.00s) --- PASS: TestNormalizeObjectKey/key_with_leading_slash (0.00s) --- PASS: TestNormalizeObjectKey/key_with_directory (0.00s) --- PASS: TestNormalizeObjectKey/key_with_leading_slash_and_directory (0.00s) --- PASS: TestNormalizeObjectKey/key_with_duplicate_slashes (0.00s) --- PASS: TestNormalizeObjectKey/Windows_backslash_-_simple (0.00s) --- PASS: TestNormalizeObjectKey/Windows_backslash_-_nested (0.00s) --- PASS: TestNormalizeObjectKey/Windows_backslash_-_with_leading_slash (0.00s) --- PASS: TestNormalizeObjectKey/mixed_slashes (0.00s) --- PASS: TestNormalizeObjectKey/Windows_full_path_style_(edge_case) (0.00s) --- PASS: TestNormalizeObjectKey/empty_string (0.00s) --- PASS: TestNormalizeObjectKey/just_a_slash (0.00s) --- PASS: TestNormalizeObjectKey/just_a_backslash (0.00s) === RUN TestIsValidObjectKey === RUN TestIsValidObjectKey/empty === RUN TestIsValidObjectKey/plain === RUN TestIsValidObjectKey/leading_slash === RUN TestIsValidObjectKey/trailing_slash === RUN TestIsValidObjectKey/hidden_file_ok === RUN TestIsValidObjectKey/dotdot_in_name_ok === RUN TestIsValidObjectKey/double_dots_inside_name === RUN TestIsValidObjectKey/bare_dotdot === RUN TestIsValidObjectKey/bare_dot === RUN TestIsValidObjectKey/leading_dotdot_segment === RUN TestIsValidObjectKey/leading_dot-slash === RUN TestIsValidObjectKey/nested_dotdot_segment === RUN TestIsValidObjectKey/trailing_dotdot_segment === RUN TestIsValidObjectKey/backslash_dotdot === RUN TestIsValidObjectKey/mixed-slash_dotdot === RUN TestIsValidObjectKey/dotdot_after_duplicate_slash === RUN TestIsValidObjectKey/nul_byte --- PASS: TestIsValidObjectKey (0.00s) --- PASS: TestIsValidObjectKey/empty (0.00s) --- PASS: TestIsValidObjectKey/plain (0.00s) --- PASS: TestIsValidObjectKey/leading_slash (0.00s) --- PASS: TestIsValidObjectKey/trailing_slash (0.00s) --- PASS: TestIsValidObjectKey/hidden_file_ok (0.00s) --- PASS: TestIsValidObjectKey/dotdot_in_name_ok (0.00s) --- PASS: TestIsValidObjectKey/double_dots_inside_name (0.00s) --- PASS: TestIsValidObjectKey/bare_dotdot (0.00s) --- PASS: TestIsValidObjectKey/bare_dot (0.00s) --- PASS: TestIsValidObjectKey/leading_dotdot_segment (0.00s) --- PASS: TestIsValidObjectKey/leading_dot-slash (0.00s) --- PASS: TestIsValidObjectKey/nested_dotdot_segment (0.00s) --- PASS: TestIsValidObjectKey/trailing_dotdot_segment (0.00s) --- PASS: TestIsValidObjectKey/backslash_dotdot (0.00s) --- PASS: TestIsValidObjectKey/mixed-slash_dotdot (0.00s) --- PASS: TestIsValidObjectKey/dotdot_after_duplicate_slash (0.00s) --- PASS: TestIsValidObjectKey/nul_byte (0.00s) === RUN TestIsValidBucketName === RUN TestIsValidBucketName/empty_ok === RUN TestIsValidBucketName/plain === RUN TestIsValidBucketName/name_containing_dots === RUN TestIsValidBucketName/bare_dot === RUN TestIsValidBucketName/bare_dotdot === RUN TestIsValidBucketName/with_slash === RUN TestIsValidBucketName/with_backslash === RUN TestIsValidBucketName/with_nul --- PASS: TestIsValidBucketName (0.00s) --- PASS: TestIsValidBucketName/empty_ok (0.00s) --- PASS: TestIsValidBucketName/plain (0.00s) --- PASS: TestIsValidBucketName/name_containing_dots (0.00s) --- PASS: TestIsValidBucketName/bare_dot (0.00s) --- PASS: TestIsValidBucketName/bare_dotdot (0.00s) --- PASS: TestIsValidBucketName/with_slash (0.00s) --- PASS: TestIsValidBucketName/with_backslash (0.00s) --- PASS: TestIsValidBucketName/with_nul (0.00s) === RUN TestIsValidPathSegment === RUN TestIsValidPathSegment/opaque-id_123 === RUN TestIsValidPathSegment/..hidden === RUN TestIsValidPathSegment/#00 === RUN TestIsValidPathSegment/. === RUN TestIsValidPathSegment/.. === RUN TestIsValidPathSegment/dir/value === RUN TestIsValidPathSegment/dir\value === RUN TestIsValidPathSegment/nul\x00value --- PASS: TestIsValidPathSegment (0.00s) --- PASS: TestIsValidPathSegment/opaque-id_123 (0.00s) --- PASS: TestIsValidPathSegment/..hidden (0.00s) --- PASS: TestIsValidPathSegment/#00 (0.00s) --- PASS: TestIsValidPathSegment/. (0.00s) --- PASS: TestIsValidPathSegment/.. (0.00s) --- PASS: TestIsValidPathSegment/dir/value (0.00s) --- PASS: TestIsValidPathSegment/dir\value (0.00s) --- PASS: TestIsValidPathSegment/nul\x00value (0.00s) === RUN TestRemoveDuplicateSlashes === RUN TestRemoveDuplicateSlashes/no_duplicates === RUN TestRemoveDuplicateSlashes/double_slash === RUN TestRemoveDuplicateSlashes/triple_slash === RUN TestRemoveDuplicateSlashes/multiple_duplicate_locations === RUN TestRemoveDuplicateSlashes/empty_string --- PASS: TestRemoveDuplicateSlashes (0.00s) --- PASS: TestRemoveDuplicateSlashes/no_duplicates (0.00s) --- PASS: TestRemoveDuplicateSlashes/double_slash (0.00s) --- PASS: TestRemoveDuplicateSlashes/triple_slash (0.00s) --- PASS: TestRemoveDuplicateSlashes/multiple_duplicate_locations (0.00s) --- PASS: TestRemoveDuplicateSlashes/empty_string (0.00s) === RUN TestIdentityHolderPropagation --- PASS: TestIdentityHolderPropagation (0.00s) === RUN TestEnsureIdentityHolderIdempotent --- PASS: TestEnsureIdentityHolderIdempotent (0.00s) === RUN TestGetIdentityNameFromContextWithoutHolder --- PASS: TestGetIdentityNameFromContextWithoutHolder (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants 0.032s ? github.com/seaweedfs/seaweedfs/weed/s3api/s3_objectlock [no test files] === RUN Test_verifyBucketName --- PASS: Test_verifyBucketName (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3bucket 0.012s === RUN TestGetAccessLogUsesAmzRequestID --- PASS: TestGetAccessLogUsesAmzRequestID (0.00s) === RUN TestGetAccessLogRemoteIP === RUN TestGetAccessLogRemoteIP/falls_back_to_RemoteAddr_(port_stripped)_when_no_headers_set === RUN TestGetAccessLogRemoteIP/preserves_IPv6_host_from_RemoteAddr === RUN TestGetAccessLogRemoteIP/returns_RemoteAddr_unchanged_when_no_port_present === RUN TestGetAccessLogRemoteIP/uses_X-Real-IP_when_X-Forwarded-For_is_absent === RUN TestGetAccessLogRemoteIP/prefers_X-Forwarded-For_over_X-Real-IP === RUN TestGetAccessLogRemoteIP/uses_first_hop_in_X-Forwarded-For_chain === RUN TestGetAccessLogRemoteIP/skips_empty_leading_entries_in_X-Forwarded-For --- PASS: TestGetAccessLogRemoteIP (0.00s) --- PASS: TestGetAccessLogRemoteIP/falls_back_to_RemoteAddr_(port_stripped)_when_no_headers_set (0.00s) --- PASS: TestGetAccessLogRemoteIP/preserves_IPv6_host_from_RemoteAddr (0.00s) --- PASS: TestGetAccessLogRemoteIP/returns_RemoteAddr_unchanged_when_no_port_present (0.00s) --- PASS: TestGetAccessLogRemoteIP/uses_X-Real-IP_when_X-Forwarded-For_is_absent (0.00s) --- PASS: TestGetAccessLogRemoteIP/prefers_X-Forwarded-For_over_X-Real-IP (0.00s) --- PASS: TestGetAccessLogRemoteIP/uses_first_hop_in_X-Forwarded-For_chain (0.00s) --- PASS: TestGetAccessLogRemoteIP/skips_empty_leading_entries_in_X-Forwarded-For (0.00s) === RUN TestGetAccessLogRequesterFromFallback --- PASS: TestGetAccessLogRequesterFromFallback (0.00s) === RUN TestGetAccessLogRequesterAnonymous --- PASS: TestGetAccessLogRequesterAnonymous (0.00s) === RUN TestAuditTrackingFlag --- PASS: TestAuditTrackingFlag (0.00s) === RUN TestWriteErrorResponseReusesRequestID --- PASS: TestWriteErrorResponseReusesRequestID (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3err 0.012s === RUN TestRuleActionKinds_SingleAction === RUN TestRuleActionKinds_SingleAction/expiration_days === RUN TestRuleActionKinds_SingleAction/expiration_date === RUN TestRuleActionKinds_SingleAction/expired_delete_marker === RUN TestRuleActionKinds_SingleAction/noncurrent_days === RUN TestRuleActionKinds_SingleAction/newer_noncurrent_alone === RUN TestRuleActionKinds_SingleAction/abort_mpu --- PASS: TestRuleActionKinds_SingleAction (0.00s) --- PASS: TestRuleActionKinds_SingleAction/expiration_days (0.00s) --- PASS: TestRuleActionKinds_SingleAction/expiration_date (0.00s) --- PASS: TestRuleActionKinds_SingleAction/expired_delete_marker (0.00s) --- PASS: TestRuleActionKinds_SingleAction/noncurrent_days (0.00s) --- PASS: TestRuleActionKinds_SingleAction/newer_noncurrent_alone (0.00s) --- PASS: TestRuleActionKinds_SingleAction/abort_mpu (0.00s) === RUN TestRuleActionKinds_MultiAction --- PASS: TestRuleActionKinds_MultiAction (0.00s) === RUN TestRuleActionKinds_NoncurrentDaysSubsumesNewerNoncurrent --- PASS: TestRuleActionKinds_NoncurrentDaysSubsumesNewerNoncurrent (0.00s) === RUN TestRuleActionKinds_Empty --- PASS: TestRuleActionKinds_Empty (0.00s) === RUN TestActionKind_StringIsStable --- PASS: TestActionKind_StringIsStable (0.00s) === RUN TestComputeDueAt_ExpirationDays --- PASS: TestComputeDueAt_ExpirationDays (0.00s) === RUN TestComputeDueAt_ExpirationDate --- PASS: TestComputeDueAt_ExpirationDate (0.00s) === RUN TestComputeDueAt_DeleteMarker_NotSoleSurvivor --- PASS: TestComputeDueAt_DeleteMarker_NotSoleSurvivor (0.00s) === RUN TestComputeDueAt_KindNotDeclared --- PASS: TestComputeDueAt_KindNotDeclared (0.00s) === RUN TestComputeDueAt_WrongShapeForKind --- PASS: TestComputeDueAt_WrongShapeForKind (0.00s) === RUN TestComputeDueAt_NoncurrentDeleteMarkerHonorsNoncurrentDays --- PASS: TestComputeDueAt_NoncurrentDeleteMarkerHonorsNoncurrentDays (0.00s) === RUN TestComputeDueAt_NoncurrentSuccessorMtime --- PASS: TestComputeDueAt_NoncurrentSuccessorMtime (0.00s) === RUN TestComputeDueAt_FilterRejects --- PASS: TestComputeDueAt_FilterRejects (0.00s) === RUN TestComputeDueAt_DisabledRule --- PASS: TestComputeDueAt_DisabledRule (0.00s) === RUN TestComputeDueAt_MPUInit --- PASS: TestComputeDueAt_MPUInit (0.00s) === RUN TestEvaluateAction_DisabledRuleNeverFires --- PASS: TestEvaluateAction_DisabledRuleNeverFires (0.00s) === RUN TestEvaluateAction_NilInputs --- PASS: TestEvaluateAction_NilInputs (0.00s) === RUN TestEvaluateAction_ExpirationDaysBoundary --- PASS: TestEvaluateAction_ExpirationDaysBoundary (0.00s) === RUN TestEvaluateAction_KindFiltersByDeclaredAction --- PASS: TestEvaluateAction_KindFiltersByDeclaredAction (0.00s) === RUN TestEvaluateAction_MultiActionRule_SiblingsIndependent --- PASS: TestEvaluateAction_MultiActionRule_SiblingsIndependent (0.00s) === RUN TestEvaluateAction_ExpirationDate --- PASS: TestEvaluateAction_ExpirationDate (0.00s) === RUN TestEvaluateAction_ExpiredObjectDeleteMarker --- PASS: TestEvaluateAction_ExpiredObjectDeleteMarker (0.00s) === RUN TestEvaluateAction_NoncurrentDeleteMarkerExpiresViaNoncurrentDays --- PASS: TestEvaluateAction_NoncurrentDeleteMarkerExpiresViaNoncurrentDays (0.00s) === RUN TestEvaluateAction_NoncurrentVersionDays --- PASS: TestEvaluateAction_NoncurrentVersionDays (0.00s) === RUN TestEvaluateAction_NoncurrentDaysFallsBackToModTime --- PASS: TestEvaluateAction_NoncurrentDaysFallsBackToModTime (0.00s) === RUN TestEvaluateAction_NoncurrentDays_WithKeepN_NilIndexIsNoOp --- PASS: TestEvaluateAction_NoncurrentDays_WithKeepN_NilIndexIsNoOp (0.00s) === RUN TestEvaluateAction_NewerNoncurrent_NilIndexIsNoOp --- PASS: TestEvaluateAction_NewerNoncurrent_NilIndexIsNoOp (0.00s) === RUN TestEvaluateAction_NewerNoncurrentCountOnly --- PASS: TestEvaluateAction_NewerNoncurrentCountOnly (0.00s) === RUN TestEvaluateAction_NoncurrentDaysAndCount --- PASS: TestEvaluateAction_NoncurrentDaysAndCount (0.00s) === RUN TestEvaluateAction_AbortMultipartUpload --- PASS: TestEvaluateAction_AbortMultipartUpload (0.00s) === RUN TestEvaluateAction_PrefixFilter --- PASS: TestEvaluateAction_PrefixFilter (0.00s) === RUN TestEvaluateAction_TagFilter --- PASS: TestEvaluateAction_TagFilter (0.00s) === RUN TestEvaluateAction_SizeFilter --- PASS: TestEvaluateAction_SizeFilter (0.00s) === RUN TestEvaluateAction_EmptyPrefixMatchesAll --- PASS: TestEvaluateAction_EmptyPrefixMatchesAll (0.00s) === RUN TestEvaluateAction_MPUInitDoesNotFireNoncurrent --- PASS: TestEvaluateAction_MPUInitDoesNotFireNoncurrent (0.00s) === RUN TestEventLogHorizon_PerActionIsIndependent === RUN TestEventLogHorizon_PerActionIsIndependent/expiration_days === RUN TestEventLogHorizon_PerActionIsIndependent/abort_mpu === RUN TestEventLogHorizon_PerActionIsIndependent/noncurrent_days === RUN TestEventLogHorizon_PerActionIsIndependent/expiration_date --- PASS: TestEventLogHorizon_PerActionIsIndependent (0.00s) --- PASS: TestEventLogHorizon_PerActionIsIndependent/expiration_days (0.00s) --- PASS: TestEventLogHorizon_PerActionIsIndependent/abort_mpu (0.00s) --- PASS: TestEventLogHorizon_PerActionIsIndependent/noncurrent_days (0.00s) --- PASS: TestEventLogHorizon_PerActionIsIndependent/expiration_date (0.00s) === RUN TestEventLogHorizon_NewerNoncurrentCountOnlyIsSmallDelay --- PASS: TestEventLogHorizon_NewerNoncurrentCountOnlyIsSmallDelay (0.00s) === RUN TestEventLogHorizon_NewerNoncurrentWithDaysIsZero --- PASS: TestEventLogHorizon_NewerNoncurrentWithDaysIsZero (0.00s) === RUN TestEventLogHorizon_ExpiredDeleteMarkerIsSmallDelay --- PASS: TestEventLogHorizon_ExpiredDeleteMarkerIsSmallDelay (0.00s) === RUN TestEventLogHorizon_DateKindReturnsZero --- PASS: TestEventLogHorizon_DateKindReturnsZero (0.00s) === RUN TestEventLogHorizon_NilRule --- PASS: TestEventLogHorizon_NilRule (0.00s) === RUN TestEventLogHorizon_KindNotDeclaredReturnsZero --- PASS: TestEventLogHorizon_KindNotDeclaredReturnsZero (0.00s) === RUN TestActionKind_StringUnspecifiedDefault --- PASS: TestActionKind_StringUnspecifiedDefault (0.00s) === RUN TestHashExtended_DirectFromLifecyclePackage --- PASS: TestHashExtended_DirectFromLifecyclePackage (0.00s) === RUN TestMinTriggerAge_PerKind === RUN TestMinTriggerAge_PerKind/expiration_days === RUN TestMinTriggerAge_PerKind/noncurrent_days === RUN TestMinTriggerAge_PerKind/abort_mpu === RUN TestMinTriggerAge_PerKind/expiration_date === RUN TestMinTriggerAge_PerKind/newer_noncurrent === RUN TestMinTriggerAge_PerKind/expired_delete_marker --- PASS: TestMinTriggerAge_PerKind (0.00s) --- PASS: TestMinTriggerAge_PerKind/expiration_days (0.00s) --- PASS: TestMinTriggerAge_PerKind/noncurrent_days (0.00s) --- PASS: TestMinTriggerAge_PerKind/abort_mpu (0.00s) --- PASS: TestMinTriggerAge_PerKind/expiration_date (0.00s) --- PASS: TestMinTriggerAge_PerKind/newer_noncurrent (0.00s) --- PASS: TestMinTriggerAge_PerKind/expired_delete_marker (0.00s) === RUN TestMinTriggerAge_KindNotSetReturnsZero --- PASS: TestMinTriggerAge_KindNotSetReturnsZero (0.00s) === RUN TestMinTriggerAge_DateOnlyReturnsZero --- PASS: TestMinTriggerAge_DateOnlyReturnsZero (0.00s) === RUN TestMinTriggerAge_NilRule --- PASS: TestMinTriggerAge_NilRule (0.00s) === RUN TestSuccessorFromEntryStamp_NilOrMissingReturnsZero --- PASS: TestSuccessorFromEntryStamp_NilOrMissingReturnsZero (0.00s) === RUN TestSuccessorFromEntryStamp_EmptyOrInvalidReturnsZero --- PASS: TestSuccessorFromEntryStamp_EmptyOrInvalidReturnsZero (0.00s) === RUN TestSuccessorFromEntryStamp_NonPositiveReturnsZero --- PASS: TestSuccessorFromEntryStamp_NonPositiveReturnsZero (0.00s) === RUN TestSuccessorFromEntryStamp_PositiveNanosRoundTrip --- PASS: TestSuccessorFromEntryStamp_PositiveNanosRoundTrip (0.00s) === RUN TestSuccessorFromEntryStamp_OrderedNanosStayOrdered --- PASS: TestSuccessorFromEntryStamp_OrderedNanosStayOrdered (0.00s) === RUN TestRuleHash_Stable --- PASS: TestRuleHash_Stable (0.00s) === RUN TestRuleHash_TagOrderInvariant --- PASS: TestRuleHash_TagOrderInvariant (0.00s) === RUN TestRuleHash_PrefixTrailingSlashMattersToHash --- PASS: TestRuleHash_PrefixTrailingSlashMattersToHash (0.00s) === RUN TestRuleHash_IDIgnored --- PASS: TestRuleHash_IDIgnored (0.00s) === RUN TestRuleHash_StatusIgnored --- PASS: TestRuleHash_StatusIgnored (0.00s) === RUN TestRuleHash_DifferentDaysHashDifferent --- PASS: TestRuleHash_DifferentDaysHashDifferent (0.00s) === RUN TestRuleHash_DifferentActionTypesHashDifferent --- PASS: TestRuleHash_DifferentActionTypesHashDifferent (0.00s) === RUN TestRuleHash_FilterMatters --- PASS: TestRuleHash_FilterMatters (0.00s) === RUN TestRuleHash_NilSafe --- PASS: TestRuleHash_NilSafe (0.00s) === RUN TestRuleHash_TagDelimiterCollisionResistant --- PASS: TestRuleHash_TagDelimiterCollisionResistant (0.00s) === RUN TestRuleHash_TagNewlineCollisionResistant --- PASS: TestRuleHash_TagNewlineCollisionResistant (0.00s) === RUN TestRuleHash_PrefixSeparatorIsolation --- PASS: TestRuleHash_PrefixSeparatorIsolation (0.00s) === RUN TestShardIDInRange --- PASS: TestShardIDInRange (0.00s) === RUN TestShardIDDeterministic --- PASS: TestShardIDDeterministic (0.00s) === RUN TestShardIDDistinctFromBucket --- PASS: TestShardIDDistinctFromBucket (0.00s) === RUN TestCompareVersionIds_EqualReturnsZero --- PASS: TestCompareVersionIds_EqualReturnsZero (0.00s) === RUN TestCompareVersionIds_NullSortsLast --- PASS: TestCompareVersionIds_NullSortsLast (0.00s) === RUN TestCompareVersionIds_BothNewFormatSmallerIsNewer --- PASS: TestCompareVersionIds_BothNewFormatSmallerIsNewer (0.00s) === RUN TestCompareVersionIds_BothOldFormatLargerIsNewer --- PASS: TestCompareVersionIds_BothOldFormatLargerIsNewer (0.00s) === RUN TestCompareVersionIds_MixedFormatComparesByTimestamp --- PASS: TestCompareVersionIds_MixedFormatComparesByTimestamp (0.00s) === RUN TestCompareVersionIds_MixedFormatEqualTimestampReturnsZero --- PASS: TestCompareVersionIds_MixedFormatEqualTimestampReturnsZero (0.00s) === RUN TestIsNewFormatVersionId_TooShortRejected --- PASS: TestIsNewFormatVersionId_TooShortRejected (0.00s) === RUN TestIsNewFormatVersionId_NullRejected --- PASS: TestIsNewFormatVersionId_NullRejected (0.00s) === RUN TestIsNewFormatVersionId_NonHexRejected --- PASS: TestIsNewFormatVersionId_NonHexRejected (0.00s) === RUN TestIsNewFormatVersionId_ThresholdBoundary --- PASS: TestIsNewFormatVersionId_ThresholdBoundary (0.00s) === RUN TestGetVersionTimestamp_ReturnsZeroOnMalformed --- PASS: TestGetVersionTimestamp_ReturnsZeroOnMalformed (0.00s) === RUN TestGetVersionTimestamp_OldFormatReturnsRawValue --- PASS: TestGetVersionTimestamp_OldFormatReturnsRawValue (0.00s) === RUN TestGetVersionTimestamp_NewFormatReturnsInvertedValue --- PASS: TestGetVersionTimestamp_NewFormatReturnsInvertedValue (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle 0.045s === RUN TestHasPrefix === RUN TestHasPrefix/matching_prefix === RUN TestHasPrefix/exact_match === RUN TestHasPrefix/non-matching_prefix === RUN TestHasPrefix/shorter_than_prefix === RUN TestHasPrefix/empty_prefix_matches_all === RUN TestHasPrefix/both_empty === RUN TestHasPrefix/empty_input_non-empty_prefix --- PASS: TestHasPrefix (0.00s) --- PASS: TestHasPrefix/matching_prefix (0.00s) --- PASS: TestHasPrefix/exact_match (0.00s) --- PASS: TestHasPrefix/non-matching_prefix (0.00s) --- PASS: TestHasPrefix/shorter_than_prefix (0.00s) --- PASS: TestHasPrefix/empty_prefix_matches_all (0.00s) --- PASS: TestHasPrefix/both_empty (0.00s) --- PASS: TestHasPrefix/empty_input_non-empty_prefix (0.00s) === RUN TestWalk_DispatchesDueActions --- PASS: TestWalk_DispatchesDueActions (0.00s) === RUN TestWalk_MultiActionRule_AllDueDispatched --- PASS: TestWalk_MultiActionRule_AllDueDispatched (0.00s) === RUN TestWalk_NotYetDueSkipped --- PASS: TestWalk_NotYetDueSkipped (0.00s) === RUN TestWalk_DateActionFiresAfterDate --- PASS: TestWalk_DateActionFiresAfterDate (0.00s) === RUN TestWalk_DateActionSkippedBeforeDate --- PASS: TestWalk_DateActionSkippedBeforeDate (0.00s) === RUN TestWalk_DirectoryEntriesSkipped --- PASS: TestWalk_DirectoryEntriesSkipped (0.00s) === RUN TestWalk_DisabledModeSkipped --- PASS: TestWalk_DisabledModeSkipped (0.00s) === RUN TestWalk_PendingBootstrapNotDispatched --- PASS: TestWalk_PendingBootstrapNotDispatched (0.00s) === RUN TestWalk_DispatchErrorHaltsAtCheckpoint W0720 21:30:02.255029 walker.go:173 lifecycle bootstrap: dispatch bk/a kind=expiration_days: dispatch boom --- PASS: TestWalk_DispatchErrorHaltsAtCheckpoint (0.00s) === RUN TestWalk_ResumeFromCheckpoint --- PASS: TestWalk_ResumeFromCheckpoint (0.00s) === RUN TestWalk_MPUInitDirMatchesByDestKey --- PASS: TestWalk_MPUInitDirMatchesByDestKey (0.00s) === RUN TestWalk_NonMPUDirectorySkipped --- PASS: TestWalk_NonMPUDirectorySkipped (0.00s) === RUN TestWalk_MPUInitDoesNotFireNoncurrent --- PASS: TestWalk_MPUInitDoesNotFireNoncurrent (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/bootstrap 0.030s === RUN TestSummarizeShardCursorLag_AllColdStart --- PASS: TestSummarizeShardCursorLag_AllColdStart (0.00s) === RUN TestSummarizeShardCursorLag_PicksMaxAcrossShards --- PASS: TestSummarizeShardCursorLag_PicksMaxAcrossShards (0.00s) === RUN TestSummarizeShardCursorLag_PartialFill --- PASS: TestSummarizeShardCursorLag_PartialFill (0.00s) === RUN TestFilerCursorPersister_EmptyLoadReturnsNotFound --- PASS: TestFilerCursorPersister_EmptyLoadReturnsNotFound (0.00s) === RUN TestFilerCursorPersister_SaveLoadRoundTrip --- PASS: TestFilerCursorPersister_SaveLoadRoundTrip (0.00s) === RUN TestFilerCursorPersister_IsolatesShards --- PASS: TestFilerCursorPersister_IsolatesShards (0.00s) === RUN TestFilerCursorPersister_CorruptDataReturnsError --- PASS: TestFilerCursorPersister_CorruptDataReturnsError (0.00s) === RUN TestFilerCursorPersister_EmptyFileReturnsError --- PASS: TestFilerCursorPersister_EmptyFileReturnsError (0.00s) === RUN TestFilerCursorPersister_WrongVersionReturnsError --- PASS: TestFilerCursorPersister_WrongVersionReturnsError (0.00s) === RUN TestFilerCursorPersister_ShardIDMismatchReturnsError --- PASS: TestFilerCursorPersister_ShardIDMismatchReturnsError (0.00s) === RUN TestFilerCursorPersister_HashLengthMismatchReturnsError --- PASS: TestFilerCursorPersister_HashLengthMismatchReturnsError (0.00s) === RUN TestFilerCursorPersister_NilStoreErrors --- PASS: TestFilerCursorPersister_NilStoreErrors (0.00s) === RUN TestDispatch_FirstAttemptSucceeds --- PASS: TestDispatch_FirstAttemptSucceeds (0.00s) === RUN TestDispatch_TransportRetryThenSucceed --- PASS: TestDispatch_TransportRetryThenSucceed (0.48s) === RUN TestDispatch_ExhaustsRetriesReturnsError --- PASS: TestDispatch_ExhaustsRetriesReturnsError (0.44s) === RUN TestDispatch_ServerOutcomeRetryLaterIsNotRetried --- PASS: TestDispatch_ServerOutcomeRetryLaterIsNotRetried (0.00s) === RUN TestDispatch_ServerOutcomeBlockedNotRetried --- PASS: TestDispatch_ServerOutcomeBlockedNotRetried (0.00s) === RUN TestDispatch_ContextCancelledShortCircuits --- PASS: TestDispatch_ContextCancelledShortCircuits (0.00s) === RUN TestBuildDeleteRequest_RuleHashAndIdentity --- PASS: TestBuildDeleteRequest_RuleHashAndIdentity (0.00s) === RUN TestFilerListFunc_EmitsFlatFiles --- PASS: TestFilerListFunc_EmitsFlatFiles (0.00s) === RUN TestFilerListFunc_PropagatesTagsOnFlatFiles --- PASS: TestFilerListFunc_PropagatesTagsOnFlatFiles (0.00s) === RUN TestFilerListFunc_RecursesIntoSubdirs --- PASS: TestFilerListFunc_RecursesIntoSubdirs (0.00s) === RUN TestFilerListFunc_MPUInitEmitsWithDestKey --- PASS: TestFilerListFunc_MPUInitEmitsWithDestKey (0.00s) === RUN TestFilerListFunc_MPUInitWithoutDestKeyIsSkipped --- PASS: TestFilerListFunc_MPUInitWithoutDestKeyIsSkipped (0.00s) === RUN TestFilerListFunc_VersionedExpansionMarksLatestByPointer --- PASS: TestFilerListFunc_VersionedExpansionMarksLatestByPointer (0.00s) === RUN TestFilerListFunc_VersionedExpansionStalePointerFallsBackToNewestSibling --- PASS: TestFilerListFunc_VersionedExpansionStalePointerFallsBackToNewestSibling (0.00s) === RUN TestFilerListFunc_VersionedExpansionNoPointerNewestSiblingWins --- PASS: TestFilerListFunc_VersionedExpansionNoPointerNewestSiblingWins (0.00s) === RUN TestFilerListFunc_VersionedExpansionExplicitNullIsLatestWhenPointerMissing --- PASS: TestFilerListFunc_VersionedExpansionExplicitNullIsLatestWhenPointerMissing (0.00s) === RUN TestFilerListFunc_VersionedExpansionExplicitNullBeatsNewerSiblingWhenPointerMissing --- PASS: TestFilerListFunc_VersionedExpansionExplicitNullBeatsNewerSiblingWhenPointerMissing (0.00s) === RUN TestFilerListFunc_VersionsDirWithoutMarkersRecursesAsRegular --- PASS: TestFilerListFunc_VersionsDirWithoutMarkersRecursesAsRegular (0.00s) === RUN TestFilerListFunc_VersionedDeleteMarkerPropagates --- PASS: TestFilerListFunc_VersionedDeleteMarkerPropagates (0.00s) === RUN TestFilerListFunc_HonorsStart --- PASS: TestFilerListFunc_HonorsStart (0.00s) === RUN TestFilerListFunc_NilClient --- PASS: TestFilerListFunc_NilClient (0.00s) === RUN TestFilerListFunc_AttributesPropagate --- PASS: TestFilerListFunc_AttributesPropagate (0.00s) === RUN TestJitterBounds --- PASS: TestJitterBounds (0.00s) === RUN TestJitterZeroAndNegative --- PASS: TestJitterZeroAndNegative (0.00s) === RUN TestJitterTinyDuration --- PASS: TestJitterTinyDuration (0.00s) === RUN TestProcessMatches_NotYetDueDoesNotSuppressDueOnSameRule --- PASS: TestProcessMatches_NotYetDueDoesNotSuppressDueOnSameRule (0.00s) === RUN TestProcessMatches_OrderingDoesNotMatter --- PASS: TestProcessMatches_OrderingDoesNotMatter (0.00s) === RUN TestProcessMatches_HaltOnServerOutcomeStopsRemaining --- PASS: TestProcessMatches_HaltOnServerOutcomeStopsRemaining (0.00s) === RUN TestProcessMatches_EmptyMatchesIsNoop --- PASS: TestProcessMatches_EmptyMatchesIsNoop (0.00s) === RUN TestProcessMatches_AllDueNoSkippedFlag --- PASS: TestProcessMatches_AllDueNoSkippedFlag (0.00s) === RUN TestProcessMatches_DispatchCounterIncrements --- PASS: TestProcessMatches_DispatchCounterIncrements (0.00s) === RUN TestWalkBuckets_DispatchesOnlyShardMatchingEntries --- PASS: TestWalkBuckets_DispatchesOnlyShardMatchingEntries (0.00s) === RUN TestWalkBuckets_NilGuards --- PASS: TestWalkBuckets_NilGuards (0.00s) === RUN TestWalkBuckets_OneBucketErrorDoesNotStopOthers --- PASS: TestWalkBuckets_OneBucketErrorDoesNotStopOthers (0.00s) === RUN TestWalkBuckets_HonorsContextCancellationBetweenBuckets --- PASS: TestWalkBuckets_HonorsContextCancellationBetweenBuckets (0.00s) === RUN TestEntryShardID_MPUUsesDestKey --- PASS: TestEntryShardID_MPUUsesDestKey (0.00s) === RUN TestWalkerDispatcher_NonVersionedSendsExpectedRequest --- PASS: TestWalkerDispatcher_NonVersionedSendsExpectedRequest (0.00s) === RUN TestWalkerDispatcher_VersionedPassesVersionID --- PASS: TestWalkerDispatcher_VersionedPassesVersionID (0.00s) === RUN TestWalkerDispatcher_MPUInitUsesUploadsPath --- PASS: TestWalkerDispatcher_MPUInitUsesUploadsPath (0.00s) === RUN TestWalkerDispatcher_MPUInitEmptyDestKeyErrors --- PASS: TestWalkerDispatcher_MPUInitEmptyDestKeyErrors (0.00s) === RUN TestWalkerDispatcher_AcceptsAllResolvedOutcomes --- PASS: TestWalkerDispatcher_AcceptsAllResolvedOutcomes (0.00s) === RUN TestWalkerDispatcher_UnresolvedOutcomeReturnsError --- PASS: TestWalkerDispatcher_UnresolvedOutcomeReturnsError (0.00s) === RUN TestWalkerDispatcher_TransportErrorReturnsWrappedError --- PASS: TestWalkerDispatcher_TransportErrorReturnsWrappedError (0.00s) === RUN TestWalkerDispatcher_NilResponseReturnsError --- PASS: TestWalkerDispatcher_NilResponseReturnsError (0.00s) === RUN TestWalkerDispatcher_LimiterWaitsBeforeDispatch --- PASS: TestWalkerDispatcher_LimiterWaitsBeforeDispatch (0.05s) === RUN TestWalkerDispatcher_LimiterContextCancelHaltsWalker --- PASS: TestWalkerDispatcher_LimiterContextCancelHaltsWalker (0.00s) === RUN TestWalkerDispatcher_NilGuardsReturnError --- PASS: TestWalkerDispatcher_NilGuardsReturnError (0.00s) === RUN TestWalkerDue === RUN TestWalkerDue/interval_zero_always_due === RUN TestWalkerDue/never_walked_is_due === RUN TestWalkerDue/interval_elapsed === RUN TestWalkerDue/interval_exactly_elapsed === RUN TestWalkerDue/interval_not_yet_elapsed === RUN TestWalkerDue/future_lastWalked --- PASS: TestWalkerDue (0.00s) --- PASS: TestWalkerDue/interval_zero_always_due (0.00s) --- PASS: TestWalkerDue/never_walked_is_due (0.00s) --- PASS: TestWalkerDue/interval_elapsed (0.00s) --- PASS: TestWalkerDue/interval_exactly_elapsed (0.00s) --- PASS: TestWalkerDue/interval_not_yet_elapsed (0.00s) --- PASS: TestWalkerDue/future_lastWalked (0.00s) === RUN TestRunShard_WalkerThrottle === RUN TestRunShard_WalkerThrottle/interval=0_fires_every_pass === RUN TestRunShard_WalkerThrottle/throttled:_second_pass_within_interval === RUN TestRunShard_WalkerThrottle/throttled:_second_pass_past_interval --- PASS: TestRunShard_WalkerThrottle (0.00s) --- PASS: TestRunShard_WalkerThrottle/interval=0_fires_every_pass (0.00s) --- PASS: TestRunShard_WalkerThrottle/throttled:_second_pass_within_interval (0.00s) --- PASS: TestRunShard_WalkerThrottle/throttled:_second_pass_past_interval (0.00s) === RUN TestValidate_RejectsNegativeWalkerInterval --- PASS: TestValidate_RejectsNegativeWalkerInterval (0.00s) === RUN TestRunShard_ColdStartDoesNotDoubleWalk --- PASS: TestRunShard_ColdStartDoesNotDoubleWalk (0.00s) === RUN TestRunShard_RecoveryWalkerSetsLastWalkedAnchor --- PASS: TestRunShard_RecoveryWalkerSetsLastWalkedAnchor (0.00s) === RUN TestRunShard_WalkerInvokedOnRecoveryBranch --- PASS: TestRunShard_WalkerInvokedOnRecoveryBranch (0.00s) === RUN TestRunShard_NilWalkerOnRecoveryIsNoop --- PASS: TestRunShard_NilWalkerOnRecoveryIsNoop (0.00s) === RUN TestRunShard_WalkerErrorPropagates --- PASS: TestRunShard_WalkerErrorPropagates (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/dailyrun 1.008s === RUN TestSurvivors_NoVersionsNoNullReturnsZero --- PASS: TestSurvivors_NoVersionsNoNullReturnsZero (0.00s) === RUN TestSurvivors_OneVersionPopulatesLoneEntry --- PASS: TestSurvivors_OneVersionPopulatesLoneEntry (0.00s) === RUN TestSurvivors_MoreThanOneClearsLoneEntry --- PASS: TestSurvivors_MoreThanOneClearsLoneEntry (0.00s) === RUN TestSurvivors_NullVersionDetectedFromBareRegularFile --- PASS: TestSurvivors_NullVersionDetectedFromBareRegularFile (0.00s) === RUN TestSurvivors_DirectoryKeyMarkerCountsAsNull --- PASS: TestSurvivors_DirectoryKeyMarkerCountsAsNull (0.00s) === RUN TestSurvivors_PlainDirectoryDoesNotCount --- PASS: TestSurvivors_PlainDirectoryDoesNotCount (0.00s) === RUN TestSurvivors_VersionsListingErrorPropagates --- PASS: TestSurvivors_VersionsListingErrorPropagates (0.00s) === RUN TestSurvivors_NullLookupErrorPropagates --- PASS: TestSurvivors_NullLookupErrorPropagates (0.00s) === RUN TestListVersions_NotFoundReturnsNil --- PASS: TestListVersions_NotFoundReturnsNil (0.00s) === RUN TestListVersions_FiltersOutDirectoriesAndMissingVersionId --- PASS: TestListVersions_FiltersOutDirectoriesAndMissingVersionId (0.00s) === RUN TestListVersions_PaginatesAcrossPages --- PASS: TestListVersions_PaginatesAcrossPages (0.00s) === RUN TestListVersions_ErrorPropagates --- PASS: TestListVersions_ErrorPropagates (0.00s) === RUN TestLookupNullVersion_RegularFileExplicitFalse --- PASS: TestLookupNullVersion_RegularFileExplicitFalse (0.00s) === RUN TestLookupNullVersion_ExplicitNullFlag --- PASS: TestLookupNullVersion_ExplicitNullFlag (0.00s) === RUN TestLookupNullVersion_DirectoryKeyMarkerQualifies --- PASS: TestLookupNullVersion_DirectoryKeyMarkerQualifies (0.00s) === RUN TestLookupNullVersion_PlainDirectoryRejected --- PASS: TestLookupNullVersion_PlainDirectoryRejected (0.00s) === RUN TestLookupNullVersion_NotFoundCollapses --- PASS: TestLookupNullVersion_NotFoundCollapses (0.00s) === RUN TestLookupNullVersion_TransportErrorPropagates --- PASS: TestLookupNullVersion_TransportErrorPropagates (0.00s) === RUN TestLookupVersion_EmptyVersionIdReturnsNil --- PASS: TestLookupVersion_EmptyVersionIdReturnsNil (0.00s) === RUN TestLookupVersion_AppliesVPrefix --- PASS: TestLookupVersion_AppliesVPrefix (0.00s) === RUN TestLookupVersion_NotFoundCollapses --- PASS: TestLookupVersion_NotFoundCollapses (0.00s) === RUN TestLookupVersion_TransportErrorPropagates --- PASS: TestLookupVersion_TransportErrorPropagates (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/dispatcher 0.061s === RUN TestRulePredicateSensitive_NilRuleReturnsFalse --- PASS: TestRulePredicateSensitive_NilRuleReturnsFalse (0.00s) === RUN TestRulePredicateSensitive_NoFilterTagsReturnsFalse --- PASS: TestRulePredicateSensitive_NoFilterTagsReturnsFalse (0.00s) === RUN TestRulePredicateSensitive_EmptyFilterTagsReturnsFalse --- PASS: TestRulePredicateSensitive_EmptyFilterTagsReturnsFalse (0.00s) === RUN TestRulePredicateSensitive_PopulatedFilterTagsReturnsTrue --- PASS: TestRulePredicateSensitive_PopulatedFilterTagsReturnsTrue (0.00s) === RUN TestCompile_SingleRuleSingleAction --- PASS: TestCompile_SingleRuleSingleAction (0.00s) === RUN TestCompile_MultiAction_SiblingsHaveOwnEntries --- PASS: TestCompile_MultiAction_SiblingsHaveOwnEntries (0.00s) === RUN TestCompile_BootstrapPendingIndexedButInactive --- PASS: TestCompile_BootstrapPendingIndexedButInactive (0.00s) === RUN TestCompile_RetentionGate --- PASS: TestCompile_RetentionGate (0.00s) === RUN TestCompile_RetentionUnboundedNeverGates --- PASS: TestCompile_RetentionUnboundedNeverGates (0.00s) === RUN TestCompile_SiblingsDegradeIndependently --- PASS: TestCompile_SiblingsDegradeIndependently (0.00s) === RUN TestCompile_PriorModePreservedOverDecideMode --- PASS: TestCompile_PriorModePreservedOverDecideMode (0.00s) === RUN TestCompile_ExpirationDateScansAtDate --- PASS: TestCompile_ExpirationDateScansAtDate (0.00s) === RUN TestCompile_DisabledRuleNeverActivates --- PASS: TestCompile_DisabledRuleNeverActivates (0.00s) === RUN TestSnapshot_MarkActiveFlipsRoutingFilter --- PASS: TestSnapshot_MarkActiveFlipsRoutingFilter (0.00s) === RUN TestCompile_CrossBucketIdenticalRulesDoNotCollide --- PASS: TestCompile_CrossBucketIdenticalRulesDoNotCollide (0.00s) === RUN TestEngine_SnapshotAtomicSwap --- PASS: TestEngine_SnapshotAtomicSwap (0.00s) === RUN TestCompile_DelayGroupsDedupeAcrossBuckets --- PASS: TestCompile_DelayGroupsDedupeAcrossBuckets (0.00s) === RUN TestReplayContentHash_EmptyWhenNoReplayActions --- PASS: TestReplayContentHash_EmptyWhenNoReplayActions (0.00s) === RUN TestReplayContentHash_PartitionIndependent --- PASS: TestReplayContentHash_PartitionIndependent (0.00s) === RUN TestReplayContentHash_ChangesOnRuleContentEdit --- PASS: TestReplayContentHash_ChangesOnRuleContentEdit (0.00s) === RUN TestReplayContentHash_StableAcrossRuleReorder --- PASS: TestReplayContentHash_StableAcrossRuleReorder (0.00s) === RUN TestReplayContentHash_ExcludesWalkerOnlyAndDisabled --- PASS: TestReplayContentHash_ExcludesWalkerOnlyAndDisabled (0.00s) === RUN TestPromotedHash_EmptyWhenNoneAreInWalk --- PASS: TestPromotedHash_EmptyWhenNoneAreInWalk (0.00s) === RUN TestPromotedHash_ChangesOnReplayToWalkPromotion --- PASS: TestPromotedHash_ChangesOnReplayToWalkPromotion (0.00s) === RUN TestPromotedHash_ChangesOnWalkToReplayDemotion --- PASS: TestPromotedHash_ChangesOnWalkToReplayDemotion (0.00s) === RUN TestPromotedHash_StableWhenContentUnchangedAndPartitionStays --- PASS: TestPromotedHash_StableWhenContentUnchangedAndPartitionStays (0.00s) === RUN TestPromotedHash_MatchesRulesForShardWalkMembership --- PASS: TestPromotedHash_MatchesRulesForShardWalkMembership (0.00s) === RUN TestMaxEffectiveTTL_NilAndEmpty --- PASS: TestMaxEffectiveTTL_NilAndEmpty (0.00s) === RUN TestMaxEffectiveTTL_ReturnsMaxAcrossActiveReplay --- PASS: TestMaxEffectiveTTL_ReturnsMaxAcrossActiveReplay (0.00s) === RUN TestMaxEffectiveTTL_OperatesOnReplayView --- PASS: TestMaxEffectiveTTL_OperatesOnReplayView (0.00s) === RUN TestMaxEffectiveTTL_IgnoresInactiveActions --- PASS: TestMaxEffectiveTTL_IgnoresInactiveActions (0.00s) === RUN TestPrefixMatches_EmptyPrefixAllowsEverything --- PASS: TestPrefixMatches_EmptyPrefixAllowsEverything (0.00s) === RUN TestPrefixMatches_HonorsExactPrefix --- PASS: TestPrefixMatches_HonorsExactPrefix (0.00s) === RUN TestPrefixMatches_RejectsNonMatchingPath --- PASS: TestPrefixMatches_RejectsNonMatchingPath (0.00s) === RUN TestPrefixMatches_PathShorterThanPrefixIsReject --- PASS: TestPrefixMatches_PathShorterThanPrefixIsReject (0.00s) === RUN TestFilterAllows_NoFiltersAcceptsAnything --- PASS: TestFilterAllows_NoFiltersAcceptsAnything (0.00s) === RUN TestFilterAllows_SizeGreaterThanGate --- PASS: TestFilterAllows_SizeGreaterThanGate (0.00s) === RUN TestFilterAllows_SizeLessThanGate --- PASS: TestFilterAllows_SizeLessThanGate (0.00s) === RUN TestFilterAllows_ZeroSizeFilterIsDisabled --- PASS: TestFilterAllows_ZeroSizeFilterIsDisabled (0.00s) === RUN TestFilterAllows_RequiredTagPresent --- PASS: TestFilterAllows_RequiredTagPresent (0.00s) === RUN TestFilterAllows_RequiredTagMissingOrWrong --- PASS: TestFilterAllows_RequiredTagMissingOrWrong (0.00s) === RUN TestFilterAllows_AllRequiredTagsMustMatch --- PASS: TestFilterAllows_AllRequiredTagsMustMatch (0.00s) === RUN TestFilterAllows_SizeAndTagAreANDed --- PASS: TestFilterAllows_SizeAndTagAreANDed (0.00s) === RUN TestMatchOriginalWrite_NilEventReturnsNil --- PASS: TestMatchOriginalWrite_NilEventReturnsNil (0.00s) === RUN TestMatchOriginalWrite_WrongShapeReturnsNil --- PASS: TestMatchOriginalWrite_WrongShapeReturnsNil (0.00s) === RUN TestMatchOriginalWrite_DelayMismatchReturnsNil --- PASS: TestMatchOriginalWrite_DelayMismatchReturnsNil (0.00s) === RUN TestMatchPredicateChange_NilEventReturnsNil --- PASS: TestMatchPredicateChange_NilEventReturnsNil (0.00s) === RUN TestMatchPredicateChange_WrongShapeReturnsNil --- PASS: TestMatchPredicateChange_WrongShapeReturnsNil (0.00s) === RUN TestMatchPredicateChange_NoTagSensitiveActionsReturnsNil --- PASS: TestMatchPredicateChange_NoTagSensitiveActionsReturnsNil (0.00s) === RUN TestMatchPath_UnknownBucketReturnsNil --- PASS: TestMatchPath_UnknownBucketReturnsNil (0.00s) === RUN TestMatchPath_NilEventStillAppliesPrefix --- PASS: TestMatchPath_NilEventStillAppliesPrefix (0.00s) === RUN TestMatchPath_FilterRespectedWhenEventProvided --- PASS: TestMatchPath_FilterRespectedWhenEventProvided (0.00s) === RUN TestFilterMatching_BucketScoping --- PASS: TestFilterMatching_BucketScoping (0.00s) === RUN TestFilterMatching_AbortMPURequiresMPUInit --- PASS: TestFilterMatching_AbortMPURequiresMPUInit (0.00s) === RUN TestFilterMatching_ExpiredDeleteMarkerRequiresMarkerLatest --- PASS: TestFilterMatching_ExpiredDeleteMarkerRequiresMarkerLatest (0.00s) === RUN TestMatchOriginalWrite_DelayGroupRoutes --- PASS: TestMatchOriginalWrite_DelayGroupRoutes (0.00s) === RUN TestMatchOriginalWrite_PrefixFilter --- PASS: TestMatchOriginalWrite_PrefixFilter (0.00s) === RUN TestMatchOriginalWrite_MarkActiveBecomesRoutable --- PASS: TestMatchOriginalWrite_MarkActiveBecomesRoutable (0.00s) === RUN TestMatchOriginalWrite_AbortMPUOnlyOnMPUInit --- PASS: TestMatchOriginalWrite_AbortMPUOnlyOnMPUInit (0.00s) === RUN TestMatchPredicateChange_OnlyTagSensitiveActions --- PASS: TestMatchPredicateChange_OnlyTagSensitiveActions (0.00s) === RUN TestMatchPath_BootstrapWalkerSeesAllActiveActions --- PASS: TestMatchPath_BootstrapWalkerSeesAllActiveActions (0.00s) === RUN TestMatchPath_PrefixMismatchExcludes --- PASS: TestMatchPath_PrefixMismatchExcludes (0.00s) === RUN TestMatchPath_FilterRejectsByTag --- PASS: TestMatchPath_FilterRejectsByTag (0.00s) === RUN TestDecideMode_NilRuleIsDisabled --- PASS: TestDecideMode_NilRuleIsDisabled (0.00s) === RUN TestDecideMode_DisabledStatusIsDisabled --- PASS: TestDecideMode_DisabledStatusIsDisabled (0.00s) === RUN TestDecideMode_ExpirationDateIsScanAtDate --- PASS: TestDecideMode_ExpirationDateIsScanAtDate (0.00s) === RUN TestDecideMode_UnboundedRetentionIsEventDriven --- PASS: TestDecideMode_UnboundedRetentionIsEventDriven (0.00s) === RUN TestDecideMode_HorizonWithinRetentionIsEventDriven --- PASS: TestDecideMode_HorizonWithinRetentionIsEventDriven (0.00s) === RUN TestDecideMode_HorizonExceedsRetentionIsScanOnly --- PASS: TestDecideMode_HorizonExceedsRetentionIsScanOnly (0.00s) === RUN TestDecideMode_LookbackPushesAcrossThreshold --- PASS: TestDecideMode_LookbackPushesAcrossThreshold (0.00s) === RUN TestDecideMode_ZeroHorizonNeverGates --- PASS: TestDecideMode_ZeroHorizonNeverGates (0.00s) === RUN TestRuleMode_StringRendersDocumentedNames === RUN TestRuleMode_StringRendersDocumentedNames/unspecified === RUN TestRuleMode_StringRendersDocumentedNames/event_driven === RUN TestRuleMode_StringRendersDocumentedNames/scan_at_date === RUN TestRuleMode_StringRendersDocumentedNames/scan_only === RUN TestRuleMode_StringRendersDocumentedNames/disabled === RUN TestRuleMode_StringRendersDocumentedNames/pending_bootstrap --- PASS: TestRuleMode_StringRendersDocumentedNames (0.00s) --- PASS: TestRuleMode_StringRendersDocumentedNames/unspecified (0.00s) --- PASS: TestRuleMode_StringRendersDocumentedNames/event_driven (0.00s) --- PASS: TestRuleMode_StringRendersDocumentedNames/scan_at_date (0.00s) --- PASS: TestRuleMode_StringRendersDocumentedNames/scan_only (0.00s) --- PASS: TestRuleMode_StringRendersDocumentedNames/disabled (0.00s) --- PASS: TestRuleMode_StringRendersDocumentedNames/pending_bootstrap (0.00s) === RUN TestRuleMode_StringFallsToUnspecifiedForUnknown --- PASS: TestRuleMode_StringFallsToUnspecifiedForUnknown (0.00s) === RUN TestSnapshot_OriginalDelayGroupsExposesCompiledGroups --- PASS: TestSnapshot_OriginalDelayGroupsExposesCompiledGroups (0.00s) === RUN TestSnapshot_OriginalDelayGroupsScanOnlyExcluded --- PASS: TestSnapshot_OriginalDelayGroupsScanOnlyExcluded (0.00s) === RUN TestSnapshot_PredicateActionsContainsTagSensitive --- PASS: TestSnapshot_PredicateActionsContainsTagSensitive (0.00s) === RUN TestSnapshot_PredicateActionsEmptyForNonTagSensitiveRule --- PASS: TestSnapshot_PredicateActionsEmptyForNonTagSensitiveRule (0.00s) === RUN TestSnapshot_DateActionsContainsExpirationDate --- PASS: TestSnapshot_DateActionsContainsExpirationDate (0.00s) === RUN TestSnapshot_DateActionsEmptyForNonDateRule --- PASS: TestSnapshot_DateActionsEmptyForNonDateRule (0.00s) === RUN TestSnapshot_MarkActiveUnknownKeyIsNoOp --- PASS: TestSnapshot_MarkActiveUnknownKeyIsNoOp (0.00s) === RUN TestSnapshot_MarkActiveFlipsCompiledActionToActive --- PASS: TestSnapshot_MarkActiveFlipsCompiledActionToActive (0.00s) === RUN TestSnapshot_BucketIndexedActionKeysCoverAllKinds --- PASS: TestSnapshot_BucketIndexedActionKeysCoverAllKinds (0.00s) === RUN TestCurrentSnapshot_NilWhenNoEngineRegistered --- PASS: TestCurrentSnapshot_NilWhenNoEngineRegistered (0.00s) === RUN TestCurrentSnapshot_ReturnsRegisteredEngineSnapshot --- PASS: TestCurrentSnapshot_ReturnsRegisteredEngineSnapshot (0.00s) === RUN TestRulesForShard_NilSnapshot --- PASS: TestRulesForShard_NilSnapshot (0.00s) === RUN TestRulesForShard_ReplayPartitionMembership --- PASS: TestRulesForShard_ReplayPartitionMembership (0.00s) === RUN TestRulesForShard_WalkOnlyKindsRouteToWalk --- PASS: TestRulesForShard_WalkOnlyKindsRouteToWalk (0.00s) === RUN TestRulesForShard_MultiActionXMLRuleSplits --- PASS: TestRulesForShard_MultiActionXMLRuleSplits (0.00s) === RUN TestRulesForShard_ScanOnlyPromotionRoutesToWalk --- PASS: TestRulesForShard_ScanOnlyPromotionRoutesToWalk (0.00s) === RUN TestRulesForShard_ReplayRewritesScanOnlyMode --- PASS: TestRulesForShard_ReplayRewritesScanOnlyMode (0.00s) === RUN TestRulesForShard_DisabledRuleExcludedFromBothViews --- PASS: TestRulesForShard_DisabledRuleExcludedFromBothViews (0.00s) === RUN TestRulesForShard_ClonesAreIndependentOfBase --- PASS: TestRulesForShard_ClonesAreIndependentOfBase (0.00s) === RUN TestRulesForShard_ZeroRetentionRoutesAllReplayKindsToWalk --- PASS: TestRulesForShard_ZeroRetentionRoutesAllReplayKindsToWalk (0.00s) === RUN TestRecoveryView_ActivatesEveryNonDisabledAction --- PASS: TestRecoveryView_ActivatesEveryNonDisabledAction (0.00s) === RUN TestRecoveryView_PreservesBaseModeForScanAtDate --- PASS: TestRecoveryView_PreservesBaseModeForScanAtDate (0.00s) === RUN TestRecoveryView_ExcludesDisabledRules --- PASS: TestRecoveryView_ExcludesDisabledRules (0.00s) === RUN TestRecoveryView_ClonesAreIndependentOfBase --- PASS: TestRecoveryView_ClonesAreIndependentOfBase (0.00s) === RUN TestRecoveryView_NilSnapshot --- PASS: TestRecoveryView_NilSnapshot (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/engine 0.045s === RUN TestEventOption_WithModTime_AppliesToOldEntryOnDelete --- PASS: TestEventOption_WithModTime_AppliesToOldEntryOnDelete (0.00s) === RUN TestEventOption_WithTtlSec_AppliesToOldEntryOnDelete --- PASS: TestEventOption_WithTtlSec_AppliesToOldEntryOnDelete (0.00s) === RUN TestEventOption_WithVersionID_AppliesToOldEntryOnDelete --- PASS: TestEventOption_WithVersionID_AppliesToOldEntryOnDelete (0.00s) === RUN TestEventOption_WithExtended_AppliesToOldEntryOnDelete --- PASS: TestEventOption_WithExtended_AppliesToOldEntryOnDelete (0.00s) === RUN TestEventOption_WithChunks_AppliesToOldEntryOnDelete --- PASS: TestEventOption_WithChunks_AppliesToOldEntryOnDelete (0.00s) === RUN TestEventOption_WithBootstrapVersion --- PASS: TestEventOption_WithBootstrapVersion (0.00s) === RUN TestEventOption_NoPanicOnEmptyEvent --- PASS: TestEventOption_NoPanicOnEmptyEvent (0.00s) === RUN TestNewCreate_PopulatesNewEntryOnly --- PASS: TestNewCreate_PopulatesNewEntryOnly (0.00s) === RUN TestNewDelete_PopulatesOldEntryOnly --- PASS: TestNewDelete_PopulatesOldEntryOnly (0.00s) === RUN TestNewUpdate_PopulatesBothEntries --- PASS: TestNewUpdate_PopulatesBothEntries (0.00s) === RUN TestNewCreate_NestedKeyUsesLeafName --- PASS: TestNewCreate_NestedKeyUsesLeafName (0.00s) === RUN TestNewCreate_DirectoryKeyUsesSlashlessLeaf --- PASS: TestNewCreate_DirectoryKeyUsesSlashlessLeaf (0.00s) === RUN TestNewCreate_ShardIDDerivedFromKey --- PASS: TestNewCreate_ShardIDDerivedFromKey (0.00s) === RUN TestEventOption_WithSize --- PASS: TestEventOption_WithSize (0.00s) === RUN TestEventOption_WithSizeAppliesToOldEntryOnDelete --- PASS: TestEventOption_WithSizeAppliesToOldEntryOnDelete (0.00s) === RUN TestEventOption_WithModTimeOverridesDefault --- PASS: TestEventOption_WithModTimeOverridesDefault (0.00s) === RUN TestEventOption_WithTtlSec --- PASS: TestEventOption_WithTtlSec (0.00s) === RUN TestEventOption_WithVersionID --- PASS: TestEventOption_WithVersionID (0.00s) === RUN TestEventOption_WithExtendedKeyValue --- PASS: TestEventOption_WithExtendedKeyValue (0.00s) === RUN TestEventOption_WithChunks --- PASS: TestEventOption_WithChunks (0.00s) === RUN TestEventOption_WithShardIDOverrides --- PASS: TestEventOption_WithShardIDOverrides (0.00s) === RUN TestEventOption_WithOldSizeTargetsOldEntryOnUpdate --- PASS: TestEventOption_WithOldSizeTargetsOldEntryOnUpdate (0.00s) === RUN TestEventOption_WithOldChunksTargetsOldEntryOnUpdate --- PASS: TestEventOption_WithOldChunksTargetsOldEntryOnUpdate (0.00s) === RUN TestEventOption_WithOldModTimeTargetsOldEntryOnUpdate --- PASS: TestEventOption_WithOldModTimeTargetsOldEntryOnUpdate (0.00s) === RUN TestEventOption_WithOldOptionsAreNoOpsOnCreate --- PASS: TestEventOption_WithOldOptionsAreNoOpsOnCreate (0.00s) === RUN TestEventOption_LaterOverridesEarlier --- PASS: TestEventOption_LaterOverridesEarlier (0.00s) === RUN TestMetaLogClock_DefaultStepIsOneSecond --- PASS: TestMetaLogClock_DefaultStepIsOneSecond (0.00s) === RUN TestMetaLogClock_CustomStep --- PASS: TestMetaLogClock_CustomStep (0.00s) === RUN TestMetaLogClock_PeekDoesNotAdvance --- PASS: TestMetaLogClock_PeekDoesNotAdvance (0.00s) === RUN TestMetaLogClock_ConcurrentNextNoRace --- PASS: TestMetaLogClock_ConcurrentNextNoRace (0.00s) === RUN TestFake_DefaultIsDoneOutOfTheBox --- PASS: TestFake_DefaultIsDoneOutOfTheBox (0.00s) === RUN TestFake_QueuedOutcomesPopFIFO --- PASS: TestFake_QueuedOutcomesPopFIFO (0.00s) === RUN TestFake_QueuesIsolatedByKey --- PASS: TestFake_QueuesIsolatedByKey (0.00s) === RUN TestFake_VersionIDPartOfKey --- PASS: TestFake_VersionIDPartOfKey (0.00s) === RUN TestFake_KeyComponentsWithDelimitersDoNotCollide --- PASS: TestFake_KeyComponentsWithDelimitersDoNotCollide (0.00s) === RUN TestFake_ErrShortCircuitsBeforeRecording --- PASS: TestFake_ErrShortCircuitsBeforeRecording (0.00s) === RUN TestFake_RecordsRequestsInOrder --- PASS: TestFake_RecordsRequestsInOrder (0.00s) === RUN TestFake_RecordedIsSnapshot --- PASS: TestFake_RecordedIsSnapshot (0.00s) === RUN TestFake_RecordedRequestsAreDeepCopies --- PASS: TestFake_RecordedRequestsAreDeepCopies (0.00s) === RUN TestFake_NilRequestUsesDefault --- PASS: TestFake_NilRequestUsesDefault (0.00s) === RUN TestFake_ConcurrentCallsSerializeWithoutDeadlock --- PASS: TestFake_ConcurrentCallsSerializeWithoutDeadlock (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/lifecycletest 0.039s === RUN TestCursorMinTsNsWithFrozenKeysIncluded --- PASS: TestCursorMinTsNsWithFrozenKeysIncluded (0.01s) === RUN TestCursorSnapshotIsDeepCopy --- PASS: TestCursorSnapshotIsDeepCopy (0.00s) === RUN TestCursorRestoreReplacesNotMerges --- PASS: TestCursorRestoreReplacesNotMerges (0.00s) === RUN TestReaderRunValidatesInputsBeforeSubscribing === RUN TestReaderRunValidatesInputsBeforeSubscribing/shard_id_below_zero === RUN TestReaderRunValidatesInputsBeforeSubscribing/shard_id_at_or_above_ShardCount === RUN TestReaderRunValidatesInputsBeforeSubscribing/nil_events_channel === RUN TestReaderRunValidatesInputsBeforeSubscribing/empty_buckets_path --- PASS: TestReaderRunValidatesInputsBeforeSubscribing (0.00s) --- PASS: TestReaderRunValidatesInputsBeforeSubscribing/shard_id_below_zero (0.00s) --- PASS: TestReaderRunValidatesInputsBeforeSubscribing/shard_id_at_or_above_ShardCount (0.00s) --- PASS: TestReaderRunValidatesInputsBeforeSubscribing/nil_events_channel (0.00s) --- PASS: TestReaderRunValidatesInputsBeforeSubscribing/empty_buckets_path (0.00s) === RUN TestCursorMinTsNsEmpty --- PASS: TestCursorMinTsNsEmpty (0.00s) === RUN TestCursorAdvanceMonotonic --- PASS: TestCursorAdvanceMonotonic (0.00s) === RUN TestCursorAdvanceIgnoresZero --- PASS: TestCursorAdvanceIgnoresZero (0.00s) === RUN TestCursorMinTsNsAcrossKeys --- PASS: TestCursorMinTsNsAcrossKeys (0.00s) === RUN TestCursorFreeze --- PASS: TestCursorFreeze (0.00s) === RUN TestCursorFreezeOnUnsetKeySeedsPosition --- PASS: TestCursorFreezeOnUnsetKeySeedsPosition (0.00s) === RUN TestCursorSnapshotRestore --- PASS: TestCursorSnapshotRestore (0.00s) === RUN TestEventIsCreate_PopulatedNewEntryNoOldEntry --- PASS: TestEventIsCreate_PopulatedNewEntryNoOldEntry (0.00s) === RUN TestEventIsDelete_PopulatedOldEntryNoNewEntry --- PASS: TestEventIsDelete_PopulatedOldEntryNoNewEntry (0.00s) === RUN TestEventBothEntries_NeitherCreateNorDelete --- PASS: TestEventBothEntries_NeitherCreateNorDelete (0.00s) === RUN TestEventNoEntries_NeitherCreateNorDelete --- PASS: TestEventNoEntries_NeitherCreateNorDelete (0.00s) === RUN TestLogStartup_ShardIDOnly --- PASS: TestLogStartup_ShardIDOnly (0.00s) === RUN TestLogStartup_ShardPredicate --- PASS: TestLogStartup_ShardPredicate (0.00s) === RUN TestLogStartup_StartTsNsOverridesCursor --- PASS: TestLogStartup_StartTsNsOverridesCursor (0.00s) === RUN TestLogStartup_CursorMinFallback --- PASS: TestLogStartup_CursorMinFallback (0.00s) === RUN TestInMemoryPersister_LoadOnUnknownShardReturnsEmptyMap --- PASS: TestInMemoryPersister_LoadOnUnknownShardReturnsEmptyMap (0.00s) === RUN TestInMemoryPersister_SaveLoadRoundTrip --- PASS: TestInMemoryPersister_SaveLoadRoundTrip (0.00s) === RUN TestInMemoryPersister_SaveCopiesInput --- PASS: TestInMemoryPersister_SaveCopiesInput (0.00s) === RUN TestInMemoryPersister_LoadReturnsACopy --- PASS: TestInMemoryPersister_LoadReturnsACopy (0.00s) === RUN TestInMemoryPersister_SaveReplacesNotMerges --- PASS: TestInMemoryPersister_SaveReplacesNotMerges (0.00s) === RUN TestInMemoryPersister_DifferentShardsIsolated --- PASS: TestInMemoryPersister_DifferentShardsIsolated (0.00s) === RUN TestInMemoryPersister_SaveEmptyMapClearsState --- PASS: TestInMemoryPersister_SaveEmptyMapClearsState (0.00s) === RUN TestInMemoryPersister_ConcurrentSaveLoadDoesNotRace --- PASS: TestInMemoryPersister_ConcurrentSaveLoadDoesNotRace (0.00s) === RUN TestExtractBucketKeyCreate --- PASS: TestExtractBucketKeyCreate (0.00s) === RUN TestExtractBucketKeyTopLevelObject --- PASS: TestExtractBucketKeyTopLevelObject (0.00s) === RUN TestExtractBucketKeyDeleteUsesOldEntry --- PASS: TestExtractBucketKeyDeleteUsesOldEntry (0.00s) === RUN TestExtractBucketKeyDeleteWithEmptyParent --- PASS: TestExtractBucketKeyDeleteWithEmptyParent (0.00s) === RUN TestExtractBucketKeyOutsideBucketsSkipped --- PASS: TestExtractBucketKeyOutsideBucketsSkipped (0.00s) === RUN TestExtractBucketKeyBucketCreateAtRoot --- PASS: TestExtractBucketKeyBucketCreateAtRoot (0.00s) === RUN TestExtractBucketKeyBucketCreateBareRoot --- PASS: TestExtractBucketKeyBucketCreateBareRoot (0.00s) === RUN TestDispatchOneFiltersByShard --- PASS: TestDispatchOneFiltersByShard (0.00s) === RUN TestDispatchOneRespectsCtxCancel --- PASS: TestDispatchOneRespectsCtxCancel (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/reader 0.037s === RUN TestSuccessorModTimeFromContainer_MissingExtReturnsZero --- PASS: TestSuccessorModTimeFromContainer_MissingExtReturnsZero (0.00s) === RUN TestSuccessorModTimeFromContainer_EmptyValueReturnsZero --- PASS: TestSuccessorModTimeFromContainer_EmptyValueReturnsZero (0.00s) === RUN TestSuccessorModTimeFromContainer_NonNumericReturnsZero --- PASS: TestSuccessorModTimeFromContainer_NonNumericReturnsZero (0.00s) === RUN TestSuccessorModTimeFromContainer_NonPositiveReturnsZero === RUN TestSuccessorModTimeFromContainer_NonPositiveReturnsZero/0 === RUN TestSuccessorModTimeFromContainer_NonPositiveReturnsZero/-1 === RUN TestSuccessorModTimeFromContainer_NonPositiveReturnsZero/-1000 --- PASS: TestSuccessorModTimeFromContainer_NonPositiveReturnsZero (0.00s) --- PASS: TestSuccessorModTimeFromContainer_NonPositiveReturnsZero/0 (0.00s) --- PASS: TestSuccessorModTimeFromContainer_NonPositiveReturnsZero/-1 (0.00s) --- PASS: TestSuccessorModTimeFromContainer_NonPositiveReturnsZero/-1000 (0.00s) === RUN TestSuccessorModTimeFromContainer_PositiveSecondsRoundTrip --- PASS: TestSuccessorModTimeFromContainer_PositiveSecondsRoundTrip (0.00s) === RUN TestLogicalKeyFromVersionPath_ExtractsLogicalKey --- PASS: TestLogicalKeyFromVersionPath_ExtractsLogicalKey (0.00s) === RUN TestLogicalKeyFromVersionPath_RejectsPathWithoutVersionsParent --- PASS: TestLogicalKeyFromVersionPath_RejectsPathWithoutVersionsParent (0.00s) === RUN TestLogicalKeyFromVersionPath_RejectsRootLevelPath --- PASS: TestLogicalKeyFromVersionPath_RejectsRootLevelPath (0.00s) === RUN TestLogicalKeyFromVersionPath_RejectsNoSlashes --- PASS: TestLogicalKeyFromVersionPath_RejectsNoSlashes (0.00s) === RUN TestLogicalKeyFromVersionPath_RejectsBareVersionsContainer --- PASS: TestLogicalKeyFromVersionPath_RejectsBareVersionsContainer (0.00s) === RUN TestIsVersionsContainerKey === RUN TestIsVersionsContainerKey/obj.versions === RUN TestIsVersionsContainerKey/a/b/obj.versions === RUN TestIsVersionsContainerKey/.versions === RUN TestIsVersionsContainerKey/obj.versions/v_x === RUN TestIsVersionsContainerKey/obj === RUN TestIsVersionsContainerKey/#00 --- PASS: TestIsVersionsContainerKey (0.00s) --- PASS: TestIsVersionsContainerKey/obj.versions (0.00s) --- PASS: TestIsVersionsContainerKey/a/b/obj.versions (0.00s) --- PASS: TestIsVersionsContainerKey/.versions (0.00s) --- PASS: TestIsVersionsContainerKey/obj.versions/v_x (0.00s) --- PASS: TestIsVersionsContainerKey/obj (0.00s) --- PASS: TestIsVersionsContainerKey/#00 (0.00s) === RUN TestIsVersionFolderPath === RUN TestIsVersionFolderPath/obj.versions/v_aaa === RUN TestIsVersionFolderPath/a/b/obj.versions/v_aaa === RUN TestIsVersionFolderPath/obj/v_aaa === RUN TestIsVersionFolderPath/obj.versions === RUN TestIsVersionFolderPath/obj.versions/ === RUN TestIsVersionFolderPath/v_aaa === RUN TestIsVersionFolderPath/#00 --- PASS: TestIsVersionFolderPath (0.00s) --- PASS: TestIsVersionFolderPath/obj.versions/v_aaa (0.00s) --- PASS: TestIsVersionFolderPath/a/b/obj.versions/v_aaa (0.00s) --- PASS: TestIsVersionFolderPath/obj/v_aaa (0.00s) --- PASS: TestIsVersionFolderPath/obj.versions (0.00s) --- PASS: TestIsVersionFolderPath/obj.versions/ (0.00s) --- PASS: TestIsVersionFolderPath/v_aaa (0.00s) --- PASS: TestIsVersionFolderPath/#00 (0.00s) === RUN TestIsDeleteMarkerEntry_NilOrEmptyExtendedReturnsFalse --- PASS: TestIsDeleteMarkerEntry_NilOrEmptyExtendedReturnsFalse (0.00s) === RUN TestIsDeleteMarkerEntry_TrueOnlyForLiteralTrue --- PASS: TestIsDeleteMarkerEntry_TrueOnlyForLiteralTrue (0.00s) === RUN TestExtractTags_NilOrEmptyReturnsNil --- PASS: TestExtractTags_NilOrEmptyReturnsNil (0.00s) === RUN TestExtractTags_OnlyKeysWithObjectTaggingPrefix --- PASS: TestExtractTags_OnlyKeysWithObjectTaggingPrefix (0.00s) === RUN TestExtractTags_ReturnsNilWhenNoTaggingPrefixedKeys --- PASS: TestExtractTags_ReturnsNilWhenNoTaggingPrefixedKeys (0.00s) === RUN TestHasActiveEventDrivenAction --- PASS: TestHasActiveEventDrivenAction (0.00s) === RUN TestHasActiveEventDrivenAction_NilActionSkipped --- PASS: TestHasActiveEventDrivenAction_NilActionSkipped (0.00s) === RUN TestSnapshot_BucketVersionedReportsCompiledFlag --- PASS: TestSnapshot_BucketVersionedReportsCompiledFlag (0.00s) === RUN TestSnapshot_BucketActionKeysReturnsCompiledList --- PASS: TestSnapshot_BucketActionKeysReturnsCompiledList (0.00s) === RUN TestSnapshot_BucketActionKeysUnknownBucketReturnsNil --- PASS: TestSnapshot_BucketActionKeysUnknownBucketReturnsNil (0.00s) === RUN TestSnapshot_ActionUnknownKeyReturnsNil --- PASS: TestSnapshot_ActionUnknownKeyReturnsNil (0.00s) === RUN TestSnapshot_AllActionsCoversEveryCompiledKind --- PASS: TestSnapshot_AllActionsCoversEveryCompiledKind (0.00s) === RUN TestSnapshot_SnapshotIDIsMonotonicAcrossRecompiles --- PASS: TestSnapshot_SnapshotIDIsMonotonicAcrossRecompiles (0.00s) === RUN TestRouteNoSnapshotNoMatches --- PASS: TestRouteNoSnapshotNoMatches (0.00s) === RUN TestRouteMissingBucketNoMatches --- PASS: TestRouteMissingBucketNoMatches (0.00s) === RUN TestRouteInactiveSkipped --- PASS: TestRouteInactiveSkipped (0.00s) === RUN TestRouteExpirationDaysFires --- PASS: TestRouteExpirationDaysFires (0.00s) === RUN TestRouteFreshObjectSchedulesInFuture --- PASS: TestRouteFreshObjectSchedulesInFuture (0.00s) === RUN TestRouteRespectsPrefixFilter --- PASS: TestRouteRespectsPrefixFilter (0.00s) === RUN TestRouteSkipsHardDelete --- PASS: TestRouteSkipsHardDelete (0.00s) === RUN TestRouteSkipsMissingAttributes --- PASS: TestRouteSkipsMissingAttributes (0.00s) === RUN TestRouteIdentityCapturedForNewEntry --- PASS: TestRouteIdentityCapturedForNewEntry (0.00s) === RUN TestRouteIdentityHashesExtended --- PASS: TestRouteIdentityHashesExtended (0.00s) === RUN TestRouteMPUInitFiresAbortAfterDelay --- PASS: TestRouteMPUInitFiresAbortAfterDelay (0.00s) === RUN TestRouteMPUInitFilteredOutByPrefix --- PASS: TestRouteMPUInitFilteredOutByPrefix (0.00s) === RUN TestRouteMPUInitMissingDestKeySkipped --- PASS: TestRouteMPUInitMissingDestKeySkipped (0.00s) === RUN TestRouteMPUPartEventSkipped --- PASS: TestRouteMPUPartEventSkipped (0.00s) === RUN TestRouteMPUInitDoesNotFireNoncurrent --- PASS: TestRouteMPUInitDoesNotFireNoncurrent (0.00s) === RUN TestRouteRegularObjectUnderDualRuleSkipsAbortMPU --- PASS: TestRouteRegularObjectUnderDualRuleSkipsAbortMPU (0.00s) === RUN TestRouteVersionedNoncurrentEventDoesNotFireFromRouter --- PASS: TestRouteVersionedNoncurrentEventDoesNotFireFromRouter (0.00s) === RUN TestRouteVersionedCurrentEventStaysLatest --- PASS: TestRouteVersionedCurrentEventStaysLatest (0.00s) === RUN TestRouteNonVersionedBucketIgnoresVersionsSuffix --- PASS: TestRouteNonVersionedBucketIgnoresVersionsSuffix (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerNilListerSuppresses --- PASS: TestRouteVersionedExpiredDeleteMarkerNilListerSuppresses (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerSoleSurvivorFires --- PASS: TestRouteVersionedExpiredDeleteMarkerSoleSurvivorFires (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerSiblingsRemainSuppressed --- PASS: TestRouteVersionedExpiredDeleteMarkerSiblingsRemainSuppressed (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerNullVersionSuppresses --- PASS: TestRouteVersionedExpiredDeleteMarkerNullVersionSuppresses (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerHardDeleteLeavesLoneMarkerFires --- PASS: TestRouteVersionedExpiredDeleteMarkerHardDeleteLeavesLoneMarkerFires (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerHardDeleteLoneNonMarkerNoFire --- PASS: TestRouteVersionedExpiredDeleteMarkerHardDeleteLoneNonMarkerNoFire (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerListerErrorSuppressed --- PASS: TestRouteVersionedExpiredDeleteMarkerListerErrorSuppressed (0.00s) === RUN TestRouteVersionedExpiredDeleteMarkerInactiveActionSkipsListing --- PASS: TestRouteVersionedExpiredDeleteMarkerInactiveActionSkipsListing (0.00s) === RUN TestRouteVersionedRegularVersionCreateSkipsListing --- PASS: TestRouteVersionedRegularVersionCreateSkipsListing (0.00s) === RUN TestRouteVersionedDeleteMarkerNoExpDMRuleSkipsListing --- PASS: TestRouteVersionedDeleteMarkerNoExpDMRuleSkipsListing (0.00s) === RUN TestRouteVersionedAllVersionFolderPathsSkipped === RUN TestRouteVersionedAllVersionFolderPathsSkipped/tracked_version_file === RUN TestRouteVersionedAllVersionFolderPathsSkipped/literal-key_collision === RUN TestRouteVersionedAllVersionFolderPathsSkipped/bucket-root_.versions === RUN TestRouteVersionedAllVersionFolderPathsSkipped/versions_dir_itself --- PASS: TestRouteVersionedAllVersionFolderPathsSkipped (0.00s) --- PASS: TestRouteVersionedAllVersionFolderPathsSkipped/tracked_version_file (0.00s) --- PASS: TestRouteVersionedAllVersionFolderPathsSkipped/literal-key_collision (0.00s) --- PASS: TestRouteVersionedAllVersionFolderPathsSkipped/bucket-root_.versions (0.00s) --- PASS: TestRouteVersionedAllVersionFolderPathsSkipped/versions_dir_itself (0.00s) === RUN TestRouteBootstrapVersionLatestExpirationDaysFires --- PASS: TestRouteBootstrapVersionLatestExpirationDaysFires (0.00s) === RUN TestRouteBootstrapVersionNoncurrentDaysFires --- PASS: TestRouteBootstrapVersionNoncurrentDaysFires (0.00s) === RUN TestRouteBootstrapVersionNoncurrentRespectsNewerNoncurrentVersions --- PASS: TestRouteBootstrapVersionNoncurrentRespectsNewerNoncurrentVersions (0.00s) === RUN TestRouteBootstrapVersionAbortMPUNeverEmittedForVersion --- PASS: TestRouteBootstrapVersionAbortMPUNeverEmittedForVersion (0.00s) === RUN TestRoutePointerTransitionFiresNoncurrentDays --- PASS: TestRoutePointerTransitionFiresNoncurrentDays (0.00s) === RUN TestRoutePointerTransitionPointerUnchangedSkipped --- PASS: TestRoutePointerTransitionPointerUnchangedSkipped (0.00s) === RUN TestRoutePointerTransitionEmptyOldPointerNoNullSkipped --- PASS: TestRoutePointerTransitionEmptyOldPointerNoNullSkipped (0.00s) === RUN TestRoutePointerTransitionEmptyOldPointerWithNullSchedules --- PASS: TestRoutePointerTransitionEmptyOldPointerWithNullSchedules (0.00s) === RUN TestRoutePointerTransitionExpansionIncludesNullVersion --- PASS: TestRoutePointerTransitionExpansionIncludesNullVersion (0.00s) === RUN TestRoutePointerTransitionDisplacedVersionMissingSuppressed --- PASS: TestRoutePointerTransitionDisplacedVersionMissingSuppressed (0.00s) === RUN TestRoutePointerTransitionNoNoncurrentRuleSkipsLookup --- PASS: TestRoutePointerTransitionNoNoncurrentRuleSkipsLookup (0.00s) === RUN TestRoutePointerTransitionNewerNoncurrentNewestNoncurrentRetained --- PASS: TestRoutePointerTransitionNewerNoncurrentNewestNoncurrentRetained (0.00s) === RUN TestRoutePointerTransitionExpansionMissingNewIDSuppressed --- PASS: TestRoutePointerTransitionExpansionMissingNewIDSuppressed (0.00s) === RUN TestRoutePointerTransitionUnversionedBucketSkipped --- PASS: TestRoutePointerTransitionUnversionedBucketSkipped (0.00s) === RUN TestRoutePointerTransitionMissingCachedMtimeSuppressed --- PASS: TestRoutePointerTransitionMissingCachedMtimeSuppressed (0.00s) === RUN TestRoutePointerTransitionUsesCachedMtimeNotStaleDirMtime --- PASS: TestRoutePointerTransitionUsesCachedMtimeNotStaleDirMtime (0.00s) === RUN TestRoutePointerTransitionNewerNoncurrentExpansionFiresOnCrossingThreshold --- PASS: TestRoutePointerTransitionNewerNoncurrentExpansionFiresOnCrossingThreshold (0.00s) === RUN TestRoutePointerTransitionExpansionEmitsOnlyThresholdCrossing --- PASS: TestRoutePointerTransitionExpansionEmitsOnlyThresholdCrossing (0.00s) === RUN TestRoutePointerTransitionSuspendedClearsPointerUsesNullMtime --- PASS: TestRoutePointerTransitionSuspendedClearsPointerUsesNullMtime (0.00s) === RUN TestRoutePointerTransitionSuspendedClearsPointerExpansionLatestPosIsNull --- PASS: TestRoutePointerTransitionSuspendedClearsPointerExpansionLatestPosIsNull (0.00s) === RUN TestScheduleEmpty --- PASS: TestScheduleEmpty (0.00s) === RUN TestScheduleOrderedByDueTime --- PASS: TestScheduleOrderedByDueTime (0.00s) === RUN TestScheduleDrainBoundaryInclusive --- PASS: TestScheduleDrainBoundaryInclusive (0.00s) === RUN TestScheduleAllowsDuplicates --- PASS: TestScheduleAllowsDuplicates (0.00s) === RUN TestScheduleDrainBeforeAnyDueReturnsNothing --- PASS: TestScheduleDrainBeforeAnyDueReturnsNothing (0.00s) === RUN TestScheduleNextDueAfterPartialDrain --- PASS: TestScheduleNextDueAfterPartialDrain (0.00s) === RUN TestScheduleAddAfterDrainKeepsOrder --- PASS: TestScheduleAddAfterDrainKeepsOrder (0.00s) === RUN TestScheduleDrainOrderIsAscendingDueTime --- PASS: TestScheduleDrainOrderIsAscendingDueTime (0.00s) === RUN TestScheduleConcurrentAddDrainNoRace --- PASS: TestScheduleConcurrentAddDrainNoRace (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/router 0.056s === RUN TestIsBucketVersioned_MissingExtendedReturnsFalse --- PASS: TestIsBucketVersioned_MissingExtendedReturnsFalse (0.00s) === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended/Enabled === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended/ENABLED === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended/enabled === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended/__enabled__ === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended/Suspended === RUN TestIsBucketVersioned_AcceptsEnabledAndSuspended/suspended --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended (0.00s) --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended/Enabled (0.00s) --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended/ENABLED (0.00s) --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended/enabled (0.00s) --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended/__enabled__ (0.00s) --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended/Suspended (0.00s) --- PASS: TestIsBucketVersioned_AcceptsEnabledAndSuspended/suspended (0.00s) === RUN TestIsBucketVersioned_RejectsOtherValues === RUN TestIsBucketVersioned_RejectsOtherValues/#00 === RUN TestIsBucketVersioned_RejectsOtherValues/Disabled === RUN TestIsBucketVersioned_RejectsOtherValues/Off === RUN TestIsBucketVersioned_RejectsOtherValues/true === RUN TestIsBucketVersioned_RejectsOtherValues/1 === RUN TestIsBucketVersioned_RejectsOtherValues/garbage --- PASS: TestIsBucketVersioned_RejectsOtherValues (0.00s) --- PASS: TestIsBucketVersioned_RejectsOtherValues/#00 (0.00s) --- PASS: TestIsBucketVersioned_RejectsOtherValues/Disabled (0.00s) --- PASS: TestIsBucketVersioned_RejectsOtherValues/Off (0.00s) --- PASS: TestIsBucketVersioned_RejectsOtherValues/true (0.00s) --- PASS: TestIsBucketVersioned_RejectsOtherValues/1 (0.00s) --- PASS: TestIsBucketVersioned_RejectsOtherValues/garbage (0.00s) === RUN TestAllActivePriorStates_EmptyInputs --- PASS: TestAllActivePriorStates_EmptyInputs (0.00s) === RUN TestAllActivePriorStates_SeedsAllActionsAsActive --- PASS: TestAllActivePriorStates_SeedsAllActionsAsActive (0.00s) === RUN TestAllActivePriorStates_KeysSeparatedByBucket --- PASS: TestAllActivePriorStates_KeysSeparatedByBucket (0.00s) === RUN TestLoadCompileInputs_EmptyBucketDir --- PASS: TestLoadCompileInputs_EmptyBucketDir (0.00s) === RUN TestLoadCompileInputs_FilesAtBucketLevelSkipped --- PASS: TestLoadCompileInputs_FilesAtBucketLevelSkipped (0.00s) === RUN TestLoadCompileInputs_BucketWithoutLifecycleConfigSkipped --- PASS: TestLoadCompileInputs_BucketWithoutLifecycleConfigSkipped (0.00s) === RUN TestLoadCompileInputs_BucketWithEmptyLifecycleXMLSkipped --- PASS: TestLoadCompileInputs_BucketWithEmptyLifecycleXMLSkipped (0.00s) === RUN TestLoadCompileInputs_ValidConfigBecomesInput --- PASS: TestLoadCompileInputs_ValidConfigBecomesInput (0.00s) === RUN TestLoadCompileInputs_VersioningAttrPropagates --- PASS: TestLoadCompileInputs_VersioningAttrPropagates (0.00s) === RUN TestLoadCompileInputs_MalformedXMLBecomesParseError --- PASS: TestLoadCompileInputs_MalformedXMLBecomesParseError (0.00s) === RUN TestLoadCompileInputs_PaginatesAcrossPages --- PASS: TestLoadCompileInputs_PaginatesAcrossPages (0.02s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/scheduler 0.050s === RUN TestDeleteTableDecoupledKeepsReusedNamePath --- PASS: TestDeleteTableDecoupledKeepsReusedNamePath (0.00s) === RUN TestDeleteTableRefusesAncestorDataPath E0720 21:30:02.306506 handler.go:199 S3Tables: error handling DeleteTable: refusing to delete table badt: data path "/buckets/renamebkt/ns" is an ancestor of catalog path "/buckets/renamebkt/ns/badt" --- PASS: TestDeleteTableRefusesAncestorDataPath (0.01s) === RUN TestDeleteTableColocatedRemovesData --- PASS: TestDeleteTableColocatedRemovesData (0.00s) === RUN TestGetAccountIDPrefersClaimsOverAccountID --- PASS: TestGetAccountIDPrefersClaimsOverAccountID (0.00s) === RUN TestGetAccountIDUsesSubWhenPreferredUsernameMissing --- PASS: TestGetAccountIDUsesSubWhenPreferredUsernameMissing (0.00s) === RUN TestGetAccountIDFallsBackToHandlerDefaultAccount --- PASS: TestGetAccountIDFallsBackToHandlerDefaultAccount (0.00s) === RUN TestGetAccountIDIgnoresEmptyClaimValues --- PASS: TestGetAccountIDIgnoresEmptyClaimValues (0.00s) === RUN TestGetAccountIDFallsBackToIdentityName --- PASS: TestGetAccountIDFallsBackToIdentityName (0.00s) === RUN TestGetAccountIDFallsBackToARNColonSegment --- PASS: TestGetAccountIDFallsBackToARNColonSegment (0.00s) === RUN TestGetAccountIDFallsBackToAmzAccountIdHeader --- PASS: TestGetAccountIDFallsBackToAmzAccountIdHeader (0.00s) === RUN TestGetAccountIDFallsBackToAccountID --- PASS: TestGetAccountIDFallsBackToAccountID (0.00s) === RUN TestGetAccountIDNonAdminDoesNotInheritAdminAccount --- PASS: TestGetAccountIDNonAdminDoesNotInheritAdminAccount (0.00s) === RUN TestGetAccountIDAdminActionKeepsAdminAccount --- PASS: TestGetAccountIDAdminActionKeepsAdminAccount (0.00s) === RUN TestDefaultAllowForOnlyAppliesToUnauthenticatedOrAnonymous --- PASS: TestDefaultAllowForOnlyAppliesToUnauthenticatedOrAnonymous (0.00s) === RUN TestGetAccountIDNormalizesAccountIDARN --- PASS: TestGetAccountIDNormalizesAccountIDARN (0.00s) === RUN TestNameValidationAllowsHyphens --- PASS: TestNameValidationAllowsHyphens (0.00s) === RUN TestParseTableFromARNAllowsHyphens --- PASS: TestParseTableFromARNAllowsHyphens (0.00s) === RUN TestRenameTablePreservesData E0720 21:30:02.320474 handler.go:199 S3Tables: error handling GetTable: attribute not found: s3tables.metadata --- PASS: TestRenameTablePreservesData (0.01s) === RUN TestRenameTableSourceMissing E0720 21:30:02.328814 handler.go:199 S3Tables: error handling RenameTable: filer: no entry is found in filer store --- PASS: TestRenameTableSourceMissing (0.00s) === RUN TestRenameTableDestExists E0720 21:30:02.332140 handler.go:199 S3Tables: error handling RenameTable: table already exists --- PASS: TestRenameTableDestExists (0.00s) === RUN TestRenameTableDestNamespaceMissing E0720 21:30:02.333164 handler.go:199 S3Tables: error handling RenameTable: filer: no entry is found in filer store --- PASS: TestRenameTableDestNamespaceMissing (0.00s) === RUN TestRenameTableDestNamespaceUnauthorized E0720 21:30:02.334192 handler.go:199 S3Tables: error handling RenameTable: unauthorized: mover is not permitted to perform RenameTable: not authorized to create table in the destination namespace --- PASS: TestRenameTableDestNamespaceUnauthorized (0.00s) === RUN TestRenameTableInvalidName E0720 21:30:02.334768 handler.go:199 S3Tables: error handling RenameTable: invalid table name: cannot be '.', '..' or contain '/' --- PASS: TestRenameTableInvalidName (0.00s) === RUN TestMetadataVersionFromLocation --- PASS: TestMetadataVersionFromLocation (0.00s) === RUN TestEntryType === RUN TestEntryType/nil_extended_is_table === RUN TestEntryType/absent_marker_is_table === RUN TestEntryType/empty_marker_is_table === RUN TestEntryType/table_marker === RUN TestEntryType/view_marker --- PASS: TestEntryType (0.00s) --- PASS: TestEntryType/nil_extended_is_table (0.00s) --- PASS: TestEntryType/absent_marker_is_table (0.00s) --- PASS: TestEntryType/empty_marker_is_table (0.00s) --- PASS: TestEntryType/table_marker (0.00s) --- PASS: TestEntryType/view_marker (0.00s) === RUN TestGenerateViewARN --- PASS: TestGenerateViewARN (0.00s) === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/flink-style_manifest_(job-id_+_checkpoint_+_operator-id_+_counter) === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/spark-style_manifest_with_two_uuids_and_-m_suffix === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/snapshot_manifest_list_with_iceberg-internal_naming === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/versioned_table_metadata === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/uuid-named_metadata_json_(newer_iceberg) === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/flink-style_metadata_json_with_dashes_and_digits === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/version_hint === RUN TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/trino/iceberg_stats_file --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/flink-style_manifest_(job-id_+_checkpoint_+_operator-id_+_counter) (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/spark-style_manifest_with_two_uuids_and_-m_suffix (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/snapshot_manifest_list_with_iceberg-internal_naming (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/versioned_table_metadata (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/uuid-named_metadata_json_(newer_iceberg) (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/flink-style_metadata_json_with_dashes_and_digits (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/version_hint (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldManifestNames/trino/iceberg_stats_file (0.00s) === RUN TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames === RUN TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/random_extension === RUN TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/executable_masquerading_as_avro_path === RUN TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/subdirectory_under_metadata_is_not_allowed === RUN TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/top-level_dir_other_than_metadata_or_data === RUN TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/catch-all_anchors_must_reject_trailing_extension --- PASS: TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames (0.00s) --- PASS: TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/random_extension (0.00s) --- PASS: TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/executable_masquerading_as_avro_path (0.00s) --- PASS: TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/subdirectory_under_metadata_is_not_allowed (0.00s) --- PASS: TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/top-level_dir_other_than_metadata_or_data (0.00s) --- PASS: TestIcebergLayoutValidator_RejectsClearlyBadMetadataNames/catch-all_anchors_must_reject_trailing_extension (0.00s) === RUN TestIcebergLayoutValidator_AcceptsRealWorldDataFiles === RUN TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/00000-0-ede83b82-08e1-40cd-af8a-6d83680a5194-00001.parquet === RUN TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/part-00000.parquet === RUN TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/some-file.orc === RUN TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/year=2026/month=05/00000-0-uuid.parquet --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldDataFiles (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/00000-0-ede83b82-08e1-40cd-af8a-6d83680a5194-00001.parquet (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/part-00000.parquet (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/some-file.orc (0.00s) --- PASS: TestIcebergLayoutValidator_AcceptsRealWorldDataFiles/data/year=2026/month=05/00000-0-uuid.parquet (0.00s) === RUN TestManagerCreateTableBucketAuthorization === RUN TestManagerCreateTableBucketAuthorization/authenticated_identity_struct_is_enforced E0720 21:30:02.335220 handler.go:199 S3Tables: error handling CreateTableBucket: unauthorized: alice is not permitted to perform CreateTableBucket: not authorized to create table buckets === RUN TestManagerCreateTableBucketAuthorization/secured_manager_denies_a_name_without_struct E0720 21:30:02.335247 handler.go:199 S3Tables: error handling CreateTableBucket: unauthorized: alice is not permitted to perform CreateTableBucket: not authorized to create table buckets === RUN TestManagerCreateTableBucketAuthorization/untrusted_name_without_struct_is_enforced E0720 21:30:02.335264 handler.go:199 S3Tables: error handling CreateTableBucket: unauthorized: alice is not permitted to perform CreateTableBucket: not authorized to create table buckets === RUN TestManagerCreateTableBucketAuthorization/trusted_manager_allows_a_name_without_struct E0720 21:30:02.335281 handler_bucket_create.go:98 S3Tables: failed to check bucket existence: filer reached E0720 21:30:02.335285 handler.go:199 S3Tables: error handling CreateTableBucket: filer reached === RUN TestManagerCreateTableBucketAuthorization/admin_principal_is_allowed E0720 21:30:02.335299 handler_bucket_create.go:98 S3Tables: failed to check bucket existence: filer reached E0720 21:30:02.335302 handler.go:199 S3Tables: error handling CreateTableBucket: filer reached --- PASS: TestManagerCreateTableBucketAuthorization (0.00s) --- PASS: TestManagerCreateTableBucketAuthorization/authenticated_identity_struct_is_enforced (0.00s) --- PASS: TestManagerCreateTableBucketAuthorization/secured_manager_denies_a_name_without_struct (0.00s) --- PASS: TestManagerCreateTableBucketAuthorization/untrusted_name_without_struct_is_enforced (0.00s) --- PASS: TestManagerCreateTableBucketAuthorization/trusted_manager_allows_a_name_without_struct (0.00s) --- PASS: TestManagerCreateTableBucketAuthorization/admin_principal_is_allowed (0.00s) === RUN TestMatchesActionPattern === RUN TestMatchesActionPattern/exact_match === RUN TestMatchesActionPattern/no_match === RUN TestMatchesActionPattern/universal_wildcard === RUN TestMatchesActionPattern/suffix_wildcard_match === RUN TestMatchesActionPattern/suffix_wildcard_no_match === RUN TestMatchesActionPattern/middle_wildcard_Get*Table === RUN TestMatchesActionPattern/middle_wildcard_Get*Table_no_match_GetTableBucket === RUN TestMatchesActionPattern/middle_wildcard_Get*Table_no_match_DeleteTable === RUN TestMatchesActionPattern/middle_wildcard_*Table* === RUN TestMatchesActionPattern/middle_wildcard_*Table*_match_CreateTable === RUN TestMatchesActionPattern/question_mark_single_char === RUN TestMatchesActionPattern/question_mark_no_match === RUN TestMatchesActionPattern/combined_*_and_?_singular === RUN TestMatchesActionPattern/combined_*_and_?_plural === RUN TestMatchesActionPattern/combined_no_match_-_?_needs_1_char --- PASS: TestMatchesActionPattern (0.00s) --- PASS: TestMatchesActionPattern/exact_match (0.00s) --- PASS: TestMatchesActionPattern/no_match (0.00s) --- PASS: TestMatchesActionPattern/universal_wildcard (0.00s) --- PASS: TestMatchesActionPattern/suffix_wildcard_match (0.00s) --- PASS: TestMatchesActionPattern/suffix_wildcard_no_match (0.00s) --- PASS: TestMatchesActionPattern/middle_wildcard_Get*Table (0.00s) --- PASS: TestMatchesActionPattern/middle_wildcard_Get*Table_no_match_GetTableBucket (0.00s) --- PASS: TestMatchesActionPattern/middle_wildcard_Get*Table_no_match_DeleteTable (0.00s) --- PASS: TestMatchesActionPattern/middle_wildcard_*Table* (0.00s) --- PASS: TestMatchesActionPattern/middle_wildcard_*Table*_match_CreateTable (0.00s) --- PASS: TestMatchesActionPattern/question_mark_single_char (0.00s) --- PASS: TestMatchesActionPattern/question_mark_no_match (0.00s) --- PASS: TestMatchesActionPattern/combined_*_and_?_singular (0.00s) --- PASS: TestMatchesActionPattern/combined_*_and_?_plural (0.00s) --- PASS: TestMatchesActionPattern/combined_no_match_-_?_needs_1_char (0.00s) === RUN TestMatchesPrincipal === RUN TestMatchesPrincipal/exact_match === RUN TestMatchesPrincipal/no_match === RUN TestMatchesPrincipal/universal_wildcard === RUN TestMatchesPrincipal/prefix_wildcard === RUN TestMatchesPrincipal/prefix_wildcard_no_match === RUN TestMatchesPrincipal/middle_wildcard === RUN TestMatchesPrincipal/array_match_first === RUN TestMatchesPrincipal/array_match_second === RUN TestMatchesPrincipal/array_no_match === RUN TestMatchesPrincipal/array_wildcard === RUN TestMatchesPrincipal/AWS_map_exact === RUN TestMatchesPrincipal/AWS_map_wildcard === RUN TestMatchesPrincipal/AWS_map_array === RUN TestMatchesPrincipal/nil_principal --- PASS: TestMatchesPrincipal (0.00s) --- PASS: TestMatchesPrincipal/exact_match (0.00s) --- PASS: TestMatchesPrincipal/no_match (0.00s) --- PASS: TestMatchesPrincipal/universal_wildcard (0.00s) --- PASS: TestMatchesPrincipal/prefix_wildcard (0.00s) --- PASS: TestMatchesPrincipal/prefix_wildcard_no_match (0.00s) --- PASS: TestMatchesPrincipal/middle_wildcard (0.00s) --- PASS: TestMatchesPrincipal/array_match_first (0.00s) --- PASS: TestMatchesPrincipal/array_match_second (0.00s) --- PASS: TestMatchesPrincipal/array_no_match (0.00s) --- PASS: TestMatchesPrincipal/array_wildcard (0.00s) --- PASS: TestMatchesPrincipal/AWS_map_exact (0.00s) --- PASS: TestMatchesPrincipal/AWS_map_wildcard (0.00s) --- PASS: TestMatchesPrincipal/AWS_map_array (0.00s) --- PASS: TestMatchesPrincipal/nil_principal (0.00s) === RUN TestEvaluatePolicyWithConditions === RUN TestEvaluatePolicyWithConditions/all_conditions_match === RUN TestEvaluatePolicyWithConditions/namespace_mismatch === RUN TestEvaluatePolicyWithConditions/table_name_mismatch === RUN TestEvaluatePolicyWithConditions/numeric_condition_failure === RUN TestEvaluatePolicyWithConditions/bool_condition_failure --- PASS: TestEvaluatePolicyWithConditions (0.00s) --- PASS: TestEvaluatePolicyWithConditions/all_conditions_match (0.00s) --- PASS: TestEvaluatePolicyWithConditions/namespace_mismatch (0.00s) --- PASS: TestEvaluatePolicyWithConditions/table_name_mismatch (0.00s) --- PASS: TestEvaluatePolicyWithConditions/numeric_condition_failure (0.00s) --- PASS: TestEvaluatePolicyWithConditions/bool_condition_failure (0.00s) === RUN TestCheckPermissionWithDefaultAllow === RUN TestCheckPermissionWithDefaultAllow/default_deny_(no_policy,_DefaultAllow=false) === RUN TestCheckPermissionWithDefaultAllow/default_allow_(no_policy,_DefaultAllow=true) === RUN TestCheckPermissionWithDefaultAllow/explicit_deny_overrides_DefaultAllow=true === RUN TestCheckPermissionWithDefaultAllow/explicit_allow_works_with_DefaultAllow=false --- PASS: TestCheckPermissionWithDefaultAllow (0.00s) --- PASS: TestCheckPermissionWithDefaultAllow/default_deny_(no_policy,_DefaultAllow=false) (0.00s) --- PASS: TestCheckPermissionWithDefaultAllow/default_allow_(no_policy,_DefaultAllow=true) (0.00s) --- PASS: TestCheckPermissionWithDefaultAllow/explicit_deny_overrides_DefaultAllow=true (0.00s) --- PASS: TestCheckPermissionWithDefaultAllow/explicit_allow_works_with_DefaultAllow=false (0.00s) === RUN TestTableDataDirFromMetadataLocation --- PASS: TestTableDataDirFromMetadataLocation (0.00s) === RUN TestValidateNamespaceSupportsMultiLevel --- PASS: TestValidateNamespaceSupportsMultiLevel (0.00s) === RUN TestValidateNamespaceSupportsDottedInput --- PASS: TestValidateNamespaceSupportsDottedInput (0.00s) === RUN TestValidateNamespaceRejectsEmptyDottedSegment --- PASS: TestValidateNamespaceRejectsEmptyDottedSegment (0.00s) === RUN TestParseNamespace === RUN TestParseNamespace/single_level === RUN TestParseNamespace/multi_level_dotted === RUN TestParseNamespace/invalid_reserved_prefix === RUN TestParseNamespace/invalid_empty_segment --- PASS: TestParseNamespace (0.00s) --- PASS: TestParseNamespace/single_level (0.00s) --- PASS: TestParseNamespace/multi_level_dotted (0.00s) --- PASS: TestParseNamespace/invalid_reserved_prefix (0.00s) --- PASS: TestParseNamespace/invalid_empty_segment (0.00s) === RUN TestParseTableFromARNWithMultiLevelNamespace --- PASS: TestParseTableFromARNWithMultiLevelNamespace (0.00s) === RUN TestBuildTableARNWithDottedNamespace --- PASS: TestBuildTableARNWithDottedNamespace (0.00s) === RUN TestExpandNamespace --- PASS: TestExpandNamespace (0.00s) === RUN TestNamespaceMetadataPropertiesRoundTrip === RUN TestNamespaceMetadataPropertiesRoundTrip/with_properties === RUN TestNamespaceMetadataPropertiesRoundTrip/nil_properties === RUN TestNamespaceMetadataPropertiesRoundTrip/empty_properties --- PASS: TestNamespaceMetadataPropertiesRoundTrip (0.00s) --- PASS: TestNamespaceMetadataPropertiesRoundTrip/with_properties (0.00s) --- PASS: TestNamespaceMetadataPropertiesRoundTrip/nil_properties (0.00s) --- PASS: TestNamespaceMetadataPropertiesRoundTrip/empty_properties (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/s3api/s3tables 0.081s === RUN TestUpdateSigningKeysRotates --- PASS: TestUpdateSigningKeysRotates (0.00s) === RUN TestUpdateSigningKeysConcurrent --- PASS: TestUpdateSigningKeysConcurrent (0.18s) === RUN TestUpdateSigningKeysTogglesWriteActive --- PASS: TestUpdateSigningKeysTogglesWriteActive (0.00s) === RUN TestIsWhiteListedEmptyConfigAllowsEverything --- PASS: TestIsWhiteListedEmptyConfigAllowsEverything (0.00s) === RUN TestIsWhiteListedWithListRejectsUnknown --- PASS: TestIsWhiteListedWithListRejectsUnknown (0.00s) === RUN TestIsWhiteListedWithSigningKeyButNoWhitelistAllowsAll --- PASS: TestIsWhiteListedWithSigningKeyButNoWhitelistAllowsAll (0.00s) === RUN TestSNI_HostnameStripsPort --- PASS: TestSNI_HostnameStripsPort (0.00s) === RUN TestSNI_IPAddressEmptySNI --- PASS: TestSNI_IPAddressEmptySNI (0.00s) === RUN TestSNI_IPv6AddressEmptySNI --- PASS: TestSNI_IPv6AddressEmptySNI (0.01s) PASS ok github.com/seaweedfs/seaweedfs/weed/security 0.212s === RUN TestServerGetCertificatePicksUpNewFile --- PASS: TestServerGetCertificatePicksUpNewFile (1.20s) PASS ok github.com/seaweedfs/seaweedfs/weed/security/certreload 1.217s === RUN TestSequencer I0720 21:30:02.308388 snowflake_sequencer.go:24 use snowflake seq id generator, nodeid:for_test hex_of_nodeid: 1 1cdb38a540c01000 1cdb38a540c01001 1cdb38a540c01002 1cdb38a540c01003 1cdb38a540c01004 1cdb38a540c01005 1cdb38a540c01006 1cdb38a540c01007 1cdb38a540c01008 1cdb38a540c01009 1cdb38a540c0100a 1cdb38a540c0100b 1cdb38a540c0100c 1cdb38a540c0100d 1cdb38a540c0100e 1cdb38a540c0100f 1cdb38a540c01010 1cdb38a540c01011 1cdb38a540c01012 1cdb38a540c01013 1cdb38a540c01014 1cdb38a540c01015 1cdb38a540c01016 1cdb38a540c01017 1cdb38a540c01018 1cdb38a540c01019 1cdb38a540c0101a 1cdb38a540c0101b 1cdb38a540c0101c 1cdb38a540c0101d 1cdb38a540c0101e 1cdb38a540c0101f 1cdb38a540c01020 1cdb38a540c01021 1cdb38a540c01022 1cdb38a540c01023 1cdb38a540c01024 1cdb38a540c01025 1cdb38a540c01026 1cdb38a540c01027 1cdb38a540c01028 1cdb38a540c01029 1cdb38a540c0102a 1cdb38a540c0102b 1cdb38a540c0102c 1cdb38a540c0102d 1cdb38a540c0102e 1cdb38a540c0102f 1cdb38a540c01030 1cdb38a540c01031 1cdb38a540c01032 1cdb38a540c01033 1cdb38a540c01034 1cdb38a540c01035 1cdb38a540c01036 1cdb38a540c01037 1cdb38a540c01038 1cdb38a540c01039 1cdb38a540c0103a 1cdb38a540c0103b 1cdb38a540c0103c 1cdb38a540c0103d 1cdb38a540c0103e 1cdb38a540c0103f 1cdb38a540c01040 1cdb38a540c01041 1cdb38a540c01042 1cdb38a540c01043 1cdb38a540c01044 1cdb38a540c01045 1cdb38a540c01046 1cdb38a540c01047 1cdb38a540c01048 1cdb38a540c01049 1cdb38a540c0104a 1cdb38a540c0104b 1cdb38a540c0104c 1cdb38a540c0104d 1cdb38a540c0104e 1cdb38a540c0104f 1cdb38a540c01050 1cdb38a540c01051 1cdb38a540c01052 1cdb38a540c01053 1cdb38a540c01054 1cdb38a540c01055 1cdb38a540c01056 1cdb38a540c01057 1cdb38a540c01058 1cdb38a540c01059 1cdb38a540c0105a 1cdb38a540c0105b 1cdb38a540c0105c 1cdb38a540c0105d 1cdb38a540c0105e 1cdb38a540c0105f 1cdb38a540c01060 1cdb38a540c01061 1cdb38a540c01062 1cdb38a540c01063 --- PASS: TestSequencer (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/sequence 0.046s === RUN TestParseURL --- PASS: TestParseURL (0.00s) === RUN TestWriteJsonNoJSONP === RUN TestWriteJsonNoJSONP/callback= === RUN TestWriteJsonNoJSONP/callback=myCb === RUN TestWriteJsonNoJSONP/callback= --- PASS: TestWriteJsonNoJSONP (0.00s) --- PASS: TestWriteJsonNoJSONP/callback= (0.00s) --- PASS: TestWriteJsonNoJSONP/callback=myCb (0.00s) --- PASS: TestWriteJsonNoJSONP/callback= (0.00s) === RUN TestWriteJsonPrettyDoesNotReadMultipartBody I0720 21:30:02.360111 common.go:109 response method:POST URL:/x with httpStatus:429 and JSON:{"error":"busy"} --- PASS: TestWriteJsonPrettyDoesNotReadMultipartBody (0.00s) === RUN TestWriteConditionSatisfied --- PASS: TestWriteConditionSatisfied (0.00s) === RUN TestWriteConditionClauses --- PASS: TestWriteConditionClauses (0.00s) === RUN TestWriteConditionObjectLockGuards --- PASS: TestWriteConditionObjectLockGuards (0.00s) === RUN TestWriteConditionUnknownKindFailsClosed --- PASS: TestWriteConditionUnknownKindFailsClosed (0.00s) === RUN TestStoredEntryETag --- PASS: TestStoredEntryETag (0.00s) === RUN TestCreateEntryConditionEnforced --- PASS: TestCreateEntryConditionEnforced (0.01s) === RUN TestCreateEntryReusesProvidedExisting --- PASS: TestCreateEntryReusesProvidedExisting (0.00s) === RUN TestCreateEntryOExclSerialized --- PASS: TestCreateEntryOExclSerialized (0.07s) === RUN TestFindLockOwnerExpiredLockReturnsNotFound I0720 21:30:02.438964 filer_grpc_server_dlm.go:122 find lock expired-lock moved to : lock not found --- PASS: TestFindLockOwnerExpiredLockReturnsNotFound (0.00s) === RUN TestMountRegister_EnabledStoresAndLists --- PASS: TestMountRegister_EnabledStoresAndLists (0.00s) === RUN TestMountRegister_RenewIsIdempotent --- PASS: TestMountRegister_RenewIsIdempotent (0.00s) === RUN TestObjectTransactionMultiEntry --- PASS: TestObjectTransactionMultiEntry (0.00s) === RUN TestObjectTransactionPatchNotifies --- PASS: TestObjectTransactionPatchNotifies (0.00s) === RUN TestObjectTransactionPatchContent --- PASS: TestObjectTransactionPatchContent (0.00s) === RUN TestObjectTransactionPreconditionAborts --- PASS: TestObjectTransactionPreconditionAborts (0.00s) === RUN TestObjectTransactionRecomputeLatest --- PASS: TestObjectTransactionRecomputeLatest (0.00s) === RUN TestObjectTransactionRecomputeAscending --- PASS: TestObjectTransactionRecomputeAscending (0.00s) === RUN TestObjectTransactionBatchIndependent --- PASS: TestObjectTransactionBatchIndependent (0.00s) === RUN TestObjectTransactionBatchNilTransaction --- PASS: TestObjectTransactionBatchNilTransaction (0.00s) === RUN TestObjectTransactionIdempotentNoops --- PASS: TestObjectTransactionIdempotentNoops (0.01s) === RUN TestObjectTransactionRecomputeDemoteAndAttrs --- PASS: TestObjectTransactionRecomputeDemoteAndAttrs (0.00s) === RUN TestObjectTransactionPutThenRecomputeLatest --- PASS: TestObjectTransactionPutThenRecomputeLatest (0.00s) === RUN TestObjectTransactionVersionDeleteWithWorm --- PASS: TestObjectTransactionVersionDeleteWithWorm (0.00s) === RUN TestObjectTransactionDeleteRemovesEmptyParent --- PASS: TestObjectTransactionDeleteRemovesEmptyParent (0.00s) === RUN TestObjectTransactionPatchTouchMtime --- PASS: TestObjectTransactionPatchTouchMtime (0.00s) === RUN TestObjectTransactionRouteKeyOwnerAppliesLocally --- PASS: TestObjectTransactionRouteKeyOwnerAppliesLocally (0.00s) === RUN TestObjectTransactionIsMovedSkipsForward --- PASS: TestObjectTransactionIsMovedSkipsForward (0.00s) === RUN TestObjectTransactionForwardsToOwner --- PASS: TestObjectTransactionForwardsToOwner (0.00s) === RUN TestObjectTransactionPutAppliesConfiguredTTLAndExpires --- PASS: TestObjectTransactionPutAppliesConfiguredTTLAndExpires (0.01s) === RUN TestObjectTransactionPutPreservesExplicitTTL --- PASS: TestObjectTransactionPutPreservesExplicitTTL (0.00s) === RUN TestObjectTransactionPutClearsTTLForRemoteEntry --- PASS: TestObjectTransactionPutClearsTTLForRemoteEntry (0.00s) === RUN TestObjectTransactionPutHonorsMaxFileNameLengthRule --- PASS: TestObjectTransactionPutHonorsMaxFileNameLengthRule (0.00s) === RUN TestObjectTransactionPutFailsOnReadOnlyStorageConfig --- PASS: TestObjectTransactionPutFailsOnReadOnlyStorageConfig (0.00s) === RUN TestPosixLockGrantAndConflict --- PASS: TestPosixLockGrantAndConflict (0.00s) === RUN TestPosixLockUnlockThenReacquire --- PASS: TestPosixLockUnlockThenReacquire (0.00s) === RUN TestPosixLockGetLk --- PASS: TestPosixLockGetLk (0.00s) === RUN TestPosixLockReleasePosixOwnerKeepsFlock --- PASS: TestPosixLockReleasePosixOwnerKeepsFlock (0.00s) === RUN TestPosixLockKeepAlive --- PASS: TestPosixLockKeepAlive (0.00s) === RUN TestPosixLockForwardsToOwner --- PASS: TestPosixLockForwardsToOwner (0.01s) === RUN TestPosixLockWarmupDefersGrants --- PASS: TestPosixLockWarmupDefersGrants (0.00s) === RUN TestRecomputeLatestEmitsPointerUpdateEvent --- PASS: TestRecomputeLatestEmitsPointerUpdateEvent (0.00s) === RUN TestRecomputeLatestDemoteEmitsEvent --- PASS: TestRecomputeLatestDemoteEmitsEvent (0.00s) === RUN TestAtomicRenameEntryEmitsLogicalRenameEvent --- PASS: TestAtomicRenameEntryEmitsLogicalRenameEvent (0.00s) === RUN TestAtomicRenameEntryOverwriteEmitsDeleteThenRename --- PASS: TestAtomicRenameEntryOverwriteEmitsDeleteThenRename (0.00s) === RUN TestAtomicRenameEntryDoesNotEmitEventOnDeleteFailure --- PASS: TestAtomicRenameEntryDoesNotEmitEventOnDeleteFailure (0.00s) === RUN TestAtomicRenameEntryDoesNotEmitEventOnCommitFailure --- PASS: TestAtomicRenameEntryDoesNotEmitEventOnCommitFailure (0.00s) === RUN TestAtomicRenameEntrySkipsDescendantTargetLookups --- PASS: TestAtomicRenameEntrySkipsDescendantTargetLookups (0.00s) === RUN TestReproStreamSerializationCeiling filer_grpc_server_stream_mutate_bench_test.go:550: service delay: 2ms filer_grpc_server_stream_mutate_bench_test.go:551: concurrency: 12 filer_grpc_server_stream_mutate_bench_test.go:552: ops per run: 1200 filer_grpc_server_stream_mutate_bench_test.go:553: unary : 1200 calls in 449.394817ms -> 2670 QPS filer_grpc_server_stream_mutate_bench_test.go:554: stream : 1200 calls in 3.071839816s -> 391 QPS filer_grpc_server_stream_mutate_bench_test.go:555: unary / stream ratio: 6.84x --- PASS: TestReproStreamSerializationCeiling (3.53s) === RUN TestServerSerialVsConcurrentHandler filer_grpc_server_stream_mutate_bench_test.go:617: service delay: 2ms filer_grpc_server_stream_mutate_bench_test.go:618: server-side concurrency cap: 64 filer_grpc_server_stream_mutate_bench_test.go:619: theoretical serial ceiling: 500 QPS filer_grpc_server_stream_mutate_bench_test.go:620: theoretical concurrent cap: 32000 QPS filer_grpc_server_stream_mutate_bench_test.go:621: ops per run: 2400 filer_grpc_server_stream_mutate_bench_test.go:622: filer_grpc_server_stream_mutate_bench_test.go:623: client workers serial QPS concurrent QPS lift filer_grpc_server_stream_mutate_bench_test.go:637: 1 408 368 0.90x filer_grpc_server_stream_mutate_bench_test.go:637: 12 320 5137 16.07x filer_grpc_server_stream_mutate_bench_test.go:637: 64 308 11425 37.08x filer_grpc_server_stream_mutate_bench_test.go:637: 256 275 15514 56.34x --- PASS: TestServerSerialVsConcurrentHandler (37.25s) === RUN TestSchedulerOrderedParallelism filer_grpc_server_stream_mutate_bench_test.go:698: service delay: 2ms filer_grpc_server_stream_mutate_bench_test.go:699: server concurrency: 64 filer_grpc_server_stream_mutate_bench_test.go:700: client workers: 12 filer_grpc_server_stream_mutate_bench_test.go:701: ops per run: 2400 filer_grpc_server_stream_mutate_bench_test.go:702: serial ceiling: 500 QPS filer_grpc_server_stream_mutate_bench_test.go:703: concurrent ceiling: 32000 QPS (64 x 500) filer_grpc_server_stream_mutate_bench_test.go:704: filer_grpc_server_stream_mutate_bench_test.go:714: workload serial QPS sem QPS sched QPS sched peak-conc filer_grpc_server_stream_mutate_bench_test.go:732: distinct paths 277 2783 2714 12 filer_grpc_server_stream_mutate_bench_test.go:732: same path 364 4810 391 1 --- PASS: TestSchedulerOrderedParallelism (23.65s) === RUN TestStreamSyncVsAsyncClient filer_grpc_server_stream_mutate_bench_test.go:772: service delay: 2ms (theoretical serial ceiling = 500 QPS) filer_grpc_server_stream_mutate_bench_test.go:774: ops per run: 2400 filer_grpc_server_stream_mutate_bench_test.go:775: filer_grpc_server_stream_mutate_bench_test.go:776: concurrency sync QPS async QPS ratio filer_grpc_server_stream_mutate_bench_test.go:789: 1 266 408 1.54 x filer_grpc_server_stream_mutate_bench_test.go:789: 12 441 437 0.99 x filer_grpc_server_stream_mutate_bench_test.go:789: 64 441 446 1.01 x filer_grpc_server_stream_mutate_bench_test.go:789: 256 439 447 1.02 x --- PASS: TestStreamSyncVsAsyncClient (51.53s) === RUN TestMutatePathAncestors --- PASS: TestMutatePathAncestors (0.00s) === RUN TestClassifyMutation === RUN TestClassifyMutation/create_file === RUN TestClassifyMutation/create_directory_(barrier) === RUN TestClassifyMutation/update_file === RUN TestClassifyMutation/update_directory_(non-barrier_attr_bump) === RUN TestClassifyMutation/delete_non-recursive_(barrier:_target_type_unknown) === RUN TestClassifyMutation/delete_recursive_(barrier) === RUN TestClassifyMutation/malformed_create_(nil_Entry) === RUN TestClassifyMutation/malformed_update_(nil_Entry) === RUN TestClassifyMutation/empty_oneof === RUN TestClassifyMutation/rename --- PASS: TestClassifyMutation (0.00s) --- PASS: TestClassifyMutation/create_file (0.00s) --- PASS: TestClassifyMutation/create_directory_(barrier) (0.00s) --- PASS: TestClassifyMutation/update_file (0.00s) --- PASS: TestClassifyMutation/update_directory_(non-barrier_attr_bump) (0.00s) --- PASS: TestClassifyMutation/delete_non-recursive_(barrier:_target_type_unknown) (0.00s) --- PASS: TestClassifyMutation/delete_recursive_(barrier) (0.00s) --- PASS: TestClassifyMutation/malformed_create_(nil_Entry) (0.00s) --- PASS: TestClassifyMutation/malformed_update_(nil_Entry) (0.00s) --- PASS: TestClassifyMutation/empty_oneof (0.00s) --- PASS: TestClassifyMutation/rename (0.00s) === RUN TestPathConflictSamePathFile --- PASS: TestPathConflictSamePathFile (0.00s) === RUN TestPathConflictBarrierBlocksSamePath --- PASS: TestPathConflictBarrierBlocksSamePath (0.00s) === RUN TestPathConflictBarrierBlocksDescendants --- PASS: TestPathConflictBarrierBlocksDescendants (0.00s) === RUN TestPathConflictIncomingBarrierWaitsForDescendants --- PASS: TestPathConflictIncomingBarrierWaitsForDescendants (0.00s) === RUN TestPathConflictNonBarrierDirAllowsDescendants --- PASS: TestPathConflictNonBarrierDirAllowsDescendants (0.00s) === RUN TestPathConflictIncomingBarrierWaitsForSamePathNonBarrier --- PASS: TestPathConflictIncomingBarrierWaitsForSamePathNonBarrier (0.00s) === RUN TestAdmitDoneLifecycle --- PASS: TestAdmitDoneLifecycle (0.05s) === RUN TestAdmitParallelDistinctPaths --- PASS: TestAdmitParallelDistinctPaths (0.00s) === RUN TestAdmitConcurrencyCap --- PASS: TestAdmitConcurrencyCap (0.05s) === RUN TestAdmitRenameTwoPathConflict --- PASS: TestAdmitRenameTwoPathConflict (0.05s) === RUN TestAdmitSamePathFIFO --- PASS: TestAdmitSamePathFIFO (0.04s) === RUN TestAdmitSamePathNonBarrierSerializes --- PASS: TestAdmitSamePathNonBarrierSerializes (0.03s) === RUN TestAdmitPressureFromManyWaiters --- PASS: TestAdmitPressureFromManyWaiters (0.11s) === RUN TestEachLogEntryFnPrefilterSkipsDecode --- PASS: TestEachLogEntryFnPrefilterSkipsDecode (0.00s) === RUN TestEachLogEntryFnNoFilterDecodesEverything --- PASS: TestEachLogEntryFnNoFilterDecodesEverything (0.00s) === RUN TestPipelinedSenderThroughput === RUN TestPipelinedSenderThroughput/direct_send filer_grpc_server_sub_meta_test.go:126: direct: 2100 events 2100 sends 2.312s 908 events/sec === RUN TestPipelinedSenderThroughput/pipelined_batched_send filer_grpc_server_sub_meta_test.go:151: pipelined+batch: 2100 events 14 sends 39ms 54059 events/sec === NAME TestPipelinedSenderThroughput filer_grpc_server_sub_meta_test.go:156: Speedup: 59.5x (pipelined+batched vs direct) --- PASS: TestPipelinedSenderThroughput (2.35s) --- PASS: TestPipelinedSenderThroughput/direct_send (2.31s) --- PASS: TestPipelinedSenderThroughput/pipelined_batched_send (0.04s) === RUN TestEachEventNotificationFnMatchesRenameTargetsForAllWatchTypes === RUN TestEachEventNotificationFnMatchesRenameTargetsForAllWatchTypes/additional_path_prefix === RUN TestEachEventNotificationFnMatchesRenameTargetsForAllWatchTypes/directory_watch --- PASS: TestEachEventNotificationFnMatchesRenameTargetsForAllWatchTypes (0.00s) --- PASS: TestEachEventNotificationFnMatchesRenameTargetsForAllWatchTypes/additional_path_prefix (0.00s) --- PASS: TestEachEventNotificationFnMatchesRenameTargetsForAllWatchTypes/directory_watch (0.00s) === RUN TestBatchingAdaptive === RUN TestBatchingAdaptive/old_events_are_batched filer_grpc_server_sub_meta_test.go:230: old events: 500 events in 2 sends (avg batch size: 250.0) === RUN TestBatchingAdaptive/recent_events_sent_individually filer_grpc_server_sub_meta_test.go:249: recent events: 500 events in 500 sends (avg batch size: 1.0) --- PASS: TestBatchingAdaptive (0.53s) --- PASS: TestBatchingAdaptive/old_events_are_batched (0.00s) --- PASS: TestBatchingAdaptive/recent_events_sent_individually (0.52s) === RUN TestPipelinedSenderErrorPropagation === RUN TestPipelinedSenderErrorPropagation/send_returns_error filer_grpc_server_sub_meta_test.go:298: Send returned error after stream failure: connection reset === RUN TestPipelinedSenderErrorPropagation/close_returns_error_if_not_consumed filer_grpc_server_sub_meta_test.go:315: Close returned error: connection reset --- PASS: TestPipelinedSenderErrorPropagation (0.00s) --- PASS: TestPipelinedSenderErrorPropagation/send_returns_error (0.00s) --- PASS: TestPipelinedSenderErrorPropagation/close_returns_error_if_not_consumed (0.00s) === RUN TestPipelinedSingleVsParallelStreams === RUN TestPipelinedSingleVsParallelStreams/1_pipelined_stream filer_grpc_server_sub_meta_test.go:383: 1 stream: 21000 events 140 sends 387ms 54232 events/sec === RUN TestPipelinedSingleVsParallelStreams/10_pipelined_streams filer_grpc_server_sub_meta_test.go:410: 10 streams: 21000 events 140 sends 40ms 518872 events/sec === NAME TestPipelinedSingleVsParallelStreams filer_grpc_server_sub_meta_test.go:415: Speedup: 9.6x (10 parallel pipelined streams vs 1) --- PASS: TestPipelinedSingleVsParallelStreams (0.43s) --- PASS: TestPipelinedSingleVsParallelStreams/1_pipelined_stream (0.39s) --- PASS: TestPipelinedSingleVsParallelStreams/10_pipelined_streams (0.04s) === RUN TestMaybeSendIdleHeartbeat === RUN TestMaybeSendIdleHeartbeat/not_opted_in === RUN TestMaybeSendIdleHeartbeat/behind_buffer_head === RUN TestMaybeSendIdleHeartbeat/caught_up_via_lastSeen === RUN TestMaybeSendIdleHeartbeat/caught_up_via_read_position_floor === RUN TestMaybeSendIdleHeartbeat/throttled_within_interval === RUN TestMaybeSendIdleHeartbeat/send_error_keeps_prior_heartbeat_time I0720 21:32:02.096272 filer_grpc_server_sub_meta.go:567 => idle heartbeat to : broken stream --- PASS: TestMaybeSendIdleHeartbeat (0.00s) --- PASS: TestMaybeSendIdleHeartbeat/not_opted_in (0.00s) --- PASS: TestMaybeSendIdleHeartbeat/behind_buffer_head (0.00s) --- PASS: TestMaybeSendIdleHeartbeat/caught_up_via_lastSeen (0.00s) --- PASS: TestMaybeSendIdleHeartbeat/caught_up_via_read_position_floor (0.00s) --- PASS: TestMaybeSendIdleHeartbeat/throttled_within_interval (0.00s) --- PASS: TestMaybeSendIdleHeartbeat/send_error_keeps_prior_heartbeat_time (0.00s) === RUN TestFilteredEventsEmitMaxUnsyncedMarker filer_grpc_server_sub_meta_test.go:543: source emits marker{EventNotification:&{}, TsNs:1784583122096285603} after 1001 filtered events --- PASS: TestFilteredEventsEmitMaxUnsyncedMarker (0.00s) === RUN TestResolveAssignStorageOptionUsesBucketRuleBeforeFilerDiskDefault --- PASS: TestResolveAssignStorageOptionUsesBucketRuleBeforeFilerDiskDefault (0.00s) === RUN TestResolveAssignStorageOptionFallsBackToFilerDiskDefault --- PASS: TestResolveAssignStorageOptionFallsBackToFilerDiskDefault (0.00s) === RUN TestPtrie matched1 /topics/abc matched1 /topics/abc/d matched2 /topics/abc matched2 /topics/abc/d --- PASS: TestPtrie (0.00s) === RUN TestTraverseBfsMetadata I0720 21:32:02.100262 filer_grpc_server_traverse_meta.go:16 TraverseBfsMetadata directory:"/" excluded_prefixes:"/.system" --- PASS: TestTraverseBfsMetadata (0.00s) === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/no_restrictions === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_prefix_match === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_prefix_mismatch === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_method_match === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_method_mismatch === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/both_match === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/prefix_match,_method_mismatch === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/multiple_prefixes_match === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/write_operation_with_method_restriction === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/root_path_with_prefix_restriction === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/root_path_without_token === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/exact_prefix_match === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_does_not_match_sibling_tenant === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_does_not_match_dashed_sibling === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_matches_own_subtree === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_with_trailing_slash_matches_own_subtree === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/root_prefix_matches_anywhere === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/dot-dot_cannot_escape_allowed_subtree === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/copy_source_within_subtree_allowed === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/copy_source_outside_subtree_denied === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/copy_source_dot-dot_escape_denied === RUN TestFilerServer_maybeCheckJwtAuthorization_Scoped/move_source_outside_subtree_denied --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/no_restrictions (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_prefix_match (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_prefix_mismatch (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_method_match (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/allowed_method_mismatch (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/both_match (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/prefix_match,_method_mismatch (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/multiple_prefixes_match (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/write_operation_with_method_restriction (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/root_path_with_prefix_restriction (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/root_path_without_token (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/exact_prefix_match (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_does_not_match_sibling_tenant (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_does_not_match_dashed_sibling (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_matches_own_subtree (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/tenant_prefix_with_trailing_slash_matches_own_subtree (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/root_prefix_matches_anywhere (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/dot-dot_cannot_escape_allowed_subtree (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/copy_source_within_subtree_allowed (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/copy_source_outside_subtree_denied (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/copy_source_dot-dot_escape_denied (0.00s) --- PASS: TestFilerServer_maybeCheckJwtAuthorization_Scoped/move_source_outside_subtree_denied (0.00s) === RUN TestPathHasComponentPrefix === RUN TestPathHasComponentPrefix/exact_match === RUN TestPathHasComponentPrefix/subtree_match === RUN TestPathHasComponentPrefix/trailing_slash_prefix === RUN TestPathHasComponentPrefix/sibling_numeric_suffix === RUN TestPathHasComponentPrefix/sibling_dash_suffix === RUN TestPathHasComponentPrefix/sibling_dot_suffix === RUN TestPathHasComponentPrefix/unrelated_tree === RUN TestPathHasComponentPrefix/root_prefix_matches_root === RUN TestPathHasComponentPrefix/root_prefix_matches_any === RUN TestPathHasComponentPrefix/empty_prefix_denies === RUN TestPathHasComponentPrefix/dot-dot_does_not_escape === RUN TestPathHasComponentPrefix/dot-dot_stays_inside === RUN TestPathHasComponentPrefix/double_slashes_normalised --- PASS: TestPathHasComponentPrefix (0.00s) --- PASS: TestPathHasComponentPrefix/exact_match (0.00s) --- PASS: TestPathHasComponentPrefix/subtree_match (0.00s) --- PASS: TestPathHasComponentPrefix/trailing_slash_prefix (0.00s) --- PASS: TestPathHasComponentPrefix/sibling_numeric_suffix (0.00s) --- PASS: TestPathHasComponentPrefix/sibling_dash_suffix (0.00s) --- PASS: TestPathHasComponentPrefix/sibling_dot_suffix (0.00s) --- PASS: TestPathHasComponentPrefix/unrelated_tree (0.00s) --- PASS: TestPathHasComponentPrefix/root_prefix_matches_root (0.00s) --- PASS: TestPathHasComponentPrefix/root_prefix_matches_any (0.00s) --- PASS: TestPathHasComponentPrefix/empty_prefix_denies (0.00s) --- PASS: TestPathHasComponentPrefix/dot-dot_does_not_escape (0.00s) --- PASS: TestPathHasComponentPrefix/dot-dot_stays_inside (0.00s) --- PASS: TestPathHasComponentPrefix/double_slashes_normalised (0.00s) === RUN TestCopyEntryRefreshesDestinationTimestamps --- PASS: TestCopyEntryRefreshesDestinationTimestamps (0.00s) === RUN TestPreserveDestinationMetadataForDataCopy --- PASS: TestPreserveDestinationMetadataForDataCopy (0.00s) === RUN TestValidateCopySourcePreconditions --- PASS: TestValidateCopySourcePreconditions (0.00s) === RUN TestValidateCopyDestinationPreconditions --- PASS: TestValidateCopyDestinationPreconditions (0.00s) === RUN TestRecentCopyRequestDeduplicatesByRequestID --- PASS: TestRecentCopyRequestDeduplicatesByRequestID (0.00s) === RUN TestIamGrpc_NoMetadata_Unauthenticated --- PASS: TestIamGrpc_NoMetadata_Unauthenticated (0.00s) === RUN TestIamGrpc_MissingAuthorizationHeader_Unauthenticated --- PASS: TestIamGrpc_MissingAuthorizationHeader_Unauthenticated (0.00s) === RUN TestIamGrpc_NonBearerAuthorization_Unauthenticated --- PASS: TestIamGrpc_NonBearerAuthorization_Unauthenticated (0.00s) === RUN TestIamGrpc_InvalidToken_Unauthenticated --- PASS: TestIamGrpc_InvalidToken_Unauthenticated (0.00s) === RUN TestIamGrpc_GarbageToken_Unauthenticated --- PASS: TestIamGrpc_GarbageToken_Unauthenticated (0.00s) === RUN TestIamGrpc_ExpiredToken_Unauthenticated --- PASS: TestIamGrpc_ExpiredToken_Unauthenticated (0.00s) === RUN TestIamGrpc_ValidToken_ReachesHandler --- PASS: TestIamGrpc_ValidToken_ReachesHandler (0.00s) === RUN TestIamGrpc_NoSigningKey_Unauthenticated_Allowed --- PASS: TestIamGrpc_NoSigningKey_Unauthenticated_Allowed (0.00s) === RUN TestIamGrpc_CreateUser_RequiresAuth --- PASS: TestIamGrpc_CreateUser_RequiresAuth (0.00s) === RUN TestProxySemaphore_LimitsConcurrency --- PASS: TestProxySemaphore_LimitsConcurrency (0.00s) === RUN TestProxySemaphore_ContextCancellation --- PASS: TestProxySemaphore_ContextCancellation (0.00s) === RUN TestProxySemaphore_PerHostIsolation --- PASS: TestProxySemaphore_PerHostIsolation (0.00s) === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack/cross-prefix_HEAD_denied I0720 21:32:02.107802 common.go:109 response method:HEAD URL:/.tus/.uploads/9f6f0d4b-6556-48f6-b953-0d8fca1966f1 with httpStatus:401 and JSON:{"error":"wrong jwt"} === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack/matching-prefix_HEAD_allowed === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack/cross-prefix_PATCH_denied I0720 21:32:02.108548 common.go:109 response method:PATCH URL:/.tus/.uploads/9f6f0d4b-6556-48f6-b953-0d8fca1966f1 with httpStatus:401 and JSON:{"error":"wrong jwt"} === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack/matching-prefix_PATCH_allowed === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack/cross-prefix_DELETE_denied I0720 21:32:02.108768 common.go:109 response method:DELETE URL:/.tus/.uploads/9f6f0d4b-6556-48f6-b953-0d8fca1966f1 with httpStatus:401 and JSON:{"error":"wrong jwt"} === RUN TestFilerServer_tusHandler_CrossPrefixSessionHijack/matching-prefix_DELETE_allowed --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack (0.00s) --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack/cross-prefix_HEAD_denied (0.00s) --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack/matching-prefix_HEAD_allowed (0.00s) --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack/cross-prefix_PATCH_denied (0.00s) --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack/matching-prefix_PATCH_allowed (0.00s) --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack/cross-prefix_DELETE_denied (0.00s) --- PASS: TestFilerServer_tusHandler_CrossPrefixSessionHijack/matching-prefix_DELETE_allowed (0.00s) === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/trailing_path_alias === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/non-canonical_route_id === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/stored_id_mismatch === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/empty_stored_target === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/root_stored_target === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/relative_stored_target === RUN TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/oversize_stored_size --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/trailing_path_alias (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/non-canonical_route_id (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/stored_id_mismatch (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/empty_stored_target (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/root_stored_target (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/relative_stored_target (0.00s) --- PASS: TestFilerServer_tusHandler_RejectsAliasesAndInvalidMetadata/oversize_stored_size (0.00s) === RUN TestFilerServer_checkJwtAuthorization === RUN TestFilerServer_checkJwtAuthorization/post_without_token_denied === RUN TestFilerServer_checkJwtAuthorization/patch_without_token_denied === RUN TestFilerServer_checkJwtAuthorization/head_without_token_denied === RUN TestFilerServer_checkJwtAuthorization/post_write_token_allowed === RUN TestFilerServer_checkJwtAuthorization/head_read_token_allowed === RUN TestFilerServer_checkJwtAuthorization/head_write_token_denied === RUN TestFilerServer_checkJwtAuthorization/post_read_token_denied === RUN TestFilerServer_checkJwtAuthorization/within_allowed_prefix === RUN TestFilerServer_checkJwtAuthorization/outside_allowed_prefix_denied === RUN TestFilerServer_checkJwtAuthorization/restricted_token_without_resource_denied === RUN TestFilerServer_checkJwtAuthorization/matching_method_allowed === RUN TestFilerServer_checkJwtAuthorization/method_not_allowed_denied --- PASS: TestFilerServer_checkJwtAuthorization (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/post_without_token_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/patch_without_token_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/head_without_token_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/post_write_token_allowed (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/head_read_token_allowed (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/head_write_token_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/post_read_token_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/within_allowed_prefix (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/outside_allowed_prefix_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/restricted_token_without_resource_denied (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/matching_method_allowed (0.00s) --- PASS: TestFilerServer_checkJwtAuthorization/method_not_allowed_denied (0.00s) === RUN TestFilerServer_tusHandler_UnauthenticatedRejected I0720 21:32:02.109940 common.go:109 response method:POST URL:/.tus/buckets/secret/owned.txt with httpStatus:401 and JSON:{"error":"wrong jwt"} I0720 21:32:02.109945 common.go:109 response method:HEAD URL:/.tus/.uploads/9f6f0d4b-6556-48f6-b953-0d8fca1966f1 with httpStatus:401 and JSON:{"error":"wrong jwt"} --- PASS: TestFilerServer_tusHandler_UnauthenticatedRejected (0.00s) === RUN TestFilerServer_refreshTusSessionChunks_RevalidatesPinnedSession === RUN TestFilerServer_refreshTusSessionChunks_RevalidatesPinnedSession/deleted_before_completion === RUN TestFilerServer_refreshTusSessionChunks_RevalidatesPinnedSession/replaced_before_completion --- PASS: TestFilerServer_refreshTusSessionChunks_RevalidatesPinnedSession (0.00s) --- PASS: TestFilerServer_refreshTusSessionChunks_RevalidatesPinnedSession/deleted_before_completion (0.00s) --- PASS: TestFilerServer_refreshTusSessionChunks_RevalidatesPinnedSession/replaced_before_completion (0.00s) === RUN TestMasterIsKnownPingTarget I0720 21:32:02.110168 node.go:471 test adds child dc1 I0720 21:32:02.110174 node.go:471 test:dc1 adds child rack1 I0720 21:32:02.110178 node.go:471 test:dc1:rack1 adds child vol1 === RUN TestMasterIsKnownPingTarget/known_filer === RUN TestMasterIsKnownPingTarget/known_volume_server === RUN TestMasterIsKnownPingTarget/known_volume_server_http_addr === RUN TestMasterIsKnownPingTarget/known_self_master === RUN TestMasterIsKnownPingTarget/unknown_localhost_low_port === RUN TestMasterIsKnownPingTarget/unknown_localhost_high_port === RUN TestMasterIsKnownPingTarget/unrelated_host === RUN TestMasterIsKnownPingTarget/unknown_target_type === RUN TestMasterIsKnownPingTarget/filer_address_checked_as_volume_server --- PASS: TestMasterIsKnownPingTarget (0.00s) --- PASS: TestMasterIsKnownPingTarget/known_filer (0.00s) --- PASS: TestMasterIsKnownPingTarget/known_volume_server (0.00s) --- PASS: TestMasterIsKnownPingTarget/known_volume_server_http_addr (0.00s) --- PASS: TestMasterIsKnownPingTarget/known_self_master (0.00s) --- PASS: TestMasterIsKnownPingTarget/unknown_localhost_low_port (0.00s) --- PASS: TestMasterIsKnownPingTarget/unknown_localhost_high_port (0.00s) --- PASS: TestMasterIsKnownPingTarget/unrelated_host (0.00s) --- PASS: TestMasterIsKnownPingTarget/unknown_target_type (0.00s) --- PASS: TestMasterIsKnownPingTarget/filer_address_checked_as_volume_server (0.00s) === RUN TestMasterIsKnownPingTargetFollowerEmptyTopology --- PASS: TestMasterIsKnownPingTargetFollowerEmptyTopology (0.00s) === RUN TestVolumeServerIsKnownPingTarget === RUN TestVolumeServerIsKnownPingTarget/seed_master === RUN TestVolumeServerIsKnownPingTarget/current_master === RUN TestVolumeServerIsKnownPingTarget/other_volume_server === RUN TestVolumeServerIsKnownPingTarget/random_filer === RUN TestVolumeServerIsKnownPingTarget/unknown_low_port === RUN TestVolumeServerIsKnownPingTarget/unknown_high_port === RUN TestVolumeServerIsKnownPingTarget/unrelated_host --- PASS: TestVolumeServerIsKnownPingTarget (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/seed_master (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/current_master (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/other_volume_server (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/random_filer (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/unknown_low_port (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/unknown_high_port (0.00s) --- PASS: TestVolumeServerIsKnownPingTarget/unrelated_host (0.00s) === RUN TestAssignShedsLoadWhenGrowthInFlight I0720 21:32:02.110307 node.go:471 test adds child dc1 I0720 21:32:02.110310 node.go:471 test:dc1 adds child rack1 I0720 21:32:02.110314 node.go:471 test:dc1:rack1 adds child dn1 I0720 21:32:02.110317 node.go:471 test:dc1:rack1:dn1 adds child --- PASS: TestAssignShedsLoadWhenGrowthInFlight (0.00s) === RUN TestAssignInitiatorWaitsForItsOwnGrowth I0720 21:32:02.110389 node.go:471 test adds child dc1 I0720 21:32:02.110391 node.go:471 test:dc1 adds child rack1 I0720 21:32:02.110393 node.go:471 test:dc1:rack1 adds child dn1 I0720 21:32:02.110396 node.go:471 test:dc1:rack1:dn1 adds child I0720 21:32:02.310454 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:32:02.310492 volume_layout.go:404 Volume 1 reaches full capacity (effective=1048576, limit=1048576). --- PASS: TestAssignInitiatorWaitsForItsOwnGrowth (0.20s) === RUN TestAssignInitiatorShedsWhenGrowthConcludesUnfulfilled I0720 21:32:02.310604 node.go:471 test adds child dc1 I0720 21:32:02.310613 node.go:471 test:dc1 adds child rack1 I0720 21:32:02.310620 node.go:471 test:dc1:rack1 adds child dn1 I0720 21:32:02.310625 node.go:471 test:dc1:rack1:dn1 adds child --- PASS: TestAssignInitiatorShedsWhenGrowthConcludesUnfulfilled (0.20s) === RUN TestAssignAbortsOnCancel I0720 21:32:02.511266 node.go:471 test adds child dc1 I0720 21:32:02.511273 node.go:471 test:dc1 adds child rack1 I0720 21:32:02.511280 node.go:471 test:dc1:rack1 adds child dn1 I0720 21:32:02.511285 node.go:471 test:dc1:rack1:dn1 adds child --- PASS: TestAssignAbortsOnCancel (0.30s) === RUN TestAssignFailsFastWhenOutOfSpace I0720 21:32:02.812428 master_grpc_server_assign.go:199 assign count:1 replication:"000" {"replication":{},"ttl":{"Count":0,"Unit":0},"version":3}: failed to find writable volumes for collection: replication:000 ttl: error: No writable volumes and no free volumes left for {"replication":{},"ttl":{"Count":0,"Unit":0},"version":3} --- PASS: TestAssignFailsFastWhenOutOfSpace (0.00s) === RUN TestRaftLeadershipTransfer_NoRaft --- PASS: TestRaftLeadershipTransfer_NoRaft (0.00s) === RUN TestRaftListClusterServers_NoRaft --- PASS: TestRaftListClusterServers_NoRaft (0.00s) === RUN TestRaftAddServer_NoRaft --- PASS: TestRaftAddServer_NoRaft (0.00s) === RUN TestRaftRemoveServer_NoRaft --- PASS: TestRaftRemoveServer_NoRaft (0.00s) === RUN TestInitialLockRingUpdateReturnsLastBroadcastForFilers I0720 21:32:02.812651 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "group-a" version 1784583122812641256: [filer1:8888 filer2:8888] --- PASS: TestInitialLockRingUpdateReturnsLastBroadcastForFilers (0.00s) === RUN TestInitialLockRingUpdateSkipsNonFilers I0720 21:32:02.812682 lock_ring_manager.go:139 LockRing: broadcasting ring update for group "group-a" version 1784583122812679028: [filer1:8888] --- PASS: TestInitialLockRingUpdateSkipsNonFilers (0.00s) === RUN TestBroadcastVolumeLocationsToClients --- PASS: TestBroadcastVolumeLocationsToClients (0.00s) === RUN TestRaftServerID === RUN TestRaftServerID/without_grpc_suffix === RUN TestRaftServerID/with_grpc_suffix --- PASS: TestRaftServerID (0.00s) --- PASS: TestRaftServerID/without_grpc_suffix (0.00s) --- PASS: TestRaftServerID/with_grpc_suffix (0.00s) === RUN TestGetPeerIdxUsesCanonicalID --- PASS: TestGetPeerIdxUsesCanonicalID (0.00s) === RUN TestAddPeersConfigurationUsesCanonicalIDs --- PASS: TestAddPeersConfigurationUsesCanonicalIDs (0.00s) === RUN TestBatchDelete_DeniesNonWhitelistedPeer I0720 21:32:02.812748 volume_grpc_admin.go:64 gRPC admin auth failed: 127.0.0.1 is not whitelisted (remote: 127.0.0.1:12345) --- PASS: TestBatchDelete_DeniesNonWhitelistedPeer (0.00s) === RUN TestLookupRaftLeaderMasterDoesNotUpdateCurrentMaster --- PASS: TestLookupRaftLeaderMasterDoesNotUpdateCurrentMaster (0.00s) === RUN TestVolumeCopy_KeepsExistingReplicaWhenSourceUnreachable I0720 21:32:02.813566 store.go:302 In dir /tmp/TestVolumeCopy_KeepsExistingReplicaWhenSourceUnreachable174353920/001 (disk ID 0) adds volume:42 collection: replicaPlacement:000 ttl: I0720 21:32:02.813740 store.go:307 add volume 42 on disk ID 0 --- PASS: TestVolumeCopy_KeepsExistingReplicaWhenSourceUnreachable (0.00s) === RUN TestWriteToFile_RemovesPartialFileOnStreamError --- PASS: TestWriteToFile_RemovesPartialFileOnStreamError (0.00s) === RUN TestWriteToFile_RemovesEmptyFileOnImmediateStreamError --- PASS: TestWriteToFile_RemovesEmptyFileOnImmediateStreamError (0.00s) === RUN TestWriteToFile_PreservesAppendModeOnError --- PASS: TestWriteToFile_PreservesAppendModeOnError (0.00s) === RUN TestWriteToFile_SucceedsOnCleanStream --- PASS: TestWriteToFile_SucceedsOnCleanStream (0.00s) === RUN TestWriteToFile_PreservesDestinationWhenOptionalSourceMissing --- PASS: TestWriteToFile_PreservesDestinationWhenOptionalSourceMissing (0.00s) === RUN TestWriteToFile_CommitsOptionalCopyWhenSourcePresent --- PASS: TestWriteToFile_CommitsOptionalCopyWhenSourcePresent (0.00s) === RUN TestFullTeardownFencedByGeneration === RUN TestFullTeardownFencedByGeneration/older_disk_wiped I0720 21:32:02.814731 store_ec.go:203 MountEcShards 55.0 on disk ID 0 I0720 21:32:02.814749 store_ec.go:203 MountEcShards 55.1 on disk ID 0 I0720 21:32:02.814759 volume_grpc_erasure_coding.go:452 ec volume ec-fence_55 full teardown fenced at generation 200 === RUN TestFullTeardownFencedByGeneration/newer_disk_preserved I0720 21:32:02.815161 store_ec.go:203 MountEcShards 55.0 on disk ID 0 I0720 21:32:02.815175 store_ec.go:203 MountEcShards 55.1 on disk ID 0 I0720 21:32:02.815181 volume_grpc_erasure_coding.go:452 ec volume ec-fence_55 full teardown fenced at generation 100 === RUN TestFullTeardownFencedByGeneration/zero_gen_preserved I0720 21:32:02.815427 store_ec.go:203 MountEcShards 55.0 on disk ID 0 I0720 21:32:02.815442 store_ec.go:203 MountEcShards 55.1 on disk ID 0 I0720 21:32:02.815447 volume_grpc_erasure_coding.go:452 ec volume ec-fence_55 full teardown fenced at generation 200 === RUN TestFullTeardownFencedByGeneration/zero_request_blanket_wipe I0720 21:32:02.815744 store_ec.go:203 MountEcShards 55.0 on disk ID 0 I0720 21:32:02.815761 store_ec.go:203 MountEcShards 55.1 on disk ID 0 I0720 21:32:02.815767 volume_grpc_erasure_coding.go:436 ec volume ec-fence_55 full teardown --- PASS: TestFullTeardownFencedByGeneration (0.00s) --- PASS: TestFullTeardownFencedByGeneration/older_disk_wiped (0.00s) --- PASS: TestFullTeardownFencedByGeneration/newer_disk_preserved (0.00s) --- PASS: TestFullTeardownFencedByGeneration/zero_gen_preserved (0.00s) --- PASS: TestFullTeardownFencedByGeneration/zero_request_blanket_wipe (0.00s) === RUN TestUnmountEcShardsFencedByGeneration I0720 21:32:02.816143 store_ec.go:203 MountEcShards 56.0 on disk ID 0 I0720 21:32:02.816156 store_ec.go:203 MountEcShards 56.1 on disk ID 0 I0720 21:32:02.816161 volume_grpc_erasure_coding.go:1012 VolumeEcShardsInfo: volume 56 I0720 21:32:02.816169 store_ec.go:293 UnmountEcShards 56.1 disk_id:0 I0720 21:32:02.816171 volume_grpc_erasure_coding.go:1012 VolumeEcShardsInfo: volume 56 --- PASS: TestUnmountEcShardsFencedByGeneration (0.00s) === RUN TestReadEcGenerationTsNs W0720 21:32:02.816344 volume_info.go:54 unmarshal error: proto: syntax error (line 1:1): invalid value not-a-valid-vif oldFormat: failed to unmarshal old version volume info: proto: syntax error (line 1:1): invalid value not-a-valid-vif --- PASS: TestReadEcGenerationTsNs (0.00s) === RUN TestEcShardDeleteKeepsSharedIndexWhileSiblingHasShards I0720 21:32:02.816714 volume_grpc_erasure_coding.go:469 ec volume ec-shared-index_77 shard delete [0 5] I0720 21:32:02.816781 volume_grpc_erasure_coding.go:469 ec volume ec-shared-index_77 shard delete [7 12] --- PASS: TestEcShardDeleteKeepsSharedIndexWhileSiblingHasShards (0.00s) === RUN TestEcShardDeleteKeepsSharedIdxSidecarWhileSiblingHasShards I0720 21:32:02.817084 volume_grpc_erasure_coding.go:469 ec volume ec-shared-idx-sidecar_78 shard delete [0] I0720 21:32:02.817161 volume_grpc_erasure_coding.go:469 ec volume ec-shared-idx-sidecar_78 shard delete [7] --- PASS: TestEcShardDeleteKeepsSharedIdxSidecarWhileSiblingHasShards (0.00s) === RUN TestFetchEcIndexFromPeers_CopiesIndexOverGrpc I0720 21:32:02.818542 volume_grpc_erasure_coding_recover.go:164 ec volume 6190: fetched missing .ecx from 127.0.0.1:33079 into /tmp/TestFetchEcIndexFromPeers_CopiesIndexOverGrpc2677968672/002/dst --- PASS: TestFetchEcIndexFromPeers_CopiesIndexOverGrpc (0.00s) === RUN TestFetchEcIndexFromPeers_SkipsPeerWithoutIndex --- PASS: TestFetchEcIndexFromPeers_SkipsPeerWithoutIndex (0.00s) === RUN TestVolumeEcShardsMount_RecoverMissingIndex I0720 21:32:02.819532 volume_grpc_erasure_coding.go:744 VolumeEcShardsMount: recover_missing_index:true I0720 21:32:02.820753 volume_grpc_erasure_coding_recover.go:164 ec volume 6190: fetched missing .ecx from 127.0.0.1:1 into /tmp/TestVolumeEcShardsMount_RecoverMissingIndex3757519228/002/dst W0720 21:32:02.820844 volume_info.go:54 unmarshal error: proto: syntax error (line 1:1): invalid value volinfo oldFormat: failed to unmarshal old version volume info: proto: syntax error (line 1:1): invalid value volinfo I0720 21:32:02.820876 volume_grpc_erasure_coding_recover.go:74 recovered missing EC index for 1 volume(s) from peers and mounted their shards --- PASS: TestVolumeEcShardsMount_RecoverMissingIndex (0.00s) === RUN TestCheckEcVolumeStatusCountOnlyDataShards --- PASS: TestCheckEcVolumeStatusCountOnlyDataShards (0.00s) === RUN TestRemoveStaleEcArtifacts --- PASS: TestRemoveStaleEcArtifacts (0.00s) === RUN TestDeleteEcShardsWithoutLocalEcx --- PASS: TestDeleteEcShardsWithoutLocalEcx (0.00s) === RUN TestVolumeEcShardsInfo_AggregatesAcrossDisks I0720 21:32:02.822004 store_ec.go:203 MountEcShards 42.0 on disk ID 0 I0720 21:32:02.822021 store_ec.go:203 MountEcShards 42.5 on disk ID 0 I0720 21:32:02.822074 store_ec.go:203 MountEcShards 42.7 on disk ID 1 I0720 21:32:02.822092 store_ec.go:203 MountEcShards 42.12 on disk ID 1 I0720 21:32:02.822095 volume_grpc_erasure_coding.go:1012 VolumeEcShardsInfo: volume 42 --- PASS: TestVolumeEcShardsInfo_AggregatesAcrossDisks (0.00s) === RUN TestValidateRemoteEndpoint === RUN TestValidateRemoteEndpoint/empty === RUN TestValidateRemoteEndpoint/loopback_literal === RUN TestValidateRemoteEndpoint/ipv6_loopback === RUN TestValidateRemoteEndpoint/imds_ipv4 === RUN TestValidateRemoteEndpoint/unspecified_ipv4 === RUN TestValidateRemoteEndpoint/link-local_ipv6 === RUN TestValidateRemoteEndpoint/ftp_scheme === RUN TestValidateRemoteEndpoint/missing_scheme === RUN TestValidateRemoteEndpoint/imds_hostname === RUN TestValidateRemoteEndpoint/imds_short_hostname === RUN TestValidateRemoteEndpoint/host_resolves_to_loopback === RUN TestValidateRemoteEndpoint/host_resolves_to_link-local === RUN TestValidateRemoteEndpoint/rfc1918_10/8_literal === RUN TestValidateRemoteEndpoint/rfc1918_172.16/12_literal === RUN TestValidateRemoteEndpoint/rfc1918_192.168/16_literal === RUN TestValidateRemoteEndpoint/cgnat_literal === RUN TestValidateRemoteEndpoint/host_resolves_to_rfc1918_10/8 === RUN TestValidateRemoteEndpoint/host_resolves_to_rfc1918_172/12 === RUN TestValidateRemoteEndpoint/host_resolves_to_rfc1918_192.168/16 === RUN TestValidateRemoteEndpoint/host_resolves_to_cgnat === RUN TestValidateRemoteEndpoint/public_s3 --- PASS: TestValidateRemoteEndpoint (0.00s) --- PASS: TestValidateRemoteEndpoint/empty (0.00s) --- PASS: TestValidateRemoteEndpoint/loopback_literal (0.00s) --- PASS: TestValidateRemoteEndpoint/ipv6_loopback (0.00s) --- PASS: TestValidateRemoteEndpoint/imds_ipv4 (0.00s) --- PASS: TestValidateRemoteEndpoint/unspecified_ipv4 (0.00s) --- PASS: TestValidateRemoteEndpoint/link-local_ipv6 (0.00s) --- PASS: TestValidateRemoteEndpoint/ftp_scheme (0.00s) --- PASS: TestValidateRemoteEndpoint/missing_scheme (0.00s) --- PASS: TestValidateRemoteEndpoint/imds_hostname (0.00s) --- PASS: TestValidateRemoteEndpoint/imds_short_hostname (0.00s) --- PASS: TestValidateRemoteEndpoint/host_resolves_to_loopback (0.00s) --- PASS: TestValidateRemoteEndpoint/host_resolves_to_link-local (0.00s) --- PASS: TestValidateRemoteEndpoint/rfc1918_10/8_literal (0.00s) --- PASS: TestValidateRemoteEndpoint/rfc1918_172.16/12_literal (0.00s) --- PASS: TestValidateRemoteEndpoint/rfc1918_192.168/16_literal (0.00s) --- PASS: TestValidateRemoteEndpoint/cgnat_literal (0.00s) --- PASS: TestValidateRemoteEndpoint/host_resolves_to_rfc1918_10/8 (0.00s) --- PASS: TestValidateRemoteEndpoint/host_resolves_to_rfc1918_172/12 (0.00s) --- PASS: TestValidateRemoteEndpoint/host_resolves_to_rfc1918_192.168/16 (0.00s) --- PASS: TestValidateRemoteEndpoint/host_resolves_to_cgnat (0.00s) --- PASS: TestValidateRemoteEndpoint/public_s3 (0.00s) === RUN TestValidateRemoteEndpointResolverFailure --- PASS: TestValidateRemoteEndpointResolverFailure (0.00s) === RUN TestGuardedDialerRebind --- PASS: TestGuardedDialerRebind (0.00s) === RUN TestGuardedDialerLiteralBlocked --- PASS: TestGuardedDialerLiteralBlocked (0.00s) === RUN TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal I0720 21:32:02.822465 store.go:302 In dir /tmp/TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal1600344672/002 (disk ID 0) adds volume:71 collection: replicaPlacement:000 ttl: I0720 21:32:02.822516 store.go:307 add volume 71 on disk ID 0 I0720 21:32:02.822600 store.go:807 UnmountVolume 71 disk_id:0 I0720 21:32:02.822714 volume_tier.go:31 volume 71 is tiered to tier_test_local_dir.default as obj-1784583122822535865 and read only I0720 21:32:02.822718 volume_loading.go:163 loading volume 71 from remote files:{backend_type:"tier_test_local_dir" backend_id:"default" key:"obj-1784583122822535865" file_size:288 modified_time:1784583122 extension:".dat"} version:3 bytes_offset:4 E0720 21:32:02.822734 volume_loading.go:229 readSuperBlock remote volume 71: I0720 21:32:02.822885 needle_map_sorted_file.go:26 Start to Generate /tmp/TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal1600344672/002/71.sdx from /tmp/TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal1600344672/002/71.idx I0720 21:32:02.823166 needle_map_sorted_file.go:28 Finished Generating /tmp/TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal1600344672/002/71.sdx from /tmp/TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal1600344672/002/71.idx I0720 21:32:02.823257 store.go:807 UnmountVolume 71 disk_id:0 I0720 21:32:02.823316 store.go:757 mount volume 71 --- PASS: TestTierMoveDatFromRemote_KeepRemote_LeavesReplicaLocal (0.00s) === RUN TestVolumeTierMoveDatPreservesModifiedTime I0720 21:32:02.823476 store.go:302 In dir /tmp/TestVolumeTierMoveDatPreservesModifiedTime202904526/001 (disk ID 0) adds volume:1 collection: replicaPlacement:000 ttl: I0720 21:32:02.823533 store.go:307 add volume 1 on disk ID 0 --- PASS: TestVolumeTierMoveDatPreservesModifiedTime (0.00s) === RUN TestDeleteHandlerRejectsOverflowingNeedleDelta I0720 21:32:02.823714 common.go:109 response method:DELETE URL:/1,ffffffffffffffff00000000_1 with httpStatus:400 and JSON:{"error":"delta 1 overflows needle id 66666666666666666666666666666666"} --- PASS: TestDeleteHandlerRejectsOverflowingNeedleDelta (0.00s) === RUN TestBatchDeleteRejectsOverflowingNeedleDelta --- PASS: TestBatchDeleteRejectsOverflowingNeedleDelta (0.00s) === RUN TestQueryRejectsOverflowingNeedleDelta I0720 21:32:02.823743 volume_grpc_query.go:25 volume query failed to parse fid 1,ffffffffffffffff00000000_1: delta 1 overflows needle id 66666666666666666666666666666666 --- PASS: TestQueryRejectsOverflowingNeedleDelta (0.00s) === RUN TestWrappedFsConfine --- PASS: TestWrappedFsConfine (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/server 120.555s ? github.com/seaweedfs/seaweedfs/weed/server/constants [no test files] === RUN TestToBreadcrumb === RUN TestToBreadcrumb/empty === RUN TestToBreadcrumb/test1 === RUN TestToBreadcrumb/test2 === RUN TestToBreadcrumb/test3 --- PASS: TestToBreadcrumb (0.00s) --- PASS: TestToBreadcrumb/empty (0.00s) --- PASS: TestToBreadcrumb/test1 (0.00s) --- PASS: TestToBreadcrumb/test2 (0.00s) --- PASS: TestToBreadcrumb/test3 (0.00s) === RUN TestPrintPathPreservesPathSafeTimestampChars --- PASS: TestPrintPathPreservesPathSafeTimestampChars (0.00s) === RUN TestQueryEscapePreservesPlusInLoadMoreCursor --- PASS: TestQueryEscapePreservesPlusInLoadMoreCursor (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/server/filer_ui 0.018s ? github.com/seaweedfs/seaweedfs/weed/server/master_ui [no test files] ? github.com/seaweedfs/seaweedfs/weed/server/postgres [no test files] === RUN TestStatusTpl --- PASS: TestStatusTpl (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/server/volume_server_ui 0.019s === RUN TestPathWithin --- PASS: TestPathWithin (0.00s) === RUN TestHasExplicitPermissionSiblingPrefix --- PASS: TestHasExplicitPermissionSiblingPrefix (0.00s) === RUN TestIsPathInHomeDirectorySiblingPrefix --- PASS: TestIsPathInHomeDirectorySiblingPrefix (0.00s) === RUN TestToAbsolutePath === RUN TestToAbsolutePath/normal_path === RUN TestToAbsolutePath/root_path === RUN TestToAbsolutePath/path_with_dot === RUN TestToAbsolutePath/path_traversal_attempts === RUN TestToAbsolutePath/path_traversal_attempts_2 === RUN TestToAbsolutePath/path_traversal_attempts_3 === RUN TestToAbsolutePath/empty_path === RUN TestToAbsolutePath/multiple_slashes === RUN TestToAbsolutePath/trailing_slash === RUN TestToAbsolutePath/empty_HomeDir_passthrough === RUN TestToAbsolutePath/root_HomeDir_passthrough --- PASS: TestToAbsolutePath (0.00s) --- PASS: TestToAbsolutePath/normal_path (0.00s) --- PASS: TestToAbsolutePath/root_path (0.00s) --- PASS: TestToAbsolutePath/path_with_dot (0.00s) --- PASS: TestToAbsolutePath/path_traversal_attempts (0.00s) --- PASS: TestToAbsolutePath/path_traversal_attempts_2 (0.00s) --- PASS: TestToAbsolutePath/path_traversal_attempts_3 (0.00s) --- PASS: TestToAbsolutePath/empty_path (0.00s) --- PASS: TestToAbsolutePath/multiple_slashes (0.00s) --- PASS: TestToAbsolutePath/trailing_slash (0.00s) --- PASS: TestToAbsolutePath/empty_HomeDir_passthrough (0.00s) --- PASS: TestToAbsolutePath/root_HomeDir_passthrough (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/sftpd 0.099s === RUN TestManager_CertificateTakesOverPublicKeyChannel --- PASS: TestManager_CertificateTakesOverPublicKeyChannel (0.01s) === RUN TestManager_PublicKeyOnlyChannel --- PASS: TestManager_PublicKeyOnlyChannel (0.00s) === RUN TestManager_CertificateRequiresCAFile --- PASS: TestManager_CertificateRequiresCAFile (0.00s) === RUN TestCertificateAuthenticator_GoldenPath --- PASS: TestCertificateAuthenticator_GoldenPath (0.00s) === RUN TestCertificateAuthenticator_RejectsPlainPublicKey --- PASS: TestCertificateAuthenticator_RejectsPlainPublicKey (0.00s) === RUN TestCertificateAuthenticator_RejectsHostCert --- PASS: TestCertificateAuthenticator_RejectsHostCert (0.01s) === RUN TestCertificateAuthenticator_RejectsEmptyPrincipals --- PASS: TestCertificateAuthenticator_RejectsEmptyPrincipals (0.00s) === RUN TestCertificateAuthenticator_RejectsWrongPrincipal --- PASS: TestCertificateAuthenticator_RejectsWrongPrincipal (0.00s) === RUN TestCertificateAuthenticator_RejectsExpiredCert --- PASS: TestCertificateAuthenticator_RejectsExpiredCert (0.00s) === RUN TestCertificateAuthenticator_RejectsUnknownCA --- PASS: TestCertificateAuthenticator_RejectsUnknownCA (0.00s) === RUN TestCertificateAuthenticator_RejectsUnknownUser --- PASS: TestCertificateAuthenticator_RejectsUnknownUser (0.00s) === RUN TestCertificateAuthenticator_LoadsMultipleCAKeys --- PASS: TestCertificateAuthenticator_LoadsMultipleCAKeys (0.00s) === RUN TestNewCertificateAuthenticator_DisabledIgnoresFile --- PASS: TestNewCertificateAuthenticator_DisabledIgnoresFile (0.00s) === RUN TestNewCertificateAuthenticator_EnabledRequiresFile --- PASS: TestNewCertificateAuthenticator_EnabledRequiresFile (0.00s) === RUN TestNewCertificateAuthenticator_MissingFile --- PASS: TestNewCertificateAuthenticator_MissingFile (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/sftpd/auth 0.037s ? github.com/seaweedfs/seaweedfs/weed/sftpd/user [no test files] ? github.com/seaweedfs/seaweedfs/weed/sftpd/utils [no test files] === RUN TestRaftLeaderTransfer_Name --- PASS: TestRaftLeaderTransfer_Name (0.00s) === RUN TestRaftLeaderTransfer_Help --- PASS: TestRaftLeaderTransfer_Help (0.00s) === RUN TestRaftLeaderTransfer_HasTag --- PASS: TestRaftLeaderTransfer_HasTag (0.00s) === RUN TestRaftLeaderTransfer_ValidateTargetIdWithoutAddress --- PASS: TestRaftLeaderTransfer_ValidateTargetIdWithoutAddress (0.00s) === RUN TestRaftLeaderTransfer_ValidateTargetAddressWithoutId --- PASS: TestRaftLeaderTransfer_ValidateTargetAddressWithoutId (0.00s) === RUN TestRaftLeaderTransfer_UnknownFlag flag provided but not defined: -unknown-flag Usage of cluster.raft.transferLeader: -address string target server grpc address (must be used with -id) -id string target server id (must be used with -address) --- PASS: TestRaftLeaderTransfer_UnknownFlag (0.00s) === RUN TestPrintClusterInfo --- PASS: TestPrintClusterInfo (0.00s) === RUN TestPrintClusterInfo_multiDiskPerNode --- PASS: TestPrintClusterInfo_multiDiskPerNode (0.00s) === RUN TestPrintVolumeInfo --- PASS: TestPrintVolumeInfo (0.01s) === RUN TestPrintStorageInfo --- PASS: TestPrintStorageInfo (0.01s) === RUN TestPrintFilesInfo --- PASS: TestPrintFilesInfo (0.00s) === RUN TestCollectCollectionInfoEC --- PASS: TestCollectCollectionInfoEC (0.00s) === RUN TestCollectCollectionInfoMixed --- PASS: TestCollectCollectionInfoMixed (0.00s) === RUN TestCollectCollectionInfoRegularVolumeDedupesReplicas --- PASS: TestCollectCollectionInfoRegularVolumeDedupesReplicas (0.00s) === RUN TestEcCheckReplication --- PASS: TestEcCheckReplication (0.00s) === RUN TestEcCheckReplicationNoVolumes === RUN TestEcCheckReplicationNoVolumes/unfiltered === RUN TestEcCheckReplicationNoVolumes/filtered_no_match --- PASS: TestEcCheckReplicationNoVolumes (0.00s) --- PASS: TestEcCheckReplicationNoVolumes/unfiltered (0.00s) --- PASS: TestEcCheckReplicationNoVolumes/filtered_no_match (0.00s) === RUN TestEcCheckReplicationUnexpectedShard --- PASS: TestEcCheckReplicationUnexpectedShard (0.00s) === RUN TestEcCheckReplicationMixedAnomalies --- PASS: TestEcCheckReplicationMixedAnomalies (0.00s) === RUN TestCountFreeShardSlotsPhysicalDiskGate --- PASS: TestCountFreeShardSlotsPhysicalDiskGate (0.00s) === RUN TestAssertEncodableRegularVolumes === RUN TestAssertEncodableRegularVolumes/regular_volume === RUN TestAssertEncodableRegularVolumes/already_EC === RUN TestAssertEncodableRegularVolumes/regular_plus_stale_orphan_shards === RUN TestAssertEncodableRegularVolumes/unknown_id --- PASS: TestAssertEncodableRegularVolumes (0.00s) --- PASS: TestAssertEncodableRegularVolumes/regular_volume (0.00s) --- PASS: TestAssertEncodableRegularVolumes/already_EC (0.00s) --- PASS: TestAssertEncodableRegularVolumes/regular_plus_stale_orphan_shards (0.00s) --- PASS: TestAssertEncodableRegularVolumes/unknown_id (0.00s) === RUN TestPrepareDataToRecover_DryRunRecoverableNoSideEffects --- PASS: TestPrepareDataToRecover_DryRunRecoverableNoSideEffects (0.00s) === RUN TestPrepareDataToRecover_UnionsLocalShardsAcrossDisks --- PASS: TestPrepareDataToRecover_UnionsLocalShardsAcrossDisks (0.00s) === RUN TestCountLocalShards_UnionsAcrossDisks --- PASS: TestCountLocalShards_UnionsAcrossDisks (0.00s) === RUN TestPrepareDataToRecover_SelfSourceNotCopied --- PASS: TestPrepareDataToRecover_SelfSourceNotCopied (0.00s) === RUN TestPrepareDataToRecover_ApplyCopyFailureDoesNotCountAsRecoverable --- PASS: TestPrepareDataToRecover_ApplyCopyFailureDoesNotCountAsRecoverable (0.00s) === RUN TestSelectVolumeIdsFromTopology skip volume 1 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 2 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 3 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 4 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 5 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 6 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 7 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 8 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 9 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) I0720 21:30:02.551666 command_ec_encode.go:1034 replica test 10 on seaweedfs-volume-0.seaweedfs-volume.sea:8080 has no free disk skip replica of volume 10 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: insufficient free disk space (free volumes: 1, required: 2) I0720 21:30:02.551849 command_ec_encode.go:1034 replica test 11 on seaweedfs-volume-0.seaweedfs-volume.sea:8080 has no free disk skip replica of volume 11 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: insufficient free disk space (free volumes: 1, required: 2) I0720 21:30:02.551853 command_ec_encode.go:1034 replica test 12 on seaweedfs-volume-0.seaweedfs-volume.sea:8080 has no free disk skip replica of volume 12 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: insufficient free disk space (free volumes: 1, required: 2) skip volume 13 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 544.0 MB, threshold: 950.0 MB, 54.4% full) skip volume 14 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 456.0 MB, threshold: 950.0 MB, 45.6% full) skip volume 1 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 2 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 3 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 4 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 5 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 6 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 7 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 8 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 9 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) I0720 21:30:02.551869 command_ec_encode.go:1034 replica test 10 on seaweedfs-volume-1.seaweedfs-volume.sea:8080 has no free disk skip replica of volume 10 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: insufficient free disk space (free volumes: 1, required: 2) I0720 21:30:02.551872 command_ec_encode.go:1034 replica test 11 on seaweedfs-volume-1.seaweedfs-volume.sea:8080 has no free disk skip replica of volume 11 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: insufficient free disk space (free volumes: 1, required: 2) I0720 21:30:02.551875 command_ec_encode.go:1034 replica test 12 on seaweedfs-volume-1.seaweedfs-volume.sea:8080 has no free disk skip replica of volume 12 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: insufficient free disk space (free volumes: 1, required: 2) skip volume 13 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 544.0 MB, threshold: 950.0 MB, 54.4% full) skip volume 14 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 456.0 MB, threshold: 950.0 MB, 45.6% full) Volume selection summary: Total volumes examined: 28 Selected for encoding: 0 Collections matched: [ encrypt-data test] Reasons for exclusion: Too small (< 95.0% full): 22 Insufficient free disk space: 3 skip volume 1 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 2 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 3 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 4 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 5 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 6 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 7 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 8 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 9 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) selected volume 10 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: size 1000.0 MB (100.0% full), last modified 5025 seconds ago, free volumes: 2 selected volume 11 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: size 1000.0 MB (100.0% full), last modified 4966 seconds ago, free volumes: 2 selected volume 12 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: size 1000.0 MB (100.0% full), last modified 4933 seconds ago, free volumes: 2 skip volume 13 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 544.0 MB, threshold: 950.0 MB, 54.4% full) skip volume 14 on seaweedfs-volume-0.seaweedfs-volume.sea:8080: too small (size: 456.0 MB, threshold: 950.0 MB, 45.6% full) skip volume 1 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 2 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 3 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 4 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 5 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 6 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.1 MB, threshold: 950.0 MB, 0.0% full) skip volume 7 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 8 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) skip volume 9 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 0.0 MB, threshold: 950.0 MB, 0.0% full) selected volume 10 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: size 1000.0 MB (100.0% full), last modified 5025 seconds ago, free volumes: 2 selected volume 11 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: size 1000.0 MB (100.0% full), last modified 4966 seconds ago, free volumes: 2 selected volume 12 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: size 1000.0 MB (100.0% full), last modified 4934 seconds ago, free volumes: 2 skip volume 13 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 544.0 MB, threshold: 950.0 MB, 54.4% full) skip volume 14 on seaweedfs-volume-1.seaweedfs-volume.sea:8080: too small (size: 456.0 MB, threshold: 950.0 MB, 45.6% full) Volume selection summary: Total volumes examined: 28 Selected for encoding: 3 Collections matched: [ encrypt-data test] Reasons for exclusion: Too small (< 95.0% full): 22 --- PASS: TestSelectVolumeIdsFromTopology (0.00s) === RUN TestEcEncodeNodeCountCheck --- PASS: TestEcEncodeNodeCountCheck (0.00s) === RUN TestParseEcEncodeVolumeIds --- PASS: TestParseEcEncodeVolumeIds (0.00s) === RUN TestChunkVolumeIds --- PASS: TestChunkVolumeIds (0.00s) === RUN TestCollectEcShardBitsByNode --- PASS: TestCollectEcShardBitsByNode (0.00s) === RUN TestCollectEcShardBitsByNode_MixedGenerations --- PASS: TestCollectEcShardBitsByNode_MixedGenerations (0.00s) === RUN TestEcShardsClumpedOnOneNode --- PASS: TestEcShardsClumpedOnOneNode (0.00s) === RUN TestEcShardMapRegister --- PASS: TestEcShardMapRegister (0.00s) === RUN TestEcShardMapShardCount === RUN TestEcShardMapShardCount/all_shards === RUN TestEcShardMapShardCount/data_shards_only === RUN TestEcShardMapShardCount/parity_shards_only === RUN TestEcShardMapShardCount/missing_some_shards === RUN TestEcShardMapShardCount/single_shard === RUN TestEcShardMapShardCount/no_shards --- PASS: TestEcShardMapShardCount (0.00s) --- PASS: TestEcShardMapShardCount/all_shards (0.00s) --- PASS: TestEcShardMapShardCount/data_shards_only (0.00s) --- PASS: TestEcShardMapShardCount/parity_shards_only (0.00s) --- PASS: TestEcShardMapShardCount/missing_some_shards (0.00s) --- PASS: TestEcShardMapShardCount/single_shard (0.00s) --- PASS: TestEcShardMapShardCount/no_shards (0.00s) === RUN TestRebuildEcVolumesInsufficientShards rebuildEcVolumes for "c1" --- PASS: TestRebuildEcVolumesInsufficientShards (0.00s) === RUN TestRebuildEcVolumesCompleteVolume rebuildEcVolumes for "c1" --- PASS: TestRebuildEcVolumesCompleteVolume (0.00s) === RUN TestRebuildEcVolumesInsufficientSpace rebuildEcVolumes for "c1" --- PASS: TestRebuildEcVolumesInsufficientSpace (0.00s) === RUN TestMultipleNodesWithShards --- PASS: TestMultipleNodesWithShards (0.00s) === RUN TestDuplicateShards --- PASS: TestDuplicateShards (0.00s) === RUN TestPrepareDataToRecoverTargetShardCount --- PASS: TestPrepareDataToRecoverTargetShardCount (0.00s) === RUN TestRebuilderVolumeIdMatches === RUN TestRebuilderVolumeIdMatches/no_target_volume_IDs_(nil) === RUN TestRebuilderVolumeIdMatches/no_target_volume_IDs === RUN TestRebuilderVolumeIdMatches/volume_ID_matches === RUN TestRebuilderVolumeIdMatches/volume_ID_mismatch --- PASS: TestRebuilderVolumeIdMatches (0.00s) --- PASS: TestRebuilderVolumeIdMatches/no_target_volume_IDs_(nil) (0.00s) --- PASS: TestRebuilderVolumeIdMatches/no_target_volume_IDs (0.00s) --- PASS: TestRebuilderVolumeIdMatches/volume_ID_matches (0.00s) --- PASS: TestRebuilderVolumeIdMatches/volume_ID_mismatch (0.00s) === RUN TestEcShardsFromString === RUN TestEcShardsFromString/single_id === RUN TestEcShardsFromString/list_of_ids === RUN TestEcShardsFromString/qualified_keeps_host:port === RUN TestEcShardsFromString/mixed_bare_and_qualified === RUN TestEcShardsFromString/colon_without_node_marker_is_not_a_shard_ID === RUN TestEcShardsFromString/shard_id_within_MaxShardCount === RUN TestEcShardsFromString/shard_id_at_MaxShardCount_is_rejected === RUN TestEcShardsFromString/negative_shard_id === RUN TestEcShardsFromString/out-of-range_shard_id_does_not_wrap === RUN TestEcShardsFromString/non-numeric_shard_id === RUN TestEcShardsFromString/empty_element --- PASS: TestEcShardsFromString (0.00s) --- PASS: TestEcShardsFromString/single_id (0.00s) --- PASS: TestEcShardsFromString/list_of_ids (0.00s) --- PASS: TestEcShardsFromString/qualified_keeps_host:port (0.00s) --- PASS: TestEcShardsFromString/mixed_bare_and_qualified (0.00s) --- PASS: TestEcShardsFromString/colon_without_node_marker_is_not_a_shard_ID (0.00s) --- PASS: TestEcShardsFromString/shard_id_within_MaxShardCount (0.00s) --- PASS: TestEcShardsFromString/shard_id_at_MaxShardCount_is_rejected (0.00s) --- PASS: TestEcShardsFromString/negative_shard_id (0.00s) --- PASS: TestEcShardsFromString/out-of-range_shard_id_does_not_wrap (0.00s) --- PASS: TestEcShardsFromString/non-numeric_shard_id (0.00s) --- PASS: TestEcShardsFromString/empty_element (0.00s) === RUN TestEcShardStringRoundTrips --- PASS: TestEcShardStringRoundTrips (0.00s) === RUN TestPickBestDiskOnNodeSelectsPhysicalDiskZero --- PASS: TestPickBestDiskOnNodeSelectsPhysicalDiskZero (0.00s) === RUN TestCommandEcBalanceSmall dn1 moves ec shard 1.0 to dn2 dn1 moves ec shard 1.1 to dn2 dn1 moves ec shard 1.2 to dn2 dn1 moves ec shard 1.3 to dn2 dn1 moves ec shard 1.4 to dn2 dn1 moves ec shard 1.10 to dn2 dn1 moves ec shard 1.11 to dn2 dn2 moves ec shard 2.0 to dn1 dn2 moves ec shard 2.1 to dn1 dn2 moves ec shard 2.2 to dn1 dn2 moves ec shard 2.3 to dn1 dn2 moves ec shard 2.4 to dn1 dn2 moves ec shard 2.10 to dn1 dn2 moves ec shard 2.11 to dn1 --- PASS: TestCommandEcBalanceSmall (0.00s) === RUN TestCommandEcBalanceNothingToMove dn1 moves ec shard 1.0 to dn2 dn1 moves ec shard 1.1 to dn2 dn2 moves ec shard 1.10 to dn1 dn2 moves ec shard 1.11 to dn1 dn2 moves ec shard 2.0 to dn1 dn2 moves ec shard 2.1 to dn1 dn1 moves ec shard 2.10 to dn2 dn1 moves ec shard 2.11 to dn2 --- PASS: TestCommandEcBalanceNothingToMove (0.00s) === RUN TestCommandEcBalanceAddNewServers dn1 moves ec shard 1.0 to dn3 dn1 moves ec shard 1.1 to dn4 dn1 moves ec shard 1.2 to dn3 dn1 moves ec shard 1.3 to dn4 dn2 moves ec shard 1.10 to dn1 dn2 moves ec shard 1.11 to dn3 dn2 moves ec shard 1.12 to dn4 dn2 moves ec shard 2.0 to dn3 dn2 moves ec shard 2.1 to dn4 dn2 moves ec shard 2.2 to dn3 dn2 moves ec shard 2.3 to dn4 dn1 moves ec shard 2.10 to dn2 dn1 moves ec shard 2.11 to dn3 dn1 moves ec shard 2.12 to dn4 dn1 moves ec shard 1.4 to dn3 dn2 moves ec shard 1.7 to dn4 --- PASS: TestCommandEcBalanceAddNewServers (0.00s) === RUN TestCommandEcBalanceAddNewRacks dn1 moves ec shard 1.0 to dn3 dn1 moves ec shard 1.1 to dn4 dn1 moves ec shard 1.2 to dn3 dn1 moves ec shard 1.3 to dn4 dn1 moves ec shard 1.4 to dn3 dn2 moves ec shard 1.10 to dn4 dn2 moves ec shard 1.11 to dn3 dn2 moves ec shard 2.0 to dn3 dn2 moves ec shard 2.1 to dn4 dn2 moves ec shard 2.2 to dn3 dn2 moves ec shard 2.3 to dn4 dn2 moves ec shard 2.4 to dn3 dn1 moves ec shard 2.10 to dn4 dn1 moves ec shard 2.11 to dn3 dn2 moves ec shard 1.12 to dn1 dn1 moves ec shard 2.12 to dn2 dn3 moves ec shard 1.0 to dn4 --- PASS: TestCommandEcBalanceAddNewRacks (0.00s) === RUN TestCommandEcBalanceVolumeEvenButRackUneven dn_shared moves ec shard 1.0 to dn3 --- PASS: TestCommandEcBalanceVolumeEvenButRackUneven (0.00s) === RUN TestCommandEcBalanceEvenDataAndParityDistribution dn1 moves ec shard 1.0 to dn2 dn1 moves ec shard 1.1 to dn3 dn1 moves ec shard 1.2 to dn4 dn1 moves ec shard 1.3 to dn5 dn1 moves ec shard 1.4 to dn6 dn1 moves ec shard 1.5 to dn2 dn1 moves ec shard 1.6 to dn3 dn1 moves ec shard 1.7 to dn4 dn1 moves ec shard 1.10 to dn2 dn1 moves ec shard 1.11 to dn3 dn1 moves ec shard 1.12 to dn4 command_ec_test.go:238: Distribution after balancing: command_ec_test.go:239: Data shards per rack: map[rack1:2 rack2:2 rack3:2 rack4:2 rack5:1 rack6:1] (max allowed: 2) command_ec_test.go:240: Parity shards per rack: map[rack1:1 rack2:1 rack3:1 rack4:1] (max allowed: 1) --- PASS: TestCommandEcBalanceEvenDataAndParityDistribution (0.00s) === RUN TestCommandEcBalanceMultipleVolumesEvenDistribution dn1 moves ec shard 1.0 to dn2 dn1 moves ec shard 1.1 to dn3 dn1 moves ec shard 1.2 to dn4 dn1 moves ec shard 1.3 to dn5 dn1 moves ec shard 1.4 to dn6 dn1 moves ec shard 1.5 to dn2 dn1 moves ec shard 1.6 to dn3 dn1 moves ec shard 1.7 to dn4 dn1 moves ec shard 1.10 to dn2 dn1 moves ec shard 1.11 to dn3 dn1 moves ec shard 1.12 to dn4 dn2 moves ec shard 2.0 to dn1 dn2 moves ec shard 2.1 to dn3 dn2 moves ec shard 2.2 to dn4 dn2 moves ec shard 2.3 to dn5 dn2 moves ec shard 2.4 to dn6 dn2 moves ec shard 2.5 to dn1 dn2 moves ec shard 2.6 to dn3 dn2 moves ec shard 2.7 to dn4 dn2 moves ec shard 2.10 to dn1 dn2 moves ec shard 2.11 to dn3 dn2 moves ec shard 2.12 to dn4 command_ec_test.go:302: Volume 1 - Data: map[rack1:2 rack2:2 rack3:2 rack4:2 rack5:1 rack6:1], Parity: map[rack1:1 rack2:1 rack3:1 rack4:1] command_ec_test.go:302: Volume 2 - Data: map[rack1:2 rack2:2 rack3:2 rack4:2 rack5:1 rack6:1], Parity: map[rack1:1 rack2:1 rack3:1 rack4:1] --- PASS: TestCommandEcBalanceMultipleVolumesEvenDistribution (0.00s) === RUN TestCommandEcBalanceAllNodesShareAllVolumes dn1 moves ec shard 1.0 to dn3 dn1 moves ec shard 1.1 to dn4 dn2 moves ec shard 1.5 to dn3 dn2 moves ec shard 1.6 to dn4 dn3 moves ec shard 1.10 to dn1 dn3 moves ec shard 1.11 to dn2 dn1 moves ec shard 2.0 to dn3 dn1 moves ec shard 2.1 to dn4 dn2 moves ec shard 2.5 to dn3 dn2 moves ec shard 2.6 to dn4 dn3 moves ec shard 2.10 to dn1 dn3 moves ec shard 2.11 to dn2 dn1 moves ec shard 1.2 to dn3 dn2 moves ec shard 1.7 to dn4 command_ec_test.go:350: node dn1: 7 shards command_ec_test.go:350: node dn2: 7 shards command_ec_test.go:350: node dn3: 7 shards command_ec_test.go:350: node dn4: 7 shards --- PASS: TestCommandEcBalanceAllNodesShareAllVolumes (0.00s) === RUN TestCommandEcBalanceIssue8793Topology command_ec_test.go:416: BEFORE node 192.168.0.12:8332 (max 80): 36 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8333 (max 80): 36 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8334 (max 80): 36 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8335 (max 80): 36 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8336 (max 80): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8337 (max 80): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8338 (max 80): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8339 (max 80): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8340 (max 80): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8341 (max 33): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8342 (max 33): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.12:8343 (max 33): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.25:8350 (max 33): 22 shards command_ec_test.go:416: BEFORE node 192.168.0.25:8351 (max 33): 22 shards 192.168.0.12:8332 moves ec shard 23.0 to 192.168.0.12:8333 192.168.0.12:8332 moves ec shard 23.1 to 192.168.0.12:8334 192.168.0.12:8332 moves ec shard 23.2 to 192.168.0.12:8335 192.168.0.12:8332 moves ec shard 23.3 to 192.168.0.12:8336 192.168.0.12:8332 moves ec shard 23.4 to 192.168.0.12:8337 192.168.0.12:8332 moves ec shard 23.5 to 192.168.0.12:8338 192.168.0.12:8332 moves ec shard 23.6 to 192.168.0.12:8339 192.168.0.12:8332 moves ec shard 23.7 to 192.168.0.12:8340 192.168.0.12:8332 moves ec shard 23.8 to 192.168.0.12:8341 192.168.0.12:8332 moves ec shard 23.10 to 192.168.0.12:8342 192.168.0.12:8332 moves ec shard 23.11 to 192.168.0.12:8343 192.168.0.12:8332 moves ec shard 23.12 to 192.168.0.25:8350 192.168.0.12:8333 moves ec shard 24.0 to 192.168.0.12:8332 192.168.0.12:8333 moves ec shard 24.1 to 192.168.0.12:8334 192.168.0.12:8333 moves ec shard 24.2 to 192.168.0.12:8335 192.168.0.12:8333 moves ec shard 24.3 to 192.168.0.12:8336 192.168.0.12:8333 moves ec shard 24.4 to 192.168.0.12:8337 192.168.0.12:8333 moves ec shard 24.5 to 192.168.0.12:8338 192.168.0.12:8333 moves ec shard 24.6 to 192.168.0.12:8339 192.168.0.12:8333 moves ec shard 24.7 to 192.168.0.12:8340 192.168.0.12:8333 moves ec shard 24.8 to 192.168.0.12:8341 192.168.0.12:8333 moves ec shard 24.10 to 192.168.0.12:8342 192.168.0.12:8333 moves ec shard 24.11 to 192.168.0.12:8343 192.168.0.12:8333 moves ec shard 24.12 to 192.168.0.25:8350 192.168.0.12:8334 moves ec shard 25.0 to 192.168.0.12:8332 192.168.0.12:8334 moves ec shard 25.1 to 192.168.0.12:8333 192.168.0.12:8334 moves ec shard 25.2 to 192.168.0.12:8335 192.168.0.12:8334 moves ec shard 25.3 to 192.168.0.12:8336 192.168.0.12:8334 moves ec shard 25.4 to 192.168.0.12:8337 192.168.0.12:8334 moves ec shard 25.5 to 192.168.0.12:8338 192.168.0.12:8334 moves ec shard 25.6 to 192.168.0.12:8339 192.168.0.12:8334 moves ec shard 25.7 to 192.168.0.12:8340 192.168.0.12:8334 moves ec shard 25.8 to 192.168.0.12:8341 192.168.0.12:8334 moves ec shard 25.10 to 192.168.0.12:8342 192.168.0.12:8334 moves ec shard 25.11 to 192.168.0.12:8343 192.168.0.12:8334 moves ec shard 25.12 to 192.168.0.25:8350 192.168.0.12:8335 moves ec shard 26.0 to 192.168.0.12:8332 192.168.0.12:8335 moves ec shard 26.1 to 192.168.0.12:8333 192.168.0.12:8335 moves ec shard 26.2 to 192.168.0.12:8334 192.168.0.12:8335 moves ec shard 26.3 to 192.168.0.12:8336 192.168.0.12:8335 moves ec shard 26.4 to 192.168.0.12:8337 192.168.0.12:8335 moves ec shard 26.5 to 192.168.0.12:8338 192.168.0.12:8335 moves ec shard 26.6 to 192.168.0.12:8339 192.168.0.12:8335 moves ec shard 26.7 to 192.168.0.12:8340 192.168.0.12:8335 moves ec shard 26.8 to 192.168.0.12:8341 192.168.0.12:8335 moves ec shard 26.10 to 192.168.0.12:8342 192.168.0.12:8335 moves ec shard 26.11 to 192.168.0.12:8343 192.168.0.12:8335 moves ec shard 26.12 to 192.168.0.25:8350 192.168.0.12:8341 moves ec shard 1.9 to 192.168.0.12:8336 192.168.0.12:8342 moves ec shard 1.10 to 192.168.0.12:8337 192.168.0.12:8343 moves ec shard 1.11 to 192.168.0.12:8338 192.168.0.25:8350 moves ec shard 1.12 to 192.168.0.12:8339 192.168.0.12:8341 moves ec shard 2.9 to 192.168.0.12:8340 192.168.0.12:8342 moves ec shard 2.10 to 192.168.0.12:8332 192.168.0.12:8343 moves ec shard 2.11 to 192.168.0.12:8333 192.168.0.25:8350 moves ec shard 2.12 to 192.168.0.12:8334 192.168.0.12:8341 moves ec shard 3.9 to 192.168.0.12:8335 192.168.0.12:8342 moves ec shard 3.10 to 192.168.0.12:8336 192.168.0.12:8343 moves ec shard 3.11 to 192.168.0.12:8337 192.168.0.25:8350 moves ec shard 3.12 to 192.168.0.12:8338 192.168.0.12:8341 moves ec shard 4.9 to 192.168.0.12:8339 192.168.0.12:8342 moves ec shard 4.10 to 192.168.0.12:8340 192.168.0.12:8343 moves ec shard 4.11 to 192.168.0.12:8332 192.168.0.25:8350 moves ec shard 4.12 to 192.168.0.12:8333 192.168.0.12:8341 moves ec shard 5.9 to 192.168.0.12:8334 192.168.0.12:8342 moves ec shard 5.10 to 192.168.0.12:8335 192.168.0.12:8343 moves ec shard 5.11 to 192.168.0.12:8336 192.168.0.25:8350 moves ec shard 5.12 to 192.168.0.12:8337 192.168.0.25:8351 moves ec shard 1.13 to 192.168.0.12:8338 192.168.0.12:8341 moves ec shard 6.9 to 192.168.0.12:8339 192.168.0.12:8342 moves ec shard 6.10 to 192.168.0.12:8340 192.168.0.12:8343 moves ec shard 6.11 to 192.168.0.12:8332 192.168.0.25:8350 moves ec shard 6.12 to 192.168.0.12:8333 192.168.0.25:8351 moves ec shard 2.13 to 192.168.0.12:8334 192.168.0.12:8341 moves ec shard 7.9 to 192.168.0.12:8335 192.168.0.12:8342 moves ec shard 7.10 to 192.168.0.12:8336 192.168.0.12:8343 moves ec shard 7.11 to 192.168.0.12:8337 192.168.0.25:8350 moves ec shard 7.12 to 192.168.0.12:8338 192.168.0.25:8351 moves ec shard 3.13 to 192.168.0.12:8339 192.168.0.12:8341 moves ec shard 8.9 to 192.168.0.12:8340 192.168.0.12:8342 moves ec shard 8.10 to 192.168.0.12:8332 192.168.0.12:8343 moves ec shard 8.11 to 192.168.0.12:8333 192.168.0.25:8350 moves ec shard 8.12 to 192.168.0.12:8334 192.168.0.25:8351 moves ec shard 4.13 to 192.168.0.12:8335 192.168.0.12:8341 moves ec shard 9.9 to 192.168.0.12:8336 192.168.0.12:8342 moves ec shard 9.10 to 192.168.0.12:8337 192.168.0.12:8343 moves ec shard 9.11 to 192.168.0.12:8338 192.168.0.25:8350 moves ec shard 9.12 to 192.168.0.12:8339 192.168.0.25:8351 moves ec shard 5.13 to 192.168.0.12:8340 192.168.0.12:8341 moves ec shard 10.9 to 192.168.0.12:8332 192.168.0.12:8342 moves ec shard 10.10 to 192.168.0.12:8333 192.168.0.12:8343 moves ec shard 10.11 to 192.168.0.12:8334 192.168.0.25:8350 moves ec shard 10.12 to 192.168.0.12:8335 192.168.0.25:8351 moves ec shard 6.13 to 192.168.0.12:8336 192.168.0.12:8341 moves ec shard 11.9 to 192.168.0.12:8337 192.168.0.12:8342 moves ec shard 11.10 to 192.168.0.12:8338 192.168.0.12:8343 moves ec shard 11.11 to 192.168.0.12:8339 192.168.0.25:8350 moves ec shard 11.12 to 192.168.0.12:8340 192.168.0.25:8351 moves ec shard 7.13 to 192.168.0.12:8332 192.168.0.12:8341 moves ec shard 12.9 to 192.168.0.12:8333 192.168.0.12:8342 moves ec shard 12.10 to 192.168.0.12:8334 192.168.0.12:8343 moves ec shard 12.11 to 192.168.0.12:8335 192.168.0.25:8350 moves ec shard 12.12 to 192.168.0.12:8336 192.168.0.25:8351 moves ec shard 8.13 to 192.168.0.12:8337 command_ec_test.go:453: AFTER node 192.168.0.12:8332: 33/80 shards (41% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8333: 33/80 shards (41% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8334: 33/80 shards (41% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8335: 33/80 shards (41% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8336: 33/80 shards (41% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8337: 33/80 shards (41% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8338: 32/80 shards (40% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8339: 32/80 shards (40% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8340: 32/80 shards (40% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8341: 14/33 shards (42% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8342: 14/33 shards (42% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.12:8343: 14/33 shards (42% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.25:8350: 14/33 shards (42% full, overall 41%) command_ec_test.go:453: AFTER node 192.168.0.25:8351: 14/33 shards (42% full, overall 41%) --- PASS: TestCommandEcBalanceIssue8793Topology (0.00s) === RUN TestShortName --- PASS: TestShortName (0.00s) === RUN TestRelevantNodes --- PASS: TestRelevantNodes (0.00s) === RUN TestComputeOwnerTarget_EvenDivision --- PASS: TestComputeOwnerTarget_EvenDivision (0.00s) === RUN TestComputeOwnerTarget_UnevenDivision --- PASS: TestComputeOwnerTarget_UnevenDivision (0.00s) === RUN TestComputeOwnerTarget_InactiveNodesDrainToZero --- PASS: TestComputeOwnerTarget_InactiveNodesDrainToZero (0.00s) === RUN TestBuildDistributionCounts --- PASS: TestBuildDistributionCounts (0.00s) === RUN TestBuildDistributionCounts_NilFidFallback --- PASS: TestBuildDistributionCounts_NilFidFallback (0.00s) === RUN TestSelectActiveNodes_AllNodes --- PASS: TestSelectActiveNodes_AllNodes (0.00s) === RUN TestSelectActiveNodes_LimitToTwo --- PASS: TestSelectActiveNodes_LimitToTwo (0.00s) === RUN TestPlanOwnerMoves_BasicBalance --- PASS: TestPlanOwnerMoves_BasicBalance (0.00s) === RUN TestPlanOwnerMoves_AlreadyBalanced --- PASS: TestPlanOwnerMoves_AlreadyBalanced (0.00s) === RUN TestPlanDistribution_Primary_BalancesChunks --- PASS: TestPlanDistribution_Primary_BalancesChunks (0.00s) === RUN TestPlanDistribution_Primary_NoMovesWhenBalanced --- PASS: TestPlanDistribution_Primary_NoMovesWhenBalanced (0.00s) === RUN TestPlanDistribution_RoundRobin_AssignsByOffset --- PASS: TestPlanDistribution_RoundRobin_AssignsByOffset (0.00s) === RUN TestPlanDistribution_RoundRobin_OffsetOrdering --- PASS: TestPlanDistribution_RoundRobin_OffsetOrdering (0.00s) === RUN TestPlanDistribution_Replica_BalancesOwnerAndCopies --- PASS: TestPlanDistribution_Replica_BalancesOwnerAndCopies (0.00s) === RUN TestPrintRedistributionPlan_Output --- PASS: TestPrintRedistributionPlan_Output (0.00s) === RUN TestCreateMergePlan_HonorsExplicitDirection --- PASS: TestCreateMergePlan_HonorsExplicitDirection (0.00s) === RUN TestCreateMergePlan_DirectedAllowsEmptyTarget --- PASS: TestCreateMergePlan_DirectedAllowsEmptyTarget (0.00s) === RUN TestCreateMergePlan_DirectedRejectsSelfMap --- PASS: TestCreateMergePlan_DirectedRejectsSelfMap (0.00s) === RUN TestCreateMergePlan_DirectedRejectsIneligible === RUN TestCreateMergePlan_DirectedRejectsIneligible/readonly_source === RUN TestCreateMergePlan_DirectedRejectsIneligible/readonly_target === RUN TestCreateMergePlan_DirectedRejectsIneligible/empty_source === RUN TestCreateMergePlan_DirectedRejectsIneligible/wrong_collection --- PASS: TestCreateMergePlan_DirectedRejectsIneligible (0.00s) --- PASS: TestCreateMergePlan_DirectedRejectsIneligible/readonly_source (0.00s) --- PASS: TestCreateMergePlan_DirectedRejectsIneligible/readonly_target (0.00s) --- PASS: TestCreateMergePlan_DirectedRejectsIneligible/empty_source (0.00s) --- PASS: TestCreateMergePlan_DirectedRejectsIneligible/wrong_collection (0.00s) === RUN TestFsMvMovesIntoExistingDestinationDirectory --- PASS: TestFsMvMovesIntoExistingDestinationDirectory (0.00s) === RUN TestPlanLocalToRemoteSync === RUN TestPlanLocalToRemoteSync/copy_local-only_files,_no_delete_flag === RUN TestPlanLocalToRemoteSync/delete_flag_removes_remote-only_files === RUN TestPlanLocalToRemoteSync/delete_with_empty_local_storage_removes_all_remote_files === RUN TestPlanLocalToRemoteSync/delete_respects_include_pattern === RUN TestPlanLocalToRemoteSync/delete_respects_exclude_pattern === RUN TestPlanLocalToRemoteSync/delete_ignores_size/age_filters === RUN TestPlanLocalToRemoteSync/delete_removes_orphaned_files,_directory_entries_left_alone === RUN TestPlanLocalToRemoteSync/exclude_pattern_protects_remote_files_under_nested_directory === RUN TestPlanLocalToRemoteSync/remote_files_outside_-dir_are_never_deleted === RUN TestPlanLocalToRemoteSync/directory_present_locally_is_not_deleted === RUN TestPlanLocalToRemoteSync/forceUpdate_copies_files_that_exist_on_remote === RUN TestPlanLocalToRemoteSync/local_directories_are_not_copied --- PASS: TestPlanLocalToRemoteSync (0.00s) --- PASS: TestPlanLocalToRemoteSync/copy_local-only_files,_no_delete_flag (0.00s) --- PASS: TestPlanLocalToRemoteSync/delete_flag_removes_remote-only_files (0.00s) --- PASS: TestPlanLocalToRemoteSync/delete_with_empty_local_storage_removes_all_remote_files (0.00s) --- PASS: TestPlanLocalToRemoteSync/delete_respects_include_pattern (0.00s) --- PASS: TestPlanLocalToRemoteSync/delete_respects_exclude_pattern (0.00s) --- PASS: TestPlanLocalToRemoteSync/delete_ignores_size/age_filters (0.00s) --- PASS: TestPlanLocalToRemoteSync/delete_removes_orphaned_files,_directory_entries_left_alone (0.00s) --- PASS: TestPlanLocalToRemoteSync/exclude_pattern_protects_remote_files_under_nested_directory (0.00s) --- PASS: TestPlanLocalToRemoteSync/remote_files_outside_-dir_are_never_deleted (0.00s) --- PASS: TestPlanLocalToRemoteSync/directory_present_locally_is_not_deleted (0.00s) --- PASS: TestPlanLocalToRemoteSync/forceUpdate_copies_files_that_exist_on_remote (0.00s) --- PASS: TestPlanLocalToRemoteSync/local_directories_are_not_copied (0.00s) === RUN TestFileFilter_matchesName === RUN TestFileFilter_matchesName/no_filters === RUN TestFileFilter_matchesName/include_match === RUN TestFileFilter_matchesName/include_mismatch === RUN TestFileFilter_matchesName/exclude_match === RUN TestFileFilter_matchesName/exclude_mismatch --- PASS: TestFileFilter_matchesName (0.00s) --- PASS: TestFileFilter_matchesName/no_filters (0.00s) --- PASS: TestFileFilter_matchesName/include_match (0.00s) --- PASS: TestFileFilter_matchesName/include_mismatch (0.00s) --- PASS: TestFileFilter_matchesName/exclude_match (0.00s) --- PASS: TestFileFilter_matchesName/exclude_mismatch (0.00s) === RUN TestFileFilter_matches_minCacheAge === RUN TestFileFilter_matches_minCacheAge/no_minCacheAge === RUN TestFileFilter_matches_minCacheAge/recent_cache,_should_not_match === RUN TestFileFilter_matches_minCacheAge/old_cache,_should_match === RUN TestFileFilter_matches_minCacheAge/no_remote_entry,_uses_crtime_-_recent,_should_not_match === RUN TestFileFilter_matches_minCacheAge/no_remote_entry,_uses_crtime_-_old,_should_match === RUN TestFileFilter_matches_minCacheAge/remote_entry_with_0_sync_ts,_uses_crtime_-_recent,_should_not_match === RUN TestFileFilter_matches_minCacheAge/nil_attributes,_should_not_match --- PASS: TestFileFilter_matches_minCacheAge (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/no_minCacheAge (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/recent_cache,_should_not_match (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/old_cache,_should_match (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/no_remote_entry,_uses_crtime_-_recent,_should_not_match (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/no_remote_entry,_uses_crtime_-_old,_should_match (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/remote_entry_with_0_sync_ts,_uses_crtime_-_recent,_should_not_match (0.00s) --- PASS: TestFileFilter_matches_minCacheAge/nil_attributes,_should_not_match (0.00s) === RUN TestUpdateBucketActions_SetActions --- PASS: TestUpdateBucketActions_SetActions (0.00s) === RUN TestUpdateBucketActions_ReplaceActions --- PASS: TestUpdateBucketActions_ReplaceActions (0.00s) === RUN TestUpdateBucketActions_None --- PASS: TestUpdateBucketActions_None (0.00s) === RUN TestUpdateBucketActions_PreservesOtherBuckets --- PASS: TestUpdateBucketActions_PreservesOtherBuckets (0.00s) === RUN TestUpdateBucketActions_PreservesGlobalActions --- PASS: TestUpdateBucketActions_PreservesGlobalActions (0.00s) === RUN TestDisplayBucketAccess --- PASS: TestDisplayBucketAccess (0.00s) === RUN TestDisplayBucketAccess_None --- PASS: TestDisplayBucketAccess_None (0.00s) === RUN TestCircuitBreakerShell --- PASS: TestCircuitBreakerShell (0.00s) === RUN TestParseShardsSpec_Range --- PASS: TestParseShardsSpec_Range (0.00s) === RUN TestParseShardsSpec_Set --- PASS: TestParseShardsSpec_Set (0.00s) === RUN TestParseShardsSpec_DedupSort --- PASS: TestParseShardsSpec_DedupSort (0.00s) === RUN TestParseShardsSpec_OutOfRange --- PASS: TestParseShardsSpec_OutOfRange (0.00s) === RUN TestParseShardsSpec_BadRange --- PASS: TestParseShardsSpec_BadRange (0.00s) === RUN TestResolveShardSelection_Mutex --- PASS: TestResolveShardSelection_Mutex (0.00s) === RUN TestResolveShardSelection_SingleShard --- PASS: TestResolveShardSelection_SingleShard (0.00s) === RUN TestFormatShardLabel --- PASS: TestFormatShardLabel (0.00s) === RUN TestIsGoodMove replication: 100 expected false name: test 100 move to wrong data centers replication: 100 expected true name: test 100 move to spread into proper data centers replication: 001 expected false name: test move to the same node replication: 001 expected false name: test move to the same rack, but existing node replication: 001 expected true name: test move to the same rack, a new node replication: 010 expected false name: test 010 move all to the same rack replication: 010 expected true name: test 010 move to spread racks replication: 010 expected true name: test 010 move to spread racks replication: 011 expected true name: test 011 switch which rack has more replicas replication: 011 expected true name: test 011 move the lonely replica to another racks replication: 011 expected false name: test 011 move to wrong racks replication: 011 expected false name: test 011 move all to the same rack replication: 001 expected false name: test 001 reject move onto a machine already holding a replica replication: 001 expected true name: test 001 allow move onto a different machine in the rack --- PASS: TestIsGoodMove (0.00s) === RUN TestBalance hdd 0.00 0.01:0.00 moving volume collection4_7 192.168.1.2:8080 => 192.168.1.6:8080 hdd 0.00 0.01:0.00 moving volume collection3_9 192.168.1.2:8080 => 192.168.1.6:8080 hdd 0.00 0.01:0.00 moving volume 4 192.168.1.2:8080 => 192.168.1.6:8080 hdd 0.00 0.01:0.00 moving volume 2 192.168.1.2:8080 => 192.168.1.6:8080 hdd 0.00 0.01:0.00 moving volume 6 192.168.1.2:8080 => 192.168.1.6:8080 hdd 0.00 0.01:0.00 moving volume 3 192.168.1.2:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume 31 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume 29 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume 30 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume 27 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume 28 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection0_25 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_80 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_69 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_84 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_63 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_74 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_85 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_54 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_97 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_174 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_105 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection0_24 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_173 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_136 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_138 192.168.1.4:8080 => 192.168.1.6:8080 hdd 0.00 0.00:0.00 moving volume collection1_116 192.168.1.4:8080 => 192.168.1.6:8080 --- PASS: TestBalance (0.00s) === RUN TestBalanceDoesNotDrainOntoOneNode hdd 0.33 1.00:0.10 moving volume 1 full => empty hdd 0.33 0.67:0.10 moving volume 2 full => empty hdd 0.33 0.43:0.22 moving volume 3 full => empty --- PASS: TestBalanceDoesNotDrainOntoOneNode (0.00s) === RUN TestBalanceByDiskUsage hdd 0.03 0.43:0.02 moving volume 104 half-full => byte-full hdd 0.03 0.36:0.02 moving volume 101 half-full => byte-full hdd 0.03 0.30:0.02 moving volume 109 half-full => byte-full hdd 0.03 0.25:0.02 moving volume 106 half-full => byte-full hdd 0.03 0.20:0.02 moving volume 107 half-full => byte-full hdd 0.03 0.15:0.02 moving volume 110 half-full => byte-full hdd 0.03 0.11:0.03 moving volume 102 half-full => byte-full hdd 0.03 0.07:0.03 moving volume 108 half-full => byte-full hdd 0.03 0.03:0.03 moving volume 103 half-full => byte-full hdd 14.00 19.00:10.00 moving volume 6 byte-full => half-full hdd 14.00 18.00:11.00 moving volume 18 byte-full => half-full hdd 14.00 17.00:12.00 moving volume 5 byte-full => half-full hdd 14.00 16.00:13.00 moving volume 1 byte-full => half-full hdd 14.00 15.00:14.00 moving volume 20 byte-full => half-full --- PASS: TestBalanceByDiskUsage (0.00s) === RUN TestBalanceByDiskUsageUsesPhysicalDiskUsageWhenReported hdd 0.52 0.99:0.10 moving volume 1 physical-full => target hdd 0.52 0.99:0.10 moving volume 5 physical-full => target hdd 0.52 0.99:0.10 moving volume 2 physical-full => target hdd 0.52 0.99:0.10 moving volume 3 physical-full => target hdd 0.52 0.99:0.10 moving volume 4 physical-full => target --- PASS: TestBalanceByDiskUsageUsesPhysicalDiskUsageWhenReported (0.00s) === RUN TestBalanceByDiskUsageMixedReportingFallsBackToDataSize hdd 6.67 19.00:1.00 moving volume 103 new-full => new-target hdd 6.67 18.00:2.00 moving volume 109 new-full => new-target hdd 6.67 17.00:2.00 moving volume 120 new-full => old-empty hdd 6.67 16.00:3.00 moving volume 115 new-full => old-empty hdd 6.67 15.00:3.00 moving volume 102 new-full => new-target hdd 6.67 14.00:4.00 moving volume 114 new-full => new-target hdd 6.67 13.00:4.00 moving volume 106 new-full => old-empty hdd 6.67 12.00:5.00 moving volume 101 new-full => old-empty hdd 6.67 11.00:5.00 moving volume 108 new-full => new-target hdd 6.67 10.00:6.00 moving volume 111 new-full => new-target hdd 6.67 9.00:6.00 moving volume 116 new-full => old-empty --- PASS: TestBalanceByDiskUsageMixedReportingFallsBackToDataSize (0.00s) === RUN TestBalanceSkipsPhysicallyFullTarget hdd 0.01 2.33:0.00 moving volume 1 source => disk-full hdd 0.01 1.50:0.00 moving volume 3 source => disk-full hdd 0.01 1.00:0.00 moving volume 7 source => disk-full hdd 0.01 0.67:0.00 moving volume 4 source => disk-full hdd 0.01 0.43:0.01 moving volume 5 source => disk-full hdd 0.01 0.25:0.01 moving volume 6 source => disk-full hdd 0.01 2.33:0.00 moving volume 4 source => disk-full hdd 0.01 1.50:0.00 moving volume 8 source => disk-full hdd 0.01 1.00:0.00 moving volume 3 source => disk-full hdd 0.01 0.67:0.00 moving volume 6 source => disk-full hdd 0.01 0.43:0.01 moving volume 5 source => disk-full hdd 0.01 0.25:0.01 moving volume 7 source => disk-full --- PASS: TestBalanceSkipsPhysicallyFullTarget (0.00s) === RUN TestBalanceGateSteersToEmptierDisk hdd 0.00 2.33:0.00 moving volume 1 source => disk-empty hdd 0.00 1.50:0.00 moving volume 4 source => disk-empty hdd 0.00 1.00:0.00 moving volume 5 source => disk-empty --- PASS: TestBalanceGateSteersToEmptierDisk (0.00s) === RUN TestBalanceVolumesPerExec hdd 0.33 1.00:0.10 moving volume 5 full => empty --- PASS: TestBalanceVolumesPerExec (0.00s) === RUN TestVolumeSelection collect volumes quiet for: 0 seconds --- PASS: TestVolumeSelection (0.00s) === RUN TestDeleteEmptySelection --- PASS: TestDeleteEmptySelection (0.00s) === RUN TestSplitCSVSet === RUN TestSplitCSVSet/empty_input_is_empty_set_(no_filter) === RUN TestSplitCSVSet/whitespace_only_is_empty_set_(no_filter) === RUN TestSplitCSVSet/commas_only_is_empty_set_(no_filter) === RUN TestSplitCSVSet/whitespace_and_commas_only_is_empty_set_(no_filter) === RUN TestSplitCSVSet/single === RUN TestSplitCSVSet/multi === RUN TestSplitCSVSet/trims_whitespace === RUN TestSplitCSVSet/skips_empty_items --- PASS: TestSplitCSVSet (0.00s) --- PASS: TestSplitCSVSet/empty_input_is_empty_set_(no_filter) (0.00s) --- PASS: TestSplitCSVSet/whitespace_only_is_empty_set_(no_filter) (0.00s) --- PASS: TestSplitCSVSet/commas_only_is_empty_set_(no_filter) (0.00s) --- PASS: TestSplitCSVSet/whitespace_and_commas_only_is_empty_set_(no_filter) (0.00s) --- PASS: TestSplitCSVSet/single (0.00s) --- PASS: TestSplitCSVSet/multi (0.00s) --- PASS: TestSplitCSVSet/trims_whitespace (0.00s) --- PASS: TestSplitCSVSet/skips_empty_items (0.00s) === RUN TestCollectVolumeServersByDcRackNode_RackFilter --- PASS: TestCollectVolumeServersByDcRackNode_RackFilter (0.00s) === RUN TestCollectVolumeServersByDcRackNode_NodeFilter --- PASS: TestCollectVolumeServersByDcRackNode_NodeFilter (0.00s) === RUN TestDoVolumeCheckDiskDoesNotResurrectAbsentNeedle --- PASS: TestDoVolumeCheckDiskDoesNotResurrectAbsentNeedle (0.00s) === RUN TestDoVolumeCheckDiskResurrectSkipsVacuumedTarget --- PASS: TestDoVolumeCheckDiskResurrectSkipsVacuumedTarget (0.01s) === RUN TestDoVolumeCheckDiskResurrectQueuesProvenMissingWrite --- PASS: TestDoVolumeCheckDiskResurrectQueuesProvenMissingWrite (0.00s) === RUN TestShouldSkipVolume === RUN TestShouldSkipVolume/identical_volumes_should_be_skipped === RUN TestShouldSkipVolume/different_file_counts_should_not_be_skipped === RUN TestShouldSkipVolume/different_delete_counts_with_syncDeletions_enabled_should_not_be_skipped === RUN TestShouldSkipVolume/different_delete_counts_with_syncDeletions_disabled_should_be_skipped_if_file_counts_match === RUN TestShouldSkipVolume/volumes_with_zero_file_counts_should_be_skipped === RUN TestShouldSkipVolume/volumes_with_zero_and_non-zero_file_counts_should_not_be_skipped === RUN TestShouldSkipVolume/recently_modified_volumes_with_same_file_counts_should_be_skipped === RUN TestShouldSkipVolume/one_volume_modified_before_pulse_cutoff_with_different_file_counts_should_not_be_skipped === RUN TestShouldSkipVolume/different_modification_times_with_same_file_counts_should_be_skipped === RUN TestShouldSkipVolume/volumes_modified_exactly_at_pulse_cutoff_boundary_with_different_counts_should_not_be_skipped === RUN TestShouldSkipVolume/volumes_modified_just_after_pulse_cutoff_boundary_with_same_counts_should_be_skipped === RUN TestShouldSkipVolume/large_file_count_difference_with_old_modification_time_should_not_be_skipped === RUN TestShouldSkipVolume/both_volumes_modified_after_pulse_cutoff_with_same_file_counts_should_be_skipped --- PASS: TestShouldSkipVolume (0.00s) --- PASS: TestShouldSkipVolume/identical_volumes_should_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/different_file_counts_should_not_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/different_delete_counts_with_syncDeletions_enabled_should_not_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/different_delete_counts_with_syncDeletions_disabled_should_be_skipped_if_file_counts_match (0.00s) --- PASS: TestShouldSkipVolume/volumes_with_zero_file_counts_should_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/volumes_with_zero_and_non-zero_file_counts_should_not_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/recently_modified_volumes_with_same_file_counts_should_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/one_volume_modified_before_pulse_cutoff_with_different_file_counts_should_not_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/different_modification_times_with_same_file_counts_should_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/volumes_modified_exactly_at_pulse_cutoff_boundary_with_different_counts_should_not_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/volumes_modified_just_after_pulse_cutoff_boundary_with_same_counts_should_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/large_file_count_difference_with_old_modification_time_should_not_be_skipped (0.00s) --- PASS: TestShouldSkipVolume/both_volumes_modified_after_pulse_cutoff_with_same_file_counts_should_be_skipped (0.00s) === RUN TestVolumeCheckDiskHelperMethods --- PASS: TestVolumeCheckDiskHelperMethods (0.00s) === RUN TestIsEmptyVolumeDeleteCandidateCollectionPattern === RUN TestIsEmptyVolumeDeleteCandidateCollectionPattern/empty_pattern_matches_named_collection === RUN TestIsEmptyVolumeDeleteCandidateCollectionPattern/wildcard_matches_collection === RUN TestIsEmptyVolumeDeleteCandidateCollectionPattern/wildcard_rejects_collection === RUN TestIsEmptyVolumeDeleteCandidateCollectionPattern/default_pattern_matches_empty_collection === RUN TestIsEmptyVolumeDeleteCandidateCollectionPattern/default_pattern_rejects_named_collection --- PASS: TestIsEmptyVolumeDeleteCandidateCollectionPattern (0.00s) --- PASS: TestIsEmptyVolumeDeleteCandidateCollectionPattern/empty_pattern_matches_named_collection (0.00s) --- PASS: TestIsEmptyVolumeDeleteCandidateCollectionPattern/wildcard_matches_collection (0.00s) --- PASS: TestIsEmptyVolumeDeleteCandidateCollectionPattern/wildcard_rejects_collection (0.00s) --- PASS: TestIsEmptyVolumeDeleteCandidateCollectionPattern/default_pattern_matches_empty_collection (0.00s) --- PASS: TestIsEmptyVolumeDeleteCandidateCollectionPattern/default_pattern_rejects_named_collection (0.00s) === RUN TestMatchCollectionPattern === RUN TestMatchCollectionPattern/empty_pattern_matches_any_collection === RUN TestMatchCollectionPattern/empty_pattern_matches_empty_collection === RUN TestMatchCollectionPattern/default_pattern_matches_empty_collection === RUN TestMatchCollectionPattern/default_pattern_rejects_named_collection === RUN TestMatchCollectionPattern/exact_match === RUN TestMatchCollectionPattern/exact_mismatch === RUN TestMatchCollectionPattern/prefix_wildcard_match === RUN TestMatchCollectionPattern/prefix_wildcard_mismatch === RUN TestMatchCollectionPattern/single_char_wildcard_match === RUN TestMatchCollectionPattern/single_char_wildcard_mismatch --- PASS: TestMatchCollectionPattern (0.00s) --- PASS: TestMatchCollectionPattern/empty_pattern_matches_any_collection (0.00s) --- PASS: TestMatchCollectionPattern/empty_pattern_matches_empty_collection (0.00s) --- PASS: TestMatchCollectionPattern/default_pattern_matches_empty_collection (0.00s) --- PASS: TestMatchCollectionPattern/default_pattern_rejects_named_collection (0.00s) --- PASS: TestMatchCollectionPattern/exact_match (0.00s) --- PASS: TestMatchCollectionPattern/exact_mismatch (0.00s) --- PASS: TestMatchCollectionPattern/prefix_wildcard_match (0.00s) --- PASS: TestMatchCollectionPattern/prefix_wildcard_mismatch (0.00s) --- PASS: TestMatchCollectionPattern/single_char_wildcard_match (0.00s) --- PASS: TestMatchCollectionPattern/single_char_wildcard_mismatch (0.00s) === RUN TestReserveTargetSpreadsAcrossServers --- PASS: TestReserveTargetSpreadsAcrossServers (0.10s) === RUN TestFixUnderReplicatedVolumesInParallel --- PASS: TestFixUnderReplicatedVolumesInParallel (0.00s) === RUN TestSatisfyReplicaPlacementComplicated replication: 100 expected false name: test 100 negative replication: 100 expected true name: test 100 positive replication: 022 expected true name: test 022 positive replication: 022 expected false name: test 022 negative replication: 210 expected true name: test 210 moved from 200 positive replication: 210 expected false name: test 210 moved from 200 negative extra dc replication: 210 expected false name: test 210 moved from 200 negative extra data node --- PASS: TestSatisfyReplicaPlacementComplicated (0.00s) === RUN TestSatisfyReplicaPlacement01x replication: 011 expected true name: test 011 same existing rack replication: 011 expected false name: test 011 negative replication: 011 expected true name: test 011 different existing racks replication: 011 expected false name: test 011 different existing racks negative --- PASS: TestSatisfyReplicaPlacement01x (0.00s) === RUN TestSatisfyReplicaPlacement00x replication: 001 expected true name: test 001 replication: 002 expected true name: test 002 positive replication: 002 expected false name: test 002 negative, repeat the same node replication: 002 expected false name: test 002 negative, enough node already --- PASS: TestSatisfyReplicaPlacement00x (0.00s) === RUN TestSatisfyReplicaPlacement100 replication: 100 expected true name: test 100 --- PASS: TestSatisfyReplicaPlacement100 (0.00s) === RUN TestMisplacedChecking replication: 001 expected true name: test 001 replication: 010 expected false name: test 010 replication: 011 expected false name: test 011 replication: 110 expected true name: test 110 replication: 100 expected true name: test 100 --- PASS: TestMisplacedChecking (0.00s) === RUN TestPickingMisplacedVolumeToDelete replication: 001 name: test 001 command_volume_fix_replication_test.go:435: test 001: picked dn2 001 replication: 100 name: test 100 command_volume_fix_replication_test.go:435: test 100: picked dn2 100 --- PASS: TestPickingMisplacedVolumeToDelete (0.00s) === RUN TestPickOneReplicaToDeleteSkipsReadOnlySurvivor === RUN TestPickOneReplicaToDeleteSkipsReadOnlySurvivor/refuses_to_delete_the_only_writable_replica_while_a_read-only_survivor_remains === RUN TestPickOneReplicaToDeleteSkipsReadOnlySurvivor/refuses_when_all_replicas_are_read-only === RUN TestPickOneReplicaToDeleteSkipsReadOnlySurvivor/trims_the_smallest_writable_replica_and_never_the_read-only_one --- PASS: TestPickOneReplicaToDeleteSkipsReadOnlySurvivor (0.00s) --- PASS: TestPickOneReplicaToDeleteSkipsReadOnlySurvivor/refuses_to_delete_the_only_writable_replica_while_a_read-only_survivor_remains (0.00s) --- PASS: TestPickOneReplicaToDeleteSkipsReadOnlySurvivor/refuses_when_all_replicas_are_read-only (0.00s) --- PASS: TestPickOneReplicaToDeleteSkipsReadOnlySurvivor/trims_the_smallest_writable_replica_and_never_the_read-only_one (0.00s) === RUN TestPickOneMisplacedVolumeRelocatesReadOnlyReplicas --- PASS: TestPickOneMisplacedVolumeRelocatesReadOnlyReplicas (0.00s) === RUN TestPickOneReplicaToDeletePreservesPlacement --- PASS: TestPickOneReplicaToDeletePreservesPlacement (0.00s) === RUN TestSatisfyReplicaCurrentLocation === RUN TestSatisfyReplicaCurrentLocation/test_001 === RUN TestSatisfyReplicaCurrentLocation/test_010 === RUN TestSatisfyReplicaCurrentLocation/test_011 === RUN TestSatisfyReplicaCurrentLocation/test_110 === RUN TestSatisfyReplicaCurrentLocation/test_100 --- PASS: TestSatisfyReplicaCurrentLocation (0.00s) --- PASS: TestSatisfyReplicaCurrentLocation/test_001 (0.00s) --- PASS: TestSatisfyReplicaCurrentLocation/test_010 (0.00s) --- PASS: TestSatisfyReplicaCurrentLocation/test_011 (0.00s) --- PASS: TestSatisfyReplicaCurrentLocation/test_110 (0.00s) --- PASS: TestSatisfyReplicaCurrentLocation/test_100 (0.00s) === RUN TestParsing --- PASS: TestParsing (0.11s) === RUN TestWriteDataNodeInfo_SplitsCollapsedDisksByPhysicalDiskId --- PASS: TestWriteDataNodeInfo_SplitsCollapsedDisksByPhysicalDiskId (0.00s) === RUN TestWriteTopologyInfo_PrintsParentHeadersOnce --- PASS: TestWriteTopologyInfo_PrintsParentHeadersOnce (0.00s) === RUN TestFindDuplicateVolumeIds === RUN TestFindDuplicateVolumeIds/flags_id_shared_across_collections === RUN TestFindDuplicateVolumeIds/replicas_of_one_volume_are_not_duplicates === RUN TestFindDuplicateVolumeIds/default_empty_collection_counts_as_its_own_collection === RUN TestFindDuplicateVolumeIds/normal_volume_and_ec_shard_sharing_an_id_across_collections --- PASS: TestFindDuplicateVolumeIds (0.00s) --- PASS: TestFindDuplicateVolumeIds/flags_id_shared_across_collections (0.00s) --- PASS: TestFindDuplicateVolumeIds/replicas_of_one_volume_are_not_duplicates (0.00s) --- PASS: TestFindDuplicateVolumeIds/default_empty_collection_counts_as_its_own_collection (0.00s) --- PASS: TestFindDuplicateVolumeIds/normal_volume_and_ec_shard_sharing_an_id_across_collections (0.00s) === RUN TestWriteDuplicateVolumeIdWarning --- PASS: TestWriteDuplicateVolumeIdWarning (0.00s) === RUN TestCollectVolumeMarkTargetsByCollection --- PASS: TestCollectVolumeMarkTargetsByCollection (0.00s) === RUN TestCollectVolumeMarkTargetsByCollectionMatchesDefaultCollection --- PASS: TestCollectVolumeMarkTargetsByCollectionMatchesDefaultCollection (0.00s) === RUN TestCollectVolumeMarkTargetsByCollectionRejectsEmptyCollection --- PASS: TestCollectVolumeMarkTargetsByCollectionRejectsEmptyCollection (0.00s) === RUN TestCollectVolumeMarkTargetsByCollectionReportsMissingCollection --- PASS: TestCollectVolumeMarkTargetsByCollectionReportsMissingCollection (0.00s) === RUN TestVolumeMarkRejectsCollectionWithNodeOrVolumeId --- PASS: TestVolumeMarkRejectsCollectionWithNodeOrVolumeId (0.00s) === RUN TestVolumeMarkRejectsEmptyCollectionFlag --- PASS: TestVolumeMarkRejectsEmptyCollectionFlag (0.00s) === RUN TestEvaluateMergedVolume --- PASS: TestEvaluateMergedVolume (0.00s) === RUN TestMergeNeedleStreamsOrdersByTimestamp --- PASS: TestMergeNeedleStreamsOrdersByTimestamp (0.00s) === RUN TestMergeNeedleStreamsDoesNotDeduplicateAcrossWindows --- PASS: TestMergeNeedleStreamsDoesNotDeduplicateAcrossWindows (0.00s) === RUN TestMergeNeedleStreamsSameStreamDuplicates --- PASS: TestMergeNeedleStreamsSameStreamDuplicates (0.00s) === RUN TestMergeNeedleStreamsWithEmptyStream --- PASS: TestMergeNeedleStreamsWithEmptyStream (0.00s) === RUN TestMergeNeedleStreamsComplexDuplication --- PASS: TestMergeNeedleStreamsComplexDuplication (0.00s) === RUN TestMergeNeedleStreamsTimeWindowDeduplication --- PASS: TestMergeNeedleStreamsTimeWindowDeduplication (0.00s) === RUN TestMergeNeedleStreamsSingleStream --- PASS: TestMergeNeedleStreamsSingleStream (0.00s) === RUN TestMergeNeedleStreamsLargeIDs --- PASS: TestMergeNeedleStreamsLargeIDs (0.00s) === RUN TestMergeNeedleStreamsLastModifiedFallback --- PASS: TestMergeNeedleStreamsLastModifiedFallback (0.00s) === RUN TestMergeWorkflowValidation command_volume_merge_test.go:452: Volume merge workflow validated: 9 stages command_volume_merge_test.go:454: 3_allocate_temporary: Create temporary merge volume on third location (not a current replica) command_volume_merge_test.go:454: 5_tail_and_merge: Tail all replica needles and merge by timestamp, deduplicating command_volume_merge_test.go:454: 6_copy_merged: Copy merged volume back to each original replica location command_volume_merge_test.go:454: 7_delete_temporary: Delete the temporary merge volume from the third location command_volume_merge_test.go:454: 8_restore_writable: Restore writable state for replicas that were originally writable command_volume_merge_test.go:454: 9_verify_completion: Log completion status to user command_volume_merge_test.go:454: 4_mark_readonly: Mark all original replicas as readonly command_volume_merge_test.go:454: 1_collect_replicas: Query master to find all replicas of the target volume command_volume_merge_test.go:454: 2_validate_replicas: Verify at least 2 replicas exist and are healthy --- PASS: TestMergeWorkflowValidation (0.00s) === RUN TestMergeEdgeCaseHandling command_volume_merge_test.go:477: ✓ Edge case handled: network_timeout_during_tail command_volume_merge_test.go:477: ✓ Edge case handled: duplicate_needles_across_streams command_volume_merge_test.go:477: ✓ Edge case handled: target_server_allocation_failure command_volume_merge_test.go:477: ✓ Edge case handled: merge_volume_writeend_failure command_volume_merge_test.go:477: ✓ Edge case handled: replica_already_readonly command_volume_merge_test.go:477: ✓ Edge case handled: different_needle_metadata command_volume_merge_test.go:477: ✓ Edge case handled: concurrent_writes_prevented command_volume_merge_test.go:477: ✓ Edge case handled: duplicate_needles_same_stream command_volume_merge_test.go:477: ✓ Edge case handled: empty_replica_stream command_volume_merge_test.go:477: ✓ Edge case handled: large_volume_memory_efficiency command_volume_merge_test.go:484: All 10 edge cases are handled --- PASS: TestMergeEdgeCaseHandling (0.00s) === RUN TestVolumeServerEvacuate moving volume collection0_15 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection0_21 192.168.1.4:8080 => 192.168.1.6:8080 moving volume collection0_22 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection0_23 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection0_24 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection0_25 192.168.1.4:8080 => 192.168.1.2:8080 moving volume 27 192.168.1.4:8080 => 192.168.1.2:8080 moving volume 28 192.168.1.4:8080 => 192.168.1.2:8080 moving volume 29 192.168.1.4:8080 => 192.168.1.2:8080 moving volume 30 192.168.1.4:8080 => 192.168.1.2:8080 moving volume 31 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_33 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_38 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_51 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_52 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_54 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_63 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_69 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_74 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_80 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_84 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_85 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_97 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_98 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_105 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_106 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_112 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_116 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_119 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_128 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_133 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_136 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_138 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_140 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_144 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_161 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_173 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_174 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_197 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection1_219 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_263 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_272 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_291 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_299 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_301 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_302 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_339 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_345 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_355 192.168.1.4:8080 => 192.168.1.2:8080 moving volume collection2_373 192.168.1.4:8080 => 192.168.1.2:8080 --- PASS: TestVolumeServerEvacuate (0.00s) === RUN TestFindRemoteVolumeInTopology === RUN TestFindRemoteVolumeInTopology/find_existing_remote_volume === RUN TestFindRemoteVolumeInTopology/find_remote_volume_with_different_backend === RUN TestFindRemoteVolumeInTopology/find_remote_volume_on_another_server === RUN TestFindRemoteVolumeInTopology/local_volume_not_found_as_remote === RUN TestFindRemoteVolumeInTopology/non-existent_volume_not_found === RUN TestFindRemoteVolumeInTopology/filter_by_matching_collection_exact === RUN TestFindRemoteVolumeInTopology/filter_by_matching_collection_regex === RUN TestFindRemoteVolumeInTopology/filter_by_non-matching_collection --- PASS: TestFindRemoteVolumeInTopology (0.00s) --- PASS: TestFindRemoteVolumeInTopology/find_existing_remote_volume (0.00s) --- PASS: TestFindRemoteVolumeInTopology/find_remote_volume_with_different_backend (0.00s) --- PASS: TestFindRemoteVolumeInTopology/find_remote_volume_on_another_server (0.00s) --- PASS: TestFindRemoteVolumeInTopology/local_volume_not_found_as_remote (0.00s) --- PASS: TestFindRemoteVolumeInTopology/non-existent_volume_not_found (0.00s) --- PASS: TestFindRemoteVolumeInTopology/filter_by_matching_collection_exact (0.00s) --- PASS: TestFindRemoteVolumeInTopology/filter_by_matching_collection_regex (0.00s) --- PASS: TestFindRemoteVolumeInTopology/filter_by_non-matching_collection (0.00s) === RUN TestFindRemoteVolumeInTopologyInvalidPattern --- PASS: TestFindRemoteVolumeInTopologyInvalidPattern (0.00s) === RUN TestCollectRemoteVolumesWithInfo === RUN TestCollectRemoteVolumesWithInfo/empty_pattern_matches_empty_collection_only === RUN TestCollectRemoteVolumesWithInfo/match_all_collections === RUN TestCollectRemoteVolumesWithInfo/match_specific_collection === RUN TestCollectRemoteVolumesWithInfo/match_collection_prefix === RUN TestCollectRemoteVolumesWithInfo/match_bucket_collection === RUN TestCollectRemoteVolumesWithInfo/no_match --- PASS: TestCollectRemoteVolumesWithInfo (0.00s) --- PASS: TestCollectRemoteVolumesWithInfo/empty_pattern_matches_empty_collection_only (0.00s) --- PASS: TestCollectRemoteVolumesWithInfo/match_all_collections (0.00s) --- PASS: TestCollectRemoteVolumesWithInfo/match_specific_collection (0.00s) --- PASS: TestCollectRemoteVolumesWithInfo/match_collection_prefix (0.00s) --- PASS: TestCollectRemoteVolumesWithInfo/match_bucket_collection (0.00s) --- PASS: TestCollectRemoteVolumesWithInfo/no_match (0.00s) === RUN TestCollectRemoteVolumesWithInfoCaptures --- PASS: TestCollectRemoteVolumesWithInfoCaptures (0.00s) === RUN TestCollectRemoteVolumesWithInfoInvalidPattern --- PASS: TestCollectRemoteVolumesWithInfoInvalidPattern (0.00s) === RUN TestVolumeSelectionByDataCenter collect volumes quiet for: 0 seconds collect volumes quiet for: 0 seconds --- PASS: TestVolumeSelectionByDataCenter (0.00s) === RUN TestFilterLocationsByDataCenter --- PASS: TestFilterLocationsByDataCenter (0.00s) === RUN TestCollectVolumeTierUploadLocationsPrefersTieredReplica --- PASS: TestCollectVolumeTierUploadLocationsPrefersTieredReplica (0.00s) === RUN TestCollectVolumeTierUploadLocationsFiltersCollection --- PASS: TestCollectVolumeTierUploadLocationsFiltersCollection (0.00s) === RUN TestECRebalanceWithLimitedSlots ec_rebalance_slots_test.go:38: Topology summary: ec_rebalance_slots_test.go:39: Number of EC nodes: 6 ec_rebalance_slots_test.go:40: Total free EC slots: 432 ec_rebalance_slots_test.go:50: Node 127.0.0.1:8080 (rack rack0): 28 shards, 52 free slots ec_rebalance_slots_test.go:50: Node 127.0.0.1:8081 (rack rack1): 28 shards, 52 free slots ec_rebalance_slots_test.go:50: Node 127.0.0.1:8082 (rack rack2): 14 shards, 76 free slots ec_rebalance_slots_test.go:50: Node 127.0.0.1:8083 (rack rack3): 28 shards, 52 free slots ec_rebalance_slots_test.go:50: Node 127.0.0.1:8084 (rack rack4): 0 shards, 100 free slots ec_rebalance_slots_test.go:50: Node 127.0.0.1:8085 (rack rack5): 0 shards, 100 free slots ec_rebalance_slots_test.go:63: Total EC shards: 98 ec_rebalance_slots_test.go:105: totalFreeEcSlots = 432, rebalance should be possible --- PASS: TestECRebalanceWithLimitedSlots (0.00s) === RUN TestECRebalanceZeroFreeSlots ec_rebalance_slots_test.go:120: Zero free slots scenario: ec_rebalance_slots_test.go:128: Node 127.0.0.1:8080: 28 shards, 0 free slots, volumeCount=10, max=10 ec_rebalance_slots_test.go:128: Node 127.0.0.1:8081: 28 shards, 0 free slots, volumeCount=10, max=10 ec_rebalance_slots_test.go:128: Node 127.0.0.1:8082: 14 shards, 0 free slots, volumeCount=10, max=10 ec_rebalance_slots_test.go:128: Node 127.0.0.1:8083: 28 shards, 0 free slots, volumeCount=10, max=10 ec_rebalance_slots_test.go:133: Total free slots: 0 ec_rebalance_slots_test.go:136: ec_rebalance_slots_test.go:137: SCENARIO REPRODUCED: totalFreeEcSlots = 0 ec_rebalance_slots_test.go:138: This would trigger: 'no free ec shard slots. only 0 left' --- PASS: TestECRebalanceZeroFreeSlots (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/shell 0.505s === RUN TestSetDiskStatusSuccess --- PASS: TestSetDiskStatusSuccess (0.00s) === RUN TestSetDiskStatusReportsRepeatedFailures --- PASS: TestSetDiskStatusReportsRepeatedFailures (0.00s) === RUN TestSetDiskStatusRequiresRepeatedSuccessesToRecover --- PASS: TestSetDiskStatusRequiresRepeatedSuccessesToRecover (0.00s) === RUN TestRobinCounter --- PASS: TestRobinCounter (0.00s) === RUN TestUpdateBucketQuotaMetrics I0720 21:30:02.313963 metrics.go:1052 delete bucket metrics, quota-test-bucket: 2 --- PASS: TestUpdateBucketQuotaMetrics (0.01s) === RUN TestECRebuildMetricsRegistered --- PASS: TestECRebuildMetricsRegistered (0.00s) === RUN TestECRebuildCounterIncrement --- PASS: TestECRebuildCounterIncrement (0.00s) === RUN TestReplicationMetricsRegistered --- PASS: TestReplicationMetricsRegistered (0.00s) === RUN TestReplicationCounterIncrement --- PASS: TestReplicationCounterIncrement (0.00s) === RUN TestReplicationTargetsHistogram --- PASS: TestReplicationTargetsHistogram (0.01s) === RUN TestUnderReplicatedVolumesGauge --- PASS: TestUnderReplicatedVolumesGauge (0.00s) === RUN TestReplicationFailuresCounter --- PASS: TestReplicationFailuresCounter (0.00s) === RUN TestBuildInfo metrics_buildinfo_test.go:72: BuildInfo metric: version=4.40, commit=, sizelimit=30GB, goos=linux, goarch=amd64 --- PASS: TestBuildInfo (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/stats 0.043s === RUN TestCalculateExpectedShardSizeWithRealEncoding === RUN TestCalculateExpectedShardSizeWithRealEncoding/5MB_file I0720 21:30:02.350242 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeWithRealEncoding1275933725/001/test_volume.dat size:5242880 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:118: ✓ SUCCESS: .dat size 5242880 → actual shard size 1048576 matches calculated size (Small file that needs 1 small block per shard) === RUN TestCalculateExpectedShardSizeWithRealEncoding/10MB_file_(exactly_10_small_blocks) I0720 21:30:02.377876 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeWithRealEncoding1275933725/001/test_volume.dat size:10485760 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:118: ✓ SUCCESS: .dat size 10485760 → actual shard size 1048576 matches calculated size (Exactly fits in 1MB small blocks) === RUN TestCalculateExpectedShardSizeWithRealEncoding/15MB_file I0720 21:30:02.414204 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeWithRealEncoding1275933725/001/test_volume.dat size:15728640 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:118: ✓ SUCCESS: .dat size 15728640 → actual shard size 2097152 matches calculated size (Requires 2 small blocks per shard) === RUN TestCalculateExpectedShardSizeWithRealEncoding/50MB_file I0720 21:30:02.469896 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeWithRealEncoding1275933725/001/test_volume.dat size:52428800 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:118: ✓ SUCCESS: .dat size 52428800 → actual shard size 5242880 matches calculated size (Requires 5 small blocks per shard) === RUN TestCalculateExpectedShardSizeWithRealEncoding/100MB_file I0720 21:30:02.595706 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeWithRealEncoding1275933725/001/test_volume.dat size:104857600 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:118: ✓ SUCCESS: .dat size 104857600 → actual shard size 10485760 matches calculated size (Requires 10 small blocks per shard) === RUN TestCalculateExpectedShardSizeWithRealEncoding/512MB_file I0720 21:30:03.034332 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeWithRealEncoding1275933725/001/test_volume.dat size:536870912 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:118: ✓ SUCCESS: .dat size 536870912 → actual shard size 54525952 matches calculated size (Requires 52 small blocks per shard (rounded up)) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding (1.25s) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding/5MB_file (0.03s) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding/10MB_file_(exactly_10_small_blocks) (0.03s) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding/15MB_file (0.04s) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding/50MB_file (0.10s) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding/100MB_file (0.19s) --- PASS: TestCalculateExpectedShardSizeWithRealEncoding/512MB_file (0.87s) === RUN TestCalculateExpectedShardSizeEdgeCases === RUN TestCalculateExpectedShardSizeEdgeCases/1_byte_file I0720 21:30:03.594252 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/1 byte file.dat size:1 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 1 → shard size 1048576 (correct) === RUN TestCalculateExpectedShardSizeEdgeCases/1KB_file I0720 21:30:03.608252 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/1KB file.dat size:1024 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 1024 → shard size 1048576 (correct) === RUN TestCalculateExpectedShardSizeEdgeCases/10KB_file I0720 21:30:03.626909 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/10KB file.dat size:10240 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 10240 → shard size 1048576 (correct) === RUN TestCalculateExpectedShardSizeEdgeCases/1MB_file_(1_small_block) I0720 21:30:03.642557 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/1MB file (1 small block).dat size:1048576 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 1048576 → shard size 1048576 (correct) === RUN TestCalculateExpectedShardSizeEdgeCases/1MB_+_1_byte I0720 21:30:03.660697 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/1MB + 1 byte.dat size:1048577 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 1048577 → shard size 1048576 (correct) === RUN TestCalculateExpectedShardSizeEdgeCases/9.9MB_(almost_1_small_block_per_shard) I0720 21:30:03.692005 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/9.9MB (almost 1 small block per shard).dat size:10358784 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 10358784 → shard size 1048576 (correct) === RUN TestCalculateExpectedShardSizeEdgeCases/10.1MB_(just_over_1_small_block_per_shard) I0720 21:30:03.721873 ec_encoder.go:138 encodeDatFile /tmp/TestCalculateExpectedShardSizeEdgeCases213676561/001/10.1MB (just over 1 small block per shard).dat size:10588160 with EC context 10+4 (total: 14) disk_location_ec_realworld_test.go:188: ✓ File size 10588160 → shard size 2097152 (correct) --- PASS: TestCalculateExpectedShardSizeEdgeCases (0.16s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/1_byte_file (0.01s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/1KB_file (0.02s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/10KB_file (0.01s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/1MB_file_(1_small_block) (0.02s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/1MB_+_1_byte (0.02s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/9.9MB_(almost_1_small_block_per_shard) (0.03s) --- PASS: TestCalculateExpectedShardSizeEdgeCases/10.1MB_(just_over_1_small_block_per_shard) (0.05s) === RUN TestCalculateExpectedShardSize === RUN TestCalculateExpectedShardSize/0_bytes_(empty_file) disk_location_ec_shard_size_test.go:138: ✓ File size: 0 → Shard size: 0 (Empty file has 0 shard size) === RUN TestCalculateExpectedShardSize/Exact_10GB_(1_large_batch) disk_location_ec_shard_size_test.go:138: ✓ File size: 10737418240 → Shard size: 1073741824 (Exactly fits in large blocks) === RUN TestCalculateExpectedShardSize/Exact_20GB_(2_large_batches) disk_location_ec_shard_size_test.go:138: ✓ File size: 21474836480 → Shard size: 2147483648 (2 complete large batches) === RUN TestCalculateExpectedShardSize/Just_under_large_batch_(10GB_-_1_byte) disk_location_ec_shard_size_test.go:138: ✓ File size: 10737418239 → Shard size: 1073741824 (Just under 10GB needs 1024 small blocks) === RUN TestCalculateExpectedShardSize/Just_over_large_batch_(10GB_+_1_byte) disk_location_ec_shard_size_test.go:138: ✓ File size: 10737418241 → Shard size: 1074790400 (Just over 10GB adds 1 small block) === RUN TestCalculateExpectedShardSize/Exact_10MB_(1_small_batch) disk_location_ec_shard_size_test.go:138: ✓ File size: 10485760 → Shard size: 1048576 (Exactly fits in 1 small batch) === RUN TestCalculateExpectedShardSize/Exact_20MB_(2_small_batches) disk_location_ec_shard_size_test.go:138: ✓ File size: 20971520 → Shard size: 2097152 (2 complete small batches) === RUN TestCalculateExpectedShardSize/Just_under_small_batch_(10MB_-_1_byte) disk_location_ec_shard_size_test.go:138: ✓ File size: 10485759 → Shard size: 1048576 (Just under 10MB rounds up to 1 small block) === RUN TestCalculateExpectedShardSize/Just_over_small_batch_(10MB_+_1_byte) disk_location_ec_shard_size_test.go:138: ✓ File size: 10485761 → Shard size: 2097152 (Just over 10MB needs 2 small blocks) === RUN TestCalculateExpectedShardSize/10GB_+_1MB disk_location_ec_shard_size_test.go:138: ✓ File size: 10738466816 → Shard size: 1074790400 (1 large batch + 1MB needs 1 small block) === RUN TestCalculateExpectedShardSize/10GB_+_5MB disk_location_ec_shard_size_test.go:138: ✓ File size: 10742661120 → Shard size: 1074790400 (1 large batch + 5MB rounds up to 1 small block) === RUN TestCalculateExpectedShardSize/10GB_+_15MB disk_location_ec_shard_size_test.go:138: ✓ File size: 10753146880 → Shard size: 1075838976 (1 large batch + 15MB needs 2 small blocks) === RUN TestCalculateExpectedShardSize/11GB_(1_large_batch_+_103_small_blocks) disk_location_ec_shard_size_test.go:138: ✓ File size: 11811160064 → Shard size: 1181745152 (1GB large + 1GB remaining needs 103 small blocks) === RUN TestCalculateExpectedShardSize/5MB_(requires_1_small_block_per_shard) disk_location_ec_shard_size_test.go:138: ✓ File size: 5242880 → Shard size: 1048576 (Small file rounds up to 1MB per shard) === RUN TestCalculateExpectedShardSize/1KB_(minimum_size) disk_location_ec_shard_size_test.go:138: ✓ File size: 1024 → Shard size: 1048576 (Tiny file needs 1 small block) === RUN TestCalculateExpectedShardSize/10.5GB_(mixed) disk_location_ec_shard_size_test.go:138: ✓ File size: 11274289152 → Shard size: 1128267776 (1GB large + 512MB remaining needs 52 small blocks) --- PASS: TestCalculateExpectedShardSize (0.00s) --- PASS: TestCalculateExpectedShardSize/0_bytes_(empty_file) (0.00s) --- PASS: TestCalculateExpectedShardSize/Exact_10GB_(1_large_batch) (0.00s) --- PASS: TestCalculateExpectedShardSize/Exact_20GB_(2_large_batches) (0.00s) --- PASS: TestCalculateExpectedShardSize/Just_under_large_batch_(10GB_-_1_byte) (0.00s) --- PASS: TestCalculateExpectedShardSize/Just_over_large_batch_(10GB_+_1_byte) (0.00s) --- PASS: TestCalculateExpectedShardSize/Exact_10MB_(1_small_batch) (0.00s) --- PASS: TestCalculateExpectedShardSize/Exact_20MB_(2_small_batches) (0.00s) --- PASS: TestCalculateExpectedShardSize/Just_under_small_batch_(10MB_-_1_byte) (0.00s) --- PASS: TestCalculateExpectedShardSize/Just_over_small_batch_(10MB_+_1_byte) (0.00s) --- PASS: TestCalculateExpectedShardSize/10GB_+_1MB (0.00s) --- PASS: TestCalculateExpectedShardSize/10GB_+_5MB (0.00s) --- PASS: TestCalculateExpectedShardSize/10GB_+_15MB (0.00s) --- PASS: TestCalculateExpectedShardSize/11GB_(1_large_batch_+_103_small_blocks) (0.00s) --- PASS: TestCalculateExpectedShardSize/5MB_(requires_1_small_block_per_shard) (0.00s) --- PASS: TestCalculateExpectedShardSize/1KB_(minimum_size) (0.00s) --- PASS: TestCalculateExpectedShardSize/10.5GB_(mixed) (0.00s) === RUN TestShardSizeValidationScenarios === RUN TestShardSizeValidationScenarios/Valid:_exact_match_for_10GB === RUN TestShardSizeValidationScenarios/Invalid:_1_byte_too_small === RUN TestShardSizeValidationScenarios/Invalid:_1_byte_too_large === RUN TestShardSizeValidationScenarios/Valid:_small_file_exact_match === RUN TestShardSizeValidationScenarios/Invalid:_wrong_size_for_small_file --- PASS: TestShardSizeValidationScenarios (0.00s) --- PASS: TestShardSizeValidationScenarios/Valid:_exact_match_for_10GB (0.00s) --- PASS: TestShardSizeValidationScenarios/Invalid:_1_byte_too_small (0.00s) --- PASS: TestShardSizeValidationScenarios/Invalid:_1_byte_too_large (0.00s) --- PASS: TestShardSizeValidationScenarios/Valid:_small_file_exact_match (0.00s) --- PASS: TestShardSizeValidationScenarios/Invalid:_wrong_size_for_small_file (0.00s) === RUN TestIncompleteEcEncodingCleanup === RUN TestIncompleteEcEncodingCleanup/Incomplete_EC:_shards_without_.ecx,_.dat_exists_-_should_cleanup W0720 21:30:03.754802 disk_location_ec.go:420 Found 14 EC shards without .ecx file for volume 100 (incomplete encoding interrupted before .ecx creation), cleaning up... === RUN TestIncompleteEcEncodingCleanup/Distributed_EC:_shards_without_.ecx,_.dat_deleted_-_should_NOT_cleanup === RUN TestIncompleteEcEncodingCleanup/Distributed_EC:_full-size_shards_with_.ecx,_<_10_of_them,_.dat_exists_-_keep W0720 21:30:03.755712 ec_volume.go:182 vif file not found, using defaults, volumeId:102, filename:/tmp/TestIncompleteEcEncodingCleanupDistributed_EC_full-size_shards3562115644/001/102.vif W0720 21:30:03.755927 ec_volume.go:182 vif file not found, using defaults, volumeId:102, filename:/tmp/TestIncompleteEcEncodingCleanupDistributed_EC_full-size_shards3562115644/001/102.vif === RUN TestIncompleteEcEncodingCleanup/Valid_local_EC:_shards_with_.ecx,_>=_10_shards,_.dat_exists_-_should_load W0720 21:30:03.756444 ec_volume.go:182 vif file not found, using defaults, volumeId:103, filename:/tmp/TestIncompleteEcEncodingCleanupValid_local_EC_shards_with_.ecx3991677726/001/103.vif W0720 21:30:03.756718 ec_volume.go:182 vif file not found, using defaults, volumeId:103, filename:/tmp/TestIncompleteEcEncodingCleanupValid_local_EC_shards_with_.ecx3991677726/001/103.vif === RUN TestIncompleteEcEncodingCleanup/Distributed_EC:_shards_with_.ecx,_.dat_deleted_-_should_load W0720 21:30:03.757222 ec_volume.go:182 vif file not found, using defaults, volumeId:104, filename:/tmp/TestIncompleteEcEncodingCleanupDistributed_EC_shards_with_.ecx3331719290/001/104.vif W0720 21:30:03.757390 ec_volume.go:182 vif file not found, using defaults, volumeId:104, filename:/tmp/TestIncompleteEcEncodingCleanupDistributed_EC_shards_with_.ecx3331719290/001/104.vif === RUN TestIncompleteEcEncodingCleanup/Incomplete_EC_with_collection:_shards_without_.ecx,_.dat_exists_-_should_cleanup W0720 21:30:03.757868 disk_location_ec.go:420 Found 14 EC shards without .ecx file for volume 105 (incomplete encoding interrupted before .ecx creation), cleaning up... --- PASS: TestIncompleteEcEncodingCleanup (0.00s) --- PASS: TestIncompleteEcEncodingCleanup/Incomplete_EC:_shards_without_.ecx,_.dat_exists_-_should_cleanup (0.00s) --- PASS: TestIncompleteEcEncodingCleanup/Distributed_EC:_shards_without_.ecx,_.dat_deleted_-_should_NOT_cleanup (0.00s) --- PASS: TestIncompleteEcEncodingCleanup/Distributed_EC:_full-size_shards_with_.ecx,_<_10_of_them,_.dat_exists_-_keep (0.00s) --- PASS: TestIncompleteEcEncodingCleanup/Valid_local_EC:_shards_with_.ecx,_>=_10_shards,_.dat_exists_-_should_load (0.00s) --- PASS: TestIncompleteEcEncodingCleanup/Distributed_EC:_shards_with_.ecx,_.dat_deleted_-_should_load (0.00s) --- PASS: TestIncompleteEcEncodingCleanup/Incomplete_EC_with_collection:_shards_without_.ecx,_.dat_exists_-_should_cleanup (0.00s) === RUN TestValidateEcVolume === RUN TestValidateEcVolume/Valid:_.dat_exists_with_10+_shards === RUN TestValidateEcVolume/Keep:_.dat_exists_with_<_10_full-size_shards_(possible_distributed_sole_copies) === RUN TestValidateEcVolume/Valid:_.dat_deleted_(distributed_EC)_with_any_shards === RUN TestValidateEcVolume/Valid:_.dat_deleted_(distributed_EC)_with_no_shards === RUN TestValidateEcVolume/Invalid:_zero-byte_shard_files_should_not_count === RUN TestValidateEcVolume/Keep:_.dat_exists_with_different_size_shards_(inconsistent,_not_trusted_for_deletion) W0720 21:30:03.759489 disk_location_ec.go:497 EC volume 205 shard 1 size 110 != 100; keeping EC shards --- PASS: TestValidateEcVolume (0.00s) --- PASS: TestValidateEcVolume/Valid:_.dat_exists_with_10+_shards (0.00s) --- PASS: TestValidateEcVolume/Keep:_.dat_exists_with_<_10_full-size_shards_(possible_distributed_sole_copies) (0.00s) --- PASS: TestValidateEcVolume/Valid:_.dat_deleted_(distributed_EC)_with_any_shards (0.00s) --- PASS: TestValidateEcVolume/Valid:_.dat_deleted_(distributed_EC)_with_no_shards (0.00s) --- PASS: TestValidateEcVolume/Invalid:_zero-byte_shard_files_should_not_count (0.00s) --- PASS: TestValidateEcVolume/Keep:_.dat_exists_with_different_size_shards_(inconsistent,_not_trusted_for_deletion) (0.00s) === RUN TestRemoveEcVolumeFiles === RUN TestRemoveEcVolumeFiles/Same_directory_for_data_and_index === RUN TestRemoveEcVolumeFiles/Separate_idx_directory --- PASS: TestRemoveEcVolumeFiles (0.00s) --- PASS: TestRemoveEcVolumeFiles/Same_directory_for_data_and_index (0.00s) --- PASS: TestRemoveEcVolumeFiles/Separate_idx_directory (0.00s) === RUN TestEcCleanupWithSeparateIdxDirectory W0720 21:30:03.761226 disk_location_ec.go:420 Found 14 EC shards without .ecx file for volume 400 (incomplete encoding interrupted before .ecx creation), cleaning up... --- PASS: TestEcCleanupWithSeparateIdxDirectory (0.00s) === RUN TestDistributedEcVolumeNoFileDeletion W0720 21:30:03.761772 ec_volume.go:182 vif file not found, using defaults, volumeId:500, filename:/tmp/TestDistributedEcVolumeNoFileDeletion2319897040/001/500.vif disk_location_ec_test.go:673: SUCCESS: Distributed EC volume files preserved (not deleted) --- PASS: TestDistributedEcVolumeNoFileDeletion (0.00s) === RUN TestLoadExistingVolumeSkipsVifWhenEcxPresent === RUN TestLoadExistingVolumeSkipsVifWhenEcxPresent/same-idx-dir === RUN TestLoadExistingVolumeSkipsVifWhenEcxPresent/split-idx-dir --- PASS: TestLoadExistingVolumeSkipsVifWhenEcxPresent (0.00s) --- PASS: TestLoadExistingVolumeSkipsVifWhenEcxPresent/same-idx-dir (0.00s) --- PASS: TestLoadExistingVolumeSkipsVifWhenEcxPresent/split-idx-dir (0.00s) === RUN TestUnUsedSpace --- PASS: TestUnUsedSpace (0.00s) === RUN TestResolveVolumeIDs --- PASS: TestResolveVolumeIDs (0.00s) === RUN TestValidateEcVolume_PartialDatNextToFullShardsKeeps --- PASS: TestValidateEcVolume_PartialDatNextToFullShardsKeeps (0.00s) === RUN TestValidateEcVolume_InterruptedEncodeReclaimsDat W0720 21:30:03.763007 disk_location_ec.go:520 EC volume 71: 10 shards of 1048576 bytes are smaller than the .dat's full encode (3145728 bytes); reclaiming the complete .dat --- PASS: TestValidateEcVolume_InterruptedEncodeReclaimsDat (0.00s) === RUN TestValidateEcVolume_TransientStatErrorKeeps E0720 21:30:03.763208 file_util.go:90 check /tmp/TestValidateEcVolume_TransientStatErrorKeeps151924596/001/locked/73.vif: stat /tmp/TestValidateEcVolume_TransientStatErrorKeeps151924596/001/locked/73.vif: permission denied W0720 21:30:03.763400 volume_info.go:25 can not read /tmp/TestValidateEcVolume_TransientStatErrorKeeps151924596/001/locked/73.vif W0720 21:30:03.763423 disk_location_ec.go:482 EC volume 73: cannot stat .dat /tmp/TestValidateEcVolume_TransientStatErrorKeeps151924596/001/locked/73.dat (stat /tmp/TestValidateEcVolume_TransientStatErrorKeeps151924596/001/locked/73.dat: permission denied); keeping EC shards --- PASS: TestValidateEcVolume_TransientStatErrorKeeps (0.00s) === RUN TestGenerateLevelDbFileStaleWatermarkRebuilds W0720 21:30:03.766351 needle_map_leveldb.go:128 stale watermark 10000 for /tmp/TestGenerateLevelDbFileStaleWatermarkRebuilds1762746813/001/1.ldb (filesize 48); rebuilding leveldb from start --- PASS: TestGenerateLevelDbFileStaleWatermarkRebuilds (0.00s) === RUN TestLevelDbNeedleMap_Concurrency --- PASS: TestLevelDbNeedleMap_Concurrency (0.01s) === RUN TestFastLoadingNeedleMapMetrics I0720 21:30:03.798036 needle_map_metric_test.go:27 FileCount expected 10000 actual 11998 I0720 21:30:03.798100 needle_map_metric_test.go:28 DeletedSize expected 1683 actual 1683 I0720 21:30:03.798115 needle_map_metric_test.go:29 ContentSize expected 10000 actual 10000 I0720 21:30:03.798128 needle_map_metric_test.go:30 DeletedCount expected 1683 actual 3681 I0720 21:30:03.798140 needle_map_metric_test.go:31 MaxFileKey expected 10000 actual 10000 --- PASS: TestFastLoadingNeedleMapMetrics (0.02s) === RUN TestSortedFileNeedleMap_DeleteAppendsTombstone I0720 21:30:03.798397 needle_map_sorted_file.go:26 Start to Generate /tmp/TestSortedFileNeedleMap_DeleteAppendsTombstone2151461545/001/v1.sdx from /tmp/TestSortedFileNeedleMap_DeleteAppendsTombstone2151461545/001/v1.idx I0720 21:30:03.799269 needle_map_sorted_file.go:28 Finished Generating /tmp/TestSortedFileNeedleMap_DeleteAppendsTombstone2151461545/001/v1.sdx from /tmp/TestSortedFileNeedleMap_DeleteAppendsTombstone2151461545/001/v1.idx --- PASS: TestSortedFileNeedleMap_DeleteAppendsTombstone (0.00s) === RUN TestRemoteTier_DiskScanLoadsRemoteOnlyVolume I0720 21:30:03.800506 volume_tier.go:31 volume 44 is tiered to test_local_dir.default as obj-1784583003800155562-64756 and read only I0720 21:30:03.800539 volume_loading.go:163 loading volume 44 from remote files:{backend_type:"test_local_dir" backend_id:"default" key:"obj-1784583003800155562-64756" file_size:3224 modified_time:1784583003 extension:".dat"} version:3 bytes_offset:4 E0720 21:30:03.800590 volume_loading.go:229 readSuperBlock remote volume 44: I0720 21:30:03.800609 needle_map_sorted_file.go:26 Start to Generate /tmp/TestRemoteTier_DiskScanLoadsRemoteOnlyVolume350124128/002/44.sdx from /tmp/TestRemoteTier_DiskScanLoadsRemoteOnlyVolume350124128/002/44.idx I0720 21:30:03.803590 needle_map_sorted_file.go:28 Finished Generating /tmp/TestRemoteTier_DiskScanLoadsRemoteOnlyVolume350124128/002/44.sdx from /tmp/TestRemoteTier_DiskScanLoadsRemoteOnlyVolume350124128/002/44.idx --- PASS: TestRemoteTier_DiskScanLoadsRemoteOnlyVolume (0.00s) === RUN TestRemoteTier_LiveTierUpload_StillReportsToMaster --- PASS: TestRemoteTier_LiveTierUpload_StillReportsToMaster (0.00s) === RUN TestRemoteTier_ReloadUnderDataLock_NoDeadlock I0720 21:30:03.804542 volume_tier.go:31 volume 68 is tiered to test_local_dir.default as obj-1784583003804403942-64756 and read only I0720 21:30:03.804548 volume_loading.go:163 loading volume 68 from remote files:{backend_type:"test_local_dir" backend_id:"default" key:"obj-1784583003804403942-64756" file_size:2864 modified_time:1784583003 extension:".dat"} version:3 bytes_offset:4 E0720 21:30:03.804574 volume_loading.go:229 readSuperBlock remote volume 68: I0720 21:30:03.804595 needle_map_sorted_file.go:26 Start to Generate /tmp/TestRemoteTier_ReloadUnderDataLock_NoDeadlock1600229733/002/68.sdx from /tmp/TestRemoteTier_ReloadUnderDataLock_NoDeadlock1600229733/002/68.idx I0720 21:30:03.805106 needle_map_sorted_file.go:28 Finished Generating /tmp/TestRemoteTier_ReloadUnderDataLock_NoDeadlock1600229733/002/68.sdx from /tmp/TestRemoteTier_ReloadUnderDataLock_NoDeadlock1600229733/002/68.idx --- PASS: TestRemoteTier_ReloadUnderDataLock_NoDeadlock (0.00s) === RUN TestRemoteTier_Move_KeepsRemoteObject I0720 21:30:03.805501 volume_tier.go:31 volume 31 is tiered to test_local_dir.default as obj-1784583003805385693-64756 and read only I0720 21:30:03.805507 volume_loading.go:163 loading volume 31 from remote files:{backend_type:"test_local_dir" backend_id:"default" key:"obj-1784583003805385693-64756" file_size:3096 modified_time:1784583003 extension:".dat"} version:3 bytes_offset:4 E0720 21:30:03.805529 volume_loading.go:229 readSuperBlock remote volume 31: I0720 21:30:03.805561 needle_map_sorted_file.go:26 Start to Generate /tmp/TestRemoteTier_Move_KeepsRemoteObject2283285108/002/31.sdx from /tmp/TestRemoteTier_Move_KeepsRemoteObject2283285108/002/31.idx I0720 21:30:03.805943 needle_map_sorted_file.go:28 Finished Generating /tmp/TestRemoteTier_Move_KeepsRemoteObject2283285108/002/31.sdx from /tmp/TestRemoteTier_Move_KeepsRemoteObject2283285108/002/31.idx I0720 21:30:03.805993 volume_write.go:132 removed volume file /tmp/TestRemoteTier_Move_KeepsRemoteObject2283285108/002/31.idx (size=80) --- PASS: TestRemoteTier_Move_KeepsRemoteObject (0.00s) === RUN TestRemoteTier_RealDelete_RemovesRemoteObject I0720 21:30:03.806440 volume_tier.go:31 volume 32 is tiered to test_local_dir.default as obj-1784583003806333050-64756 and read only I0720 21:30:03.806447 volume_loading.go:163 loading volume 32 from remote files:{backend_type:"test_local_dir" backend_id:"default" key:"obj-1784583003806333050-64756" file_size:2120 modified_time:1784583003 extension:".dat"} version:3 bytes_offset:4 E0720 21:30:03.806469 volume_loading.go:229 readSuperBlock remote volume 32: I0720 21:30:03.806487 needle_map_sorted_file.go:26 Start to Generate /tmp/TestRemoteTier_RealDelete_RemovesRemoteObject117059272/002/32.sdx from /tmp/TestRemoteTier_RealDelete_RemovesRemoteObject117059272/002/32.idx I0720 21:30:03.807273 needle_map_sorted_file.go:28 Finished Generating /tmp/TestRemoteTier_RealDelete_RemovesRemoteObject117059272/002/32.sdx from /tmp/TestRemoteTier_RealDelete_RemovesRemoteObject117059272/002/32.idx I0720 21:30:03.807345 volume_write.go:132 removed volume file /tmp/TestRemoteTier_RealDelete_RemovesRemoteObject117059272/002/32.idx (size=80) --- PASS: TestRemoteTier_RealDelete_RemovesRemoteObject (0.00s) === RUN TestRemoteTier_Vacuum_DoesNotDeleteRemote I0720 21:30:03.807791 volume_tier.go:31 volume 33 is tiered to test_local_dir.default as obj-1784583003807680539-64756 and read only I0720 21:30:03.807796 volume_loading.go:163 loading volume 33 from remote files:{backend_type:"test_local_dir" backend_id:"default" key:"obj-1784583003807680539-64756" file_size:2512 modified_time:1784583003 extension:".dat"} version:3 bytes_offset:4 E0720 21:30:03.807817 volume_loading.go:229 readSuperBlock remote volume 33: I0720 21:30:03.807839 needle_map_sorted_file.go:26 Start to Generate /tmp/TestRemoteTier_Vacuum_DoesNotDeleteRemote3026892596/002/33.sdx from /tmp/TestRemoteTier_Vacuum_DoesNotDeleteRemote3026892596/002/33.idx I0720 21:30:03.812051 needle_map_sorted_file.go:28 Finished Generating /tmp/TestRemoteTier_Vacuum_DoesNotDeleteRemote3026892596/002/33.sdx from /tmp/TestRemoteTier_Vacuum_DoesNotDeleteRemote3026892596/002/33.idx I0720 21:30:03.812634 volume_tier.go:31 volume 33 is tiered to test_local_dir.default as obj-1784583003807680539-64756 and read only I0720 21:30:03.812644 volume_loading.go:163 loading volume 33 from remote files:{backend_type:"test_local_dir" backend_id:"default" key:"obj-1784583003807680539-64756" file_size:2512 modified_time:1784583003 extension:".dat"} version:3 bytes_offset:4 E0720 21:30:03.812689 volume_loading.go:229 readSuperBlock remote volume 33: --- PASS: TestRemoteTier_Vacuum_DoesNotDeleteRemote (0.01s) === RUN TestRemoteTier_ECEncode_RequiresLocalDat --- PASS: TestRemoteTier_ECEncode_RequiresLocalDat (0.00s) === RUN TestRemoteTier_ECEncodeDecode_AfterDownload I0720 21:30:03.817080 ec_encoder.go:138 encodeDatFile /tmp/TestRemoteTier_ECEncodeDecode_AfterDownload1412159076/002/35.dat size:3072 with EC context 10+4 (total: 14) I0720 21:30:03.831599 ec_encoder.go:108 Rebuilding EC files for /tmp/TestRemoteTier_ECEncodeDecode_AfterDownload1412159076/002/35 with default config I0720 21:30:03.831726 ec_encoder.go:254 rebuilding /tmp/TestRemoteTier_ECEncodeDecode_AfterDownload1412159076/002/35: 10 shards present, 4 missing [10 11 12 13], config 10+4 (total: 14) --- PASS: TestRemoteTier_ECEncodeDecode_AfterDownload (0.02s) === RUN TestHasFreeDiskLocation === RUN TestHasFreeDiskLocation/low_disk_space_prevents_allocation === RUN TestHasFreeDiskLocation/normal_disk_space_and_available_volume_count_allows_allocation === RUN TestHasFreeDiskLocation/volume_count_at_max_prevents_allocation === RUN TestHasFreeDiskLocation/volume_count_over_max_prevents_allocation === RUN TestHasFreeDiskLocation/volume_count_just_under_max_allows_allocation === RUN TestHasFreeDiskLocation/max_volume_count_is_0_allows_allocation === RUN TestHasFreeDiskLocation/max_volume_count_is_0_but_low_disk_space_prevents_allocation --- PASS: TestHasFreeDiskLocation (0.00s) --- PASS: TestHasFreeDiskLocation/low_disk_space_prevents_allocation (0.00s) --- PASS: TestHasFreeDiskLocation/normal_disk_space_and_available_volume_count_allows_allocation (0.00s) --- PASS: TestHasFreeDiskLocation/volume_count_at_max_prevents_allocation (0.00s) --- PASS: TestHasFreeDiskLocation/volume_count_over_max_prevents_allocation (0.00s) --- PASS: TestHasFreeDiskLocation/volume_count_just_under_max_allows_allocation (0.00s) --- PASS: TestHasFreeDiskLocation/max_volume_count_is_0_allows_allocation (0.00s) --- PASS: TestHasFreeDiskLocation/max_volume_count_is_0_but_low_disk_space_prevents_allocation (0.00s) === RUN TestCollectHeartbeatRespectsLowDiskSpace === RUN TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space === RUN TestCollectHeartbeatRespectsLowDiskSpace/normal_disk_space === RUN TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space_zero_volumes === RUN TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space_with_ec_shards === RUN TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space_with_multiple_ec_volumes --- PASS: TestCollectHeartbeatRespectsLowDiskSpace (0.00s) --- PASS: TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space (0.00s) --- PASS: TestCollectHeartbeatRespectsLowDiskSpace/normal_disk_space (0.00s) --- PASS: TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space_zero_volumes (0.00s) --- PASS: TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space_with_ec_shards (0.00s) --- PASS: TestCollectHeartbeatRespectsLowDiskSpace/low_disk_space_with_multiple_ec_volumes (0.00s) === RUN TestUnmountVolumeRemovesAllDuplicateCopies I0720 21:30:03.841183 store.go:807 UnmountVolume 4242 disk_id:0 I0720 21:30:03.841193 store.go:807 UnmountVolume 4242 disk_id:0 --- PASS: TestUnmountVolumeRemovesAllDuplicateCopies (0.00s) === RUN TestDeleteVolumeRemovesAllDuplicateCopies I0720 21:30:03.841755 volume_write.go:132 removed volume file /tmp/TestDeleteVolumeRemovesAllDuplicateCopies3102491659/001/dir0/4243.dat (size=8) I0720 21:30:03.841766 volume_write.go:132 removed volume file /tmp/TestDeleteVolumeRemovesAllDuplicateCopies3102491659/001/dir0/4243.idx (size=0) I0720 21:30:03.841898 store.go:835 DeleteVolume 4243 disk_id:0 I0720 21:30:03.841920 volume_write.go:132 removed volume file /tmp/TestDeleteVolumeRemovesAllDuplicateCopies3102491659/001/dir1/4243.dat (size=8) I0720 21:30:03.841929 volume_write.go:132 removed volume file /tmp/TestDeleteVolumeRemovesAllDuplicateCopies3102491659/001/dir1/4243.idx (size=0) I0720 21:30:03.842061 store.go:835 DeleteVolume 4243 disk_id:0 --- PASS: TestDeleteVolumeRemovesAllDuplicateCopies (0.00s) === RUN TestEc9plus3MultiDiskRebootLoadsAllShards I0720 21:30:03.843507 store_ec_mirror.go:56 ec volume 7 (collection="loki"): mirrored 3 sidecar(s) from /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk0 to /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk1 for same-disk invariant I0720 21:30:03.843658 store_ec_mirror.go:56 ec volume 7 (collection="loki"): mirrored 3 sidecar(s) from /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk0 to /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk2 for same-disk invariant I0720 21:30:03.843838 store_ec_mirror.go:56 ec volume 7 (collection="loki"): mirrored 3 sidecar(s) from /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk0 to /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk3 for same-disk invariant I0720 21:30:03.844019 store_ec_mirror.go:56 ec volume 7 (collection="loki"): mirrored 3 sidecar(s) from /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk0 to /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk4 for same-disk invariant I0720 21:30:03.844179 store_ec_mirror.go:56 ec volume 7 (collection="loki"): mirrored 3 sidecar(s) from /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk0 to /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk5 for same-disk invariant I0720 21:30:03.844295 store_ec_reconcile.go:85 ec volume 7 (collection="loki"): loading orphan shards [loki_7.ec02 loki_7.ec03] on /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk1 against locally-mirrored sidecars I0720 21:30:03.844418 store_ec_reconcile.go:85 ec volume 7 (collection="loki"): loading orphan shards [loki_7.ec04 loki_7.ec05] on /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk2 against locally-mirrored sidecars I0720 21:30:03.844515 store_ec_reconcile.go:85 ec volume 7 (collection="loki"): loading orphan shards [loki_7.ec06 loki_7.ec07] on /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk3 against locally-mirrored sidecars I0720 21:30:03.844598 store_ec_reconcile.go:85 ec volume 7 (collection="loki"): loading orphan shards [loki_7.ec08 loki_7.ec09] on /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk4 against locally-mirrored sidecars I0720 21:30:03.844685 store_ec_reconcile.go:85 ec volume 7 (collection="loki"): loading orphan shards [loki_7.ec10 loki_7.ec11] on /tmp/TestEc9plus3MultiDiskRebootLoadsAllShards3478222958/001/disk5 against locally-mirrored sidecars --- PASS: TestEc9plus3MultiDiskRebootLoadsAllShards (0.00s) === RUN TestEc9plus3ValidateKeepsVolumeWithDat --- PASS: TestEc9plus3ValidateKeepsVolumeWithDat (0.04s) === RUN TestEc9plus3PruneKeepsFullDataShardSet --- PASS: TestEc9plus3PruneKeepsFullDataShardSet (0.04s) === RUN TestMountEcShards_AppliesSourceDiskType I0720 21:30:03.927424 store_ec.go:203 MountEcShards 4242.0 on disk ID 0 --- PASS: TestMountEcShards_AppliesSourceDiskType (0.02s) === RUN TestMountEcShards_FallsBackToLocationDiskTypeWhenEmpty I0720 21:30:03.941586 store_ec.go:203 MountEcShards 4242.0 on disk ID 0 --- PASS: TestMountEcShards_FallsBackToLocationDiskTypeWhenEmpty (0.01s) === RUN TestIssue9478_PartialEcOnSiblingDiskOfHealthyDat store_ec_hybrid_repro_test.go:143: after per-disk EC load on sdf: inMemoryShards=1 newEcShardMessages=1 W0720 21:30:03.952840 store_ec_reconcile.go:314 ec volume 122 (collection="") on /tmp/TestIssue9478_PartialEcOnSiblingDiskOfHealthyDat518372750/001/sdf has only 1 shards (need 10) while a byte-exact source .dat exists on sibling disk /tmp/TestIssue9478_PartialEcOnSiblingDiskOfHealthyDat518372750/001/sdd; cleaning up leftover EC files store_ec_hybrid_repro_test.go:157: after Store-level cleanup: shardFileLeft=false ecxLeft=false ecjLeft=false inMemoryShards=0 deletedEcShardMessages=1 --- PASS: TestIssue9478_PartialEcOnSiblingDiskOfHealthyDat (0.00s) === RUN TestIssue9478_ZeroByteSiblingDatKeepsPartialEc W0720 21:30:03.953353 volume_info.go:46 empty volume info file /tmp/TestIssue9478_ZeroByteSiblingDatKeepsPartialEc414204466/001/sdf/122.vif, treating as non-existent W0720 21:30:03.953363 ec_volume.go:182 vif file not found, using defaults, volumeId:122, filename:/tmp/TestIssue9478_ZeroByteSiblingDatKeepsPartialEc414204466/001/sdf/122.vif W0720 21:30:03.953400 store_ec_reconcile.go:290 ec volume 122 (collection="") on /tmp/TestIssue9478_ZeroByteSiblingDatKeepsPartialEc414204466/001/sdf has only 1 shards; sibling .dat on /tmp/TestIssue9478_ZeroByteSiblingDatKeepsPartialEc414204466/001/sdd is 0 bytes but .vif recorded 0 (need byte-exact match); leaving partial EC in place --- PASS: TestIssue9478_ZeroByteSiblingDatKeepsPartialEc (0.00s) === RUN TestMirrorEcMetadataOnStartup_PhysicallyCopiesSidecars I0720 21:30:03.957826 store_ec_mirror.go:56 ec volume 4121 (collection="video-recordings"): mirrored 3 sidecar(s) from /tmp/TestMirrorEcMetadataOnStartup_PhysicallyCopiesSidecars1903480938/001/data2 to /tmp/TestMirrorEcMetadataOnStartup_PhysicallyCopiesSidecars1903480938/001/data1 for same-disk invariant I0720 21:30:03.957907 store_ec_reconcile.go:85 ec volume 4121 (collection="video-recordings"): loading orphan shards [video-recordings_4121.ec00 video-recordings_4121.ec12] on /tmp/TestMirrorEcMetadataOnStartup_PhysicallyCopiesSidecars1903480938/001/data1 against locally-mirrored sidecars --- PASS: TestMirrorEcMetadataOnStartup_PhysicallyCopiesSidecars (0.00s) === RUN TestMirrorEcMetadataOnStartup_NoOpWhenAlreadyMirrored --- PASS: TestMirrorEcMetadataOnStartup_NoOpWhenAlreadyMirrored (0.00s) === RUN TestMountEcShards_LocatesEcxOnSiblingDisk I0720 21:30:03.959689 store_ec.go:203 MountEcShards 5.6 on disk ID 0 --- PASS: TestMountEcShards_LocatesEcxOnSiblingDisk (0.00s) === RUN TestMountEcShards_SameDiskEcxStillWorks I0720 21:30:03.960486 store_ec.go:203 MountEcShards 7.3 on disk ID 0 --- PASS: TestMountEcShards_SameDiskEcxStillWorks (0.00s) === RUN TestMountEcShards_MissingEcxOnAllDisks --- PASS: TestMountEcShards_MissingEcxOnAllDisks (0.00s) === RUN TestMountEcShards_ZeroByteEcxIsIgnored I0720 21:30:03.961398 store_ec.go:203 MountEcShards 12.7 on disk ID 0 --- PASS: TestMountEcShards_ZeroByteEcxIsIgnored (0.00s) === RUN TestMountEcShards_EmptyEcxMountsSuccessfully I0720 21:30:03.961877 store_ec.go:203 MountEcShards 13.0 on disk ID 0 --- PASS: TestMountEcShards_EmptyEcxMountsSuccessfully (0.00s) === RUN TestIndexEcxOwners_IgnoresZeroByteStub --- PASS: TestIndexEcxOwners_IgnoresZeroByteStub (0.00s) === RUN TestLoadEcShardsWhenIndexFilesOnDifferentDisk I0720 21:30:03.962978 store_ec_mirror.go:56 ec volume 1093 (collection="grafana-loki"): mirrored 3 sidecar(s) from /tmp/TestLoadEcShardsWhenIndexFilesOnDifferentDisk1482072585/001/data2 to /tmp/TestLoadEcShardsWhenIndexFilesOnDifferentDisk1482072585/001/data1 for same-disk invariant I0720 21:30:03.963038 store_ec_reconcile.go:85 ec volume 1093 (collection="grafana-loki"): loading orphan shards [grafana-loki_1093.ec00 grafana-loki_1093.ec12] on /tmp/TestLoadEcShardsWhenIndexFilesOnDifferentDisk1482072585/001/data1 against locally-mirrored sidecars --- PASS: TestLoadEcShardsWhenIndexFilesOnDifferentDisk (0.00s) === RUN TestLoadEcShardsOrphanWithoutSiblingEcx --- PASS: TestLoadEcShardsOrphanWithoutSiblingEcx (0.00s) === RUN TestReconcileNoOpWhenEachDiskIsSelfContained --- PASS: TestReconcileNoOpWhenEachDiskIsSelfContained (0.00s) === RUN TestLoadEcShardsWhenOwnerEcxIsInDataDir I0720 21:30:03.965482 store_ec_mirror.go:56 ec volume 4242 (collection="grafana-loki"): mirrored 3 sidecar(s) from /tmp/TestLoadEcShardsWhenOwnerEcxIsInDataDir3178700998/001/data2 to /tmp/TestLoadEcShardsWhenOwnerEcxIsInDataDir3178700998/001/data1 for same-disk invariant I0720 21:30:03.965575 store_ec_reconcile.go:85 ec volume 4242 (collection="grafana-loki"): loading orphan shards [grafana-loki_4242.ec00 grafana-loki_4242.ec12] on /tmp/TestLoadEcShardsWhenOwnerEcxIsInDataDir3178700998/001/data1 against locally-mirrored sidecars --- PASS: TestLoadEcShardsWhenOwnerEcxIsInDataDir (0.00s) === RUN TestLoneVifDoesNotCreatePhantomDat --- PASS: TestLoneVifDoesNotCreatePhantomDat (0.00s) === RUN TestEmptyEcDatStubIsSwept W0720 21:30:03.966847 disk_location.go:193 removing leftover empty .dat stub for EC volume 41 (collection="warp-cal") W0720 21:30:03.966917 disk_location.go:193 removing leftover empty .dat stub for EC volume 41 (collection="warp-cal") --- PASS: TestEmptyEcDatStubIsSwept (0.00s) === RUN TestRemoveEmptyEcDatStubFindsVifInIdxDir W0720 21:30:03.967183 disk_location.go:193 removing leftover empty .dat stub for EC volume 42 (collection="warp-cal") --- PASS: TestRemoveEmptyEcDatStubFindsVifInIdxDir (0.00s) === RUN TestEmptyDatStubNextToEcxDoesNotDeleteShards W0720 21:30:03.967671 disk_location.go:193 removing leftover empty .dat stub for EC volume 87 (collection="warp-rec") --- PASS: TestEmptyDatStubNextToEcxDoesNotDeleteShards (0.00s) === RUN TestEmptyDatWithoutVifDoesNotDeleteShards W0720 21:30:03.968250 ec_volume.go:182 vif file not found, using defaults, volumeId:88, filename:/tmp/TestEmptyDatWithoutVifDoesNotDeleteShards1075501173/001/warp-rec_88.vif --- PASS: TestEmptyDatWithoutVifDoesNotDeleteShards (0.00s) === RUN TestEcIndexRecovery_CrossServerOrphan I0720 21:30:03.969035 store_ec_mirror.go:56 ec volume 6190 (collection="video-recordings"): mirrored 3 sidecar(s) from /tmp/TestEcIndexRecovery_CrossServerOrphan243059781/001/disk0 to /tmp/TestEcIndexRecovery_CrossServerOrphan243059781/001/disk1 for same-disk invariant --- PASS: TestEcIndexRecovery_CrossServerOrphan (0.00s) === RUN TestCollectEcVolumesMissingIndex_ExcludesCrossDiskOrphan --- PASS: TestCollectEcVolumesMissingIndex_ExcludesCrossDiskOrphan (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_SufficientShards store_ec_recovery_test.go:52: Successfully identified 13 available shards (need 10) --- PASS: TestRecoverOneRemoteEcShardInterval_SufficientShards (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_InsufficientShards store_ec_recovery_test.go:80: Correctly identified insufficient shards: 7 available (need 10) --- PASS: TestRecoverOneRemoteEcShardInterval_InsufficientShards (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_ShardCounting === RUN TestRecoverOneRemoteEcShardInterval_ShardCounting/All_shards_available_except_one store_ec_recovery_test.go:155: Available shards: 13 [0 1 2 3 4 6 7 8 9 10 11 12 13], Missing shards: 1 [5] === RUN TestRecoverOneRemoteEcShardInterval_ShardCounting/Exactly_minimum_shards_(DataShardsCount) store_ec_recovery_test.go:155: Available shards: 10 [0 1 2 3 4 5 6 7 8 9], Missing shards: 4 [10 11 12 13] === RUN TestRecoverOneRemoteEcShardInterval_ShardCounting/One_less_than_minimum store_ec_recovery_test.go:155: Available shards: 9 [0 1 2 3 4 5 6 7 8], Missing shards: 5 [9 10 11 12 13] === RUN TestRecoverOneRemoteEcShardInterval_ShardCounting/Only_half_the_shards store_ec_recovery_test.go:155: Available shards: 7 [1 2 3 4 5 6 7], Missing shards: 7 [0 8 9 10 11 12 13] === RUN TestRecoverOneRemoteEcShardInterval_ShardCounting/All_data_shards_available store_ec_recovery_test.go:155: Available shards: 10 [0 1 2 3 4 5 6 7 8 9], Missing shards: 4 [10 11 12 13] --- PASS: TestRecoverOneRemoteEcShardInterval_ShardCounting (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ShardCounting/All_shards_available_except_one (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ShardCounting/Exactly_minimum_shards_(DataShardsCount) (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ShardCounting/One_less_than_minimum (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ShardCounting/Only_half_the_shards (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ShardCounting/All_data_shards_available (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_ErrorMessage store_ec_recovery_test.go:190: Error message format validated: cannot recover shard 42.7: only 7 shards available [0 1 2 3 4 5 6], need at least 10 (missing: [7 8 9 10 11 12 13]) --- PASS: TestRecoverOneRemoteEcShardInterval_ErrorMessage (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_ReconstructDataSlicing store_ec_recovery_test.go:248: Successfully reconstructed shard with proper buffer slicing --- PASS: TestRecoverOneRemoteEcShardInterval_ReconstructDataSlicing (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_ParityShardRecovery === RUN TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity10 === RUN TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity11 === RUN TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity12 === RUN TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity13 --- PASS: TestRecoverOneRemoteEcShardInterval_ParityShardRecovery (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity10 (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity11 (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity12 (0.00s) --- PASS: TestRecoverOneRemoteEcShardInterval_ParityShardRecovery/RecoverParity13 (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_ConcurrentShardReading store_ec_recovery_test.go:363: Successfully simulated concurrent reading of 13 shards --- PASS: TestRecoverOneRemoteEcShardInterval_ConcurrentShardReading (0.00s) === RUN TestRecoverOneRemoteEcShardInterval_BuggyMaxShardCount store_ec_recovery_test.go:419: Corrected code: 9 shards (correct, insufficient) store_ec_recovery_test.go:420: Buggy code would have counted: 27 shards (incorrect, falsely sufficient) store_ec_recovery_test.go:421: Missing shards: [9 10 11 12 13] --- PASS: TestRecoverOneRemoteEcShardInterval_BuggyMaxShardCount (0.00s) === RUN TestFindEcShardTargetLocation_PinsToEcxOnDisk --- PASS: TestFindEcShardTargetLocation_PinsToEcxOnDisk (0.00s) === RUN TestFindEcShardTargetLocation_PrefersMountedOverEcx --- PASS: TestFindEcShardTargetLocation_PrefersMountedOverEcx (0.00s) === RUN TestFindEcShardTargetLocation_FallsThroughToHddWhenNothingMatches --- PASS: TestFindEcShardTargetLocation_FallsThroughToHddWhenNothingMatches (0.00s) === RUN TestFindEcShardTargetLocation_HonoursUnlimitedDisk --- PASS: TestFindEcShardTargetLocation_HonoursUnlimitedDisk (0.00s) === RUN TestFindEcShardTargetLocation_TightProvisioningKeepsEcxDisk --- PASS: TestFindEcShardTargetLocation_TightProvisioningKeepsEcxDisk (0.00s) === RUN TestLoadBalancingDistribution I0720 21:30:03.974821 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir0 (disk ID 0) adds volume:1 collection: replicaPlacement:000 ttl: I0720 21:30:03.974939 store.go:307 add volume 1 on disk ID 0 I0720 21:30:03.974945 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir1 (disk ID 1) adds volume:2 collection: replicaPlacement:000 ttl: I0720 21:30:03.975009 store.go:307 add volume 2 on disk ID 1 I0720 21:30:03.975012 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir2 (disk ID 2) adds volume:3 collection: replicaPlacement:000 ttl: I0720 21:30:03.975073 store.go:307 add volume 3 on disk ID 2 I0720 21:30:03.975076 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir0 (disk ID 0) adds volume:4 collection: replicaPlacement:000 ttl: I0720 21:30:03.975135 store.go:307 add volume 4 on disk ID 0 I0720 21:30:03.975142 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir1 (disk ID 1) adds volume:5 collection: replicaPlacement:000 ttl: I0720 21:30:03.975207 store.go:307 add volume 5 on disk ID 1 I0720 21:30:03.975210 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir2 (disk ID 2) adds volume:6 collection: replicaPlacement:000 ttl: I0720 21:30:03.975273 store.go:307 add volume 6 on disk ID 2 I0720 21:30:03.975276 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir0 (disk ID 0) adds volume:7 collection: replicaPlacement:000 ttl: I0720 21:30:03.975332 store.go:307 add volume 7 on disk ID 0 I0720 21:30:03.975336 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir1 (disk ID 1) adds volume:8 collection: replicaPlacement:000 ttl: I0720 21:30:03.975402 store.go:307 add volume 8 on disk ID 1 I0720 21:30:03.975405 store.go:302 In dir /tmp/TestLoadBalancingDistribution756688181/001/dir2 (disk ID 2) adds volume:9 collection: replicaPlacement:000 ttl: I0720 21:30:03.975464 store.go:307 add volume 9 on disk ID 2 --- PASS: TestLoadBalancingDistribution (0.00s) === RUN TestLocalVolumesLen === RUN TestLocalVolumesLen/all_local_volumes === RUN TestLocalVolumesLen/all_remote_volumes === RUN TestLocalVolumesLen/mixed_local_and_remote === RUN TestLocalVolumesLen/no_volumes --- PASS: TestLocalVolumesLen (0.00s) --- PASS: TestLocalVolumesLen/all_local_volumes (0.00s) --- PASS: TestLocalVolumesLen/all_remote_volumes (0.00s) --- PASS: TestLocalVolumesLen/mixed_local_and_remote (0.00s) --- PASS: TestLocalVolumesLen/no_volumes (0.00s) === RUN TestVolumeLoadBalancing === RUN TestVolumeLoadBalancing/even_distribution_across_empty_locations I0720 21:30:03.976411 store.go:302 In dir /tmp/TestVolumeLoadBalancingeven_distribution_across_empty_locations377904839/001/dir0 (disk ID 0) adds volume:1 collection: replicaPlacement:000 ttl: I0720 21:30:03.976503 store.go:307 add volume 1 on disk ID 0 I0720 21:30:03.976507 store.go:302 In dir /tmp/TestVolumeLoadBalancingeven_distribution_across_empty_locations377904839/001/dir1 (disk ID 1) adds volume:2 collection: replicaPlacement:000 ttl: I0720 21:30:03.976590 store.go:307 add volume 2 on disk ID 1 I0720 21:30:03.976593 store.go:302 In dir /tmp/TestVolumeLoadBalancingeven_distribution_across_empty_locations377904839/001/dir2 (disk ID 2) adds volume:3 collection: replicaPlacement:000 ttl: I0720 21:30:03.976681 store.go:307 add volume 3 on disk ID 2 I0720 21:30:03.976685 store.go:302 In dir /tmp/TestVolumeLoadBalancingeven_distribution_across_empty_locations377904839/001/dir0 (disk ID 0) adds volume:4 collection: replicaPlacement:000 ttl: I0720 21:30:03.976749 store.go:307 add volume 4 on disk ID 0 I0720 21:30:03.976754 store.go:302 In dir /tmp/TestVolumeLoadBalancingeven_distribution_across_empty_locations377904839/001/dir1 (disk ID 1) adds volume:5 collection: replicaPlacement:000 ttl: I0720 21:30:03.976826 store.go:307 add volume 5 on disk ID 1 I0720 21:30:03.976831 store.go:302 In dir /tmp/TestVolumeLoadBalancingeven_distribution_across_empty_locations377904839/001/dir2 (disk ID 2) adds volume:6 collection: replicaPlacement:000 ttl: I0720 21:30:03.976902 store.go:307 add volume 6 on disk ID 2 === RUN TestVolumeLoadBalancing/prefers_location_with_fewer_local_volumes I0720 21:30:03.977538 store.go:302 In dir /tmp/TestVolumeLoadBalancingprefers_location_with_fewer_local_volume2101160802/001/dir1 (disk ID 1) adds volume:1 collection: replicaPlacement:000 ttl: I0720 21:30:03.977615 store.go:307 add volume 1 on disk ID 1 I0720 21:30:03.977620 store.go:302 In dir /tmp/TestVolumeLoadBalancingprefers_location_with_fewer_local_volume2101160802/001/dir1 (disk ID 1) adds volume:2 collection: replicaPlacement:000 ttl: I0720 21:30:03.977679 store.go:307 add volume 2 on disk ID 1 I0720 21:30:03.977683 store.go:302 In dir /tmp/TestVolumeLoadBalancingprefers_location_with_fewer_local_volume2101160802/001/dir1 (disk ID 1) adds volume:3 collection: replicaPlacement:000 ttl: I0720 21:30:03.977749 store.go:307 add volume 3 on disk ID 1 === RUN TestVolumeLoadBalancing/ignores_remote_volumes_in_count I0720 21:30:03.978217 store.go:302 In dir /tmp/TestVolumeLoadBalancingignores_remote_volumes_in_count752628811/001/dir0 (disk ID 0) adds volume:1 collection: replicaPlacement:000 ttl: I0720 21:30:03.978283 store.go:307 add volume 1 on disk ID 0 I0720 21:30:03.978287 store.go:302 In dir /tmp/TestVolumeLoadBalancingignores_remote_volumes_in_count752628811/001/dir0 (disk ID 0) adds volume:2 collection: replicaPlacement:000 ttl: I0720 21:30:03.978364 store.go:307 add volume 2 on disk ID 0 I0720 21:30:03.978368 store.go:302 In dir /tmp/TestVolumeLoadBalancingignores_remote_volumes_in_count752628811/001/dir2 (disk ID 2) adds volume:3 collection: replicaPlacement:000 ttl: I0720 21:30:03.978425 store.go:307 add volume 3 on disk ID 2 === RUN TestVolumeLoadBalancing/balances_when_some_locations_have_remote_volumes I0720 21:30:03.978903 store.go:302 In dir /tmp/TestVolumeLoadBalancingbalances_when_some_locations_have_remote2554937062/001/dir2 (disk ID 2) adds volume:1 collection: replicaPlacement:000 ttl: I0720 21:30:03.978971 store.go:307 add volume 1 on disk ID 2 I0720 21:30:03.978977 store.go:302 In dir /tmp/TestVolumeLoadBalancingbalances_when_some_locations_have_remote2554937062/001/dir0 (disk ID 0) adds volume:2 collection: replicaPlacement:000 ttl: I0720 21:30:03.979032 store.go:307 add volume 2 on disk ID 0 I0720 21:30:03.979036 store.go:302 In dir /tmp/TestVolumeLoadBalancingbalances_when_some_locations_have_remote2554937062/001/dir1 (disk ID 1) adds volume:3 collection: replicaPlacement:000 ttl: I0720 21:30:03.979089 store.go:307 add volume 3 on disk ID 1 --- PASS: TestVolumeLoadBalancing (0.00s) --- PASS: TestVolumeLoadBalancing/even_distribution_across_empty_locations (0.00s) --- PASS: TestVolumeLoadBalancing/prefers_location_with_fewer_local_volumes (0.00s) --- PASS: TestVolumeLoadBalancing/ignores_remote_volumes_in_count (0.00s) --- PASS: TestVolumeLoadBalancing/balances_when_some_locations_have_remote_volumes (0.00s) === RUN TestMaybeAdjustVolumeMaxNoDeadZone === RUN TestMaybeAdjustVolumeMaxNoDeadZone/ratio=0.5 store_maxvolume_deadzone_test.go:49: free=28.2GiB limit=56.3GiB ratio≈0.50 -> MaxVolumeCount=1 (want 1) === RUN TestMaybeAdjustVolumeMaxNoDeadZone/ratio=1.5 store_maxvolume_deadzone_test.go:49: free=28.2GiB limit=18.8GiB ratio≈1.50 -> MaxVolumeCount=1 (want 1) === RUN TestMaybeAdjustVolumeMaxNoDeadZone/ratio=2.5 store_maxvolume_deadzone_test.go:49: free=28.2GiB limit=11.3GiB ratio≈2.50 -> MaxVolumeCount=2 (want 2) === RUN TestMaybeAdjustVolumeMaxNoDeadZone/ratio=3.5 store_maxvolume_deadzone_test.go:49: free=28.2GiB limit=8.0GiB ratio≈3.50 -> MaxVolumeCount=3 (want 3) --- PASS: TestMaybeAdjustVolumeMaxNoDeadZone (0.00s) --- PASS: TestMaybeAdjustVolumeMaxNoDeadZone/ratio=0.5 (0.00s) --- PASS: TestMaybeAdjustVolumeMaxNoDeadZone/ratio=1.5 (0.00s) --- PASS: TestMaybeAdjustVolumeMaxNoDeadZone/ratio=2.5 (0.00s) --- PASS: TestMaybeAdjustVolumeMaxNoDeadZone/ratio=3.5 (0.00s) === RUN TestCollectStatForOneVolumeModifiedAtSecond --- PASS: TestCollectStatForOneVolumeModifiedAtSecond (0.00s) === RUN TestSpaceCalculation === RUN TestSpaceCalculation/Large_volume,_small_preallocate store_vacuum_test.go:47: Volume size: 261993005056 bytes, Space needed: 288193458995 bytes (110.00% of volume size) === RUN TestSpaceCalculation/Small_volume,_large_preallocate store_vacuum_test.go:47: Volume size: 104857600 bytes, Space needed: 1181116006 bytes (1126.40% of volume size) --- PASS: TestSpaceCalculation (0.00s) --- PASS: TestSpaceCalculation/Large_volume,_small_preallocate (0.00s) --- PASS: TestSpaceCalculation/Small_volume,_large_preallocate (0.00s) === RUN TestBinarySearch --- PASS: TestBinarySearch (0.00s) === RUN TestScrubVolumeData === RUN TestScrubVolumeData/healthy_volume === RUN TestScrubVolumeData/zero-size_volume_without_index === RUN TestScrubVolumeData/zero-size_volume_with_index E0720 21:30:03.980421 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 75688 offset 8 fileSize 8: EOF E0720 21:30:03.980429 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 4448 offset 75696 fileSize 8: EOF E0720 21:30:03.980451 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 83640 offset 80144 fileSize 8: EOF E0720 21:30:03.980483 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 65472 offset 163784 fileSize 8: EOF E0720 21:30:03.980528 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 79928 offset 229256 fileSize 8: EOF E0720 21:30:03.980730 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 86696 offset 309184 fileSize 8: EOF E0720 21:30:03.980751 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 32984 offset 395880 fileSize 8: EOF E0720 21:30:03.980756 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 74648 offset 428864 fileSize 8: EOF E0720 21:30:03.980782 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 60400 offset 503512 fileSize 8: EOF E0720 21:30:03.980786 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 12328 offset 563912 fileSize 8: EOF E0720 21:30:03.980796 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 81992 offset 576240 fileSize 8: EOF E0720 21:30:03.980804 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 46544 offset 658232 fileSize 8: EOF E0720 21:30:03.980808 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 88080 offset 704776 fileSize 8: EOF E0720 21:30:03.980813 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 5512 offset 792856 fileSize 8: EOF E0720 21:30:03.980817 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 81648 offset 798368 fileSize 8: EOF E0720 21:30:03.980833 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 33200 offset 880016 fileSize 8: EOF E0720 21:30:03.980867 needle_read.go:52 ./test_files/empty_volume.dat read 0 dataSize 29320 offset 913216 fileSize 8: EOF === RUN TestScrubVolumeData/volume_without_index === RUN TestScrubVolumeData/bitrot_volume --- PASS: TestScrubVolumeData (0.00s) --- PASS: TestScrubVolumeData/healthy_volume (0.00s) --- PASS: TestScrubVolumeData/zero-size_volume_without_index (0.00s) --- PASS: TestScrubVolumeData/zero-size_volume_with_index (0.00s) --- PASS: TestScrubVolumeData/volume_without_index (0.00s) --- PASS: TestScrubVolumeData/bitrot_volume (0.00s) === RUN TestScrubVolumeData_IgnoresOffset0Tombstone --- PASS: TestScrubVolumeData_IgnoresOffset0Tombstone (0.00s) === RUN TestCheckVolumeDataIntegrityWithDeletionTombstone --- PASS: TestCheckVolumeDataIntegrityWithDeletionTombstone (0.00s) === RUN TestCheckVolumeDataIntegritySortedIndex --- PASS: TestCheckVolumeDataIntegritySortedIndex (0.00s) === RUN TestCheckVolumeDataIntegrityVerifiesDeletionTail W0720 21:30:03.982690 volume_checking.go:357 data file /tmp/TestCheckVolumeDataIntegrityVerifiesDeletionTail3701647305/001/1.dat actual 1408 bytes expected 1344 bytes! --- PASS: TestCheckVolumeDataIntegrityVerifiesDeletionTail (0.00s) === RUN TestVolumeLoadStaysWritableWithKeySortedIndex --- PASS: TestVolumeLoadStaysWritableWithKeySortedIndex (0.00s) === RUN TestMaxNeedleEnd --- PASS: TestMaxNeedleEnd (0.00s) === RUN TestDestroyKeepsVifWhenEcCoexists I0720 21:30:03.983630 volume_write.go:132 removed volume file /tmp/TestDestroyKeepsVifWhenEcCoexists1572583532/001/1.dat (size=336) I0720 21:30:03.983640 volume_write.go:132 removed volume file /tmp/TestDestroyKeepsVifWhenEcCoexists1572583532/001/1.idx (size=16) --- PASS: TestDestroyKeepsVifWhenEcCoexists (0.00s) === RUN TestDestroyRemovesVifWhenNoEc I0720 21:30:03.983990 volume_write.go:132 removed volume file /tmp/TestDestroyRemovesVifWhenNoEc3140237076/001/1.dat (size=1032) I0720 21:30:03.984029 volume_write.go:132 removed volume file /tmp/TestDestroyRemovesVifWhenNoEc3140237076/001/1.idx (size=16) --- PASS: TestDestroyRemovesVifWhenNoEc (0.00s) === RUN TestSortVolumeInfos --- PASS: TestSortVolumeInfos (0.00s) === RUN TestCheckReadWriteErrorTracksConsecutiveEIO --- PASS: TestCheckReadWriteErrorTracksConsecutiveEIO (0.00s) === RUN TestCheckReadWriteErrorNonEIOResetsStreak --- PASS: TestCheckReadWriteErrorNonEIOResetsStreak (0.00s) === RUN TestCheckReadWriteErrorIgnoresPlainError --- PASS: TestCheckReadWriteErrorIgnoresPlainError (0.00s) === RUN TestIoErrorToleranceGate --- PASS: TestIoErrorToleranceGate (0.00s) === RUN TestQuarantineIsSticky --- PASS: TestQuarantineIsSticky (0.00s) === RUN TestReadNeedleBlobTracksEIO E0720 21:30:03.984355 needle_read.go:52 eio read 0 dataSize 32 offset 0 fileSize 0: input/output error --- PASS: TestReadNeedleBlobTracksEIO (0.00s) === RUN TestIoErrorStateIsRaceFree --- PASS: TestIoErrorStateIsRaceFree (0.00s) === RUN TestLoad_CorruptIdx_NoSegfault I0720 21:30:03.985894 volume_loading.go:257 open to read file /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.idx I0720 21:30:03.985921 volume_loading.go:276 volumeDataIntegrityChecking failed verifyIndexFileIntegrity /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.idx failed: index file's size is 15 bytes, maybe corrupted I0720 21:30:03.985932 needle_map_sorted_file.go:26 Start to Generate /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.sdx from /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.idx I0720 21:30:03.988067 needle_map_sorted_file.go:28 Finished Generating /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.sdx from /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.idx I0720 21:30:03.988097 volume_loading.go:290 loading sorted db /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.sdx error: unexpected file /tmp/TestLoad_CorruptIdx_NoSegfault1115590396/001/1.idx size: 15 --- PASS: TestLoad_CorruptIdx_NoSegfault (0.00s) === RUN TestMarkVolumeWritable_ReopensPersistedReadOnly I0720 21:30:03.988421 volume_loading.go:257 open to read file /tmp/TestMarkVolumeWritable_ReopensPersistedReadOnly3407899376/001/1.idx I0720 21:30:03.988444 needle_map_sorted_file.go:26 Start to Generate /tmp/TestMarkVolumeWritable_ReopensPersistedReadOnly3407899376/001/1.sdx from /tmp/TestMarkVolumeWritable_ReopensPersistedReadOnly3407899376/001/1.idx I0720 21:30:03.989233 needle_map_sorted_file.go:28 Finished Generating /tmp/TestMarkVolumeWritable_ReopensPersistedReadOnly3407899376/001/1.sdx from /tmp/TestMarkVolumeWritable_ReopensPersistedReadOnly3407899376/001/1.idx --- PASS: TestMarkVolumeWritable_ReopensPersistedReadOnly (0.00s) === RUN TestReopenIdxForWrite_NoopWhenAlreadyWritable --- PASS: TestReopenIdxForWrite_NoopWhenAlreadyWritable (0.00s) === RUN TestReadNeedleNilNeedleMap I0720 21:30:03.989832 volume_read.go:26 volume 1: needle map not loaded; read returns not-found I0720 21:30:03.989842 volume_read.go:120 volume 1: needle map not loaded; read returns not-found --- PASS: TestReadNeedleNilNeedleMap (0.00s) === RUN TestReadNeedMetaWithWritesAndUpdates --- PASS: TestReadNeedMetaWithWritesAndUpdates (0.00s) === RUN TestReadNeedMetaWithDeletesThenWrites --- PASS: TestReadNeedMetaWithDeletesThenWrites (0.00s) === RUN TestReconcileRollForwardMarkerOnly I0720 21:30:03.992014 volume_loading.go:319 loading leveldb index /tmp/TestReconcileRollForwardMarkerOnly2214285458/001/1.ldb I0720 21:30:03.995443 needle_map_leveldb.go:345 loading idx to leveldb from offset 0 for file: /tmp/TestReconcileRollForwardMarkerOnly2214285458/001/1.cpx I0720 21:30:03.996243 volume_vacuum.go:323 volume 1: rolling forward interrupted compaction commit I0720 21:30:03.996448 volume_loading.go:319 loading leveldb index /tmp/TestReconcileRollForwardMarkerOnly2214285458/001/1.ldb --- PASS: TestReconcileRollForwardMarkerOnly (0.01s) === RUN TestReconcileRollForwardPartialRename I0720 21:30:04.001122 volume_loading.go:319 loading leveldb index /tmp/TestReconcileRollForwardPartialRename1915885917/001/1.ldb I0720 21:30:04.005745 needle_map_leveldb.go:345 loading idx to leveldb from offset 0 for file: /tmp/TestReconcileRollForwardPartialRename1915885917/001/1.cpx I0720 21:30:04.006920 volume_vacuum.go:323 volume 1: rolling forward interrupted compaction commit I0720 21:30:04.007175 volume_loading.go:319 loading leveldb index /tmp/TestReconcileRollForwardPartialRename1915885917/001/1.ldb --- PASS: TestReconcileRollForwardPartialRename (0.02s) === RUN TestReconcileRollBackNoMarker I0720 21:30:04.027159 needle_map_memory.go:113 loading idx from offset 0 for file: /tmp/TestReconcileRollBackNoMarker3349890256/001/1.cpx I0720 21:30:04.027384 volume_vacuum.go:332 volume 1: rolling back orphan compaction file .cpd I0720 21:30:04.027409 volume_vacuum.go:332 volume 1: rolling back orphan compaction file .cpx --- PASS: TestReconcileRollBackNoMarker (0.00s) === RUN TestReconcileSkipsLoadedVolumeMidVacuum I0720 21:30:04.031301 needle_map_memory.go:113 loading idx from offset 0 for file: /tmp/TestReconcileSkipsLoadedVolumeMidVacuum2007773522/001/1.cpx --- PASS: TestReconcileSkipsLoadedVolumeMidVacuum (0.00s) === RUN TestApplyCompactSwapMissingTempFilesPreservesLive --- PASS: TestApplyCompactSwapMissingTempFilesPreservesLive (0.00s) === RUN TestDestroyRemovesCommitMarker I0720 21:30:04.031933 volume_write.go:132 removed volume file /tmp/TestDestroyRemovesCommitMarker3203273607/001/1.dat (size=1456) I0720 21:30:04.031943 volume_write.go:132 removed volume file /tmp/TestDestroyRemovesCommitMarker3203273607/001/1.idx (size=48) --- PASS: TestDestroyRemovesCommitMarker (0.00s) === RUN TestMakeDiff --- PASS: TestMakeDiff (0.00s) === RUN TestMemIndexCompaction I0720 21:30:04.158178 needle_map_memory.go:113 loading idx from offset 0 for file: /tmp/TestMemIndexCompaction719238856/001/1.cpx volume_vacuum_test.go:97: compaction speed: 71227655.25 bytes/s I0720 21:30:04.297664 volume_vacuum.go:166 Committing volume 1 vacuuming... I0720 21:30:04.420429 needle_map_memory.go:113 loading idx from offset 9715 for file: /tmp/TestMemIndexCompaction719238856/001/1.cpx volume_vacuum_test.go:115: realRecordCount:29715, v.FileCount():29715 mm.DeletedCount():9837 --- PASS: TestMemIndexCompaction (0.44s) === RUN TestLDBIndexCompaction I0720 21:30:04.474344 volume_loading.go:319 loading leveldb index /tmp/TestLDBIndexCompaction255107421/001/1.ldb I0720 21:30:04.657525 needle_map_leveldb.go:345 loading idx to leveldb from offset 0 for file: /tmp/TestLDBIndexCompaction255107421/001/1.cpx volume_vacuum_test.go:97: compaction speed: 60682705.90 bytes/s I0720 21:30:04.976593 volume_vacuum.go:166 Committing volume 1 vacuuming... I0720 21:30:05.086837 needle_map_leveldb.go:345 loading idx to leveldb from offset 9705 for file: /tmp/TestLDBIndexCompaction255107421/001/1.cpx I0720 21:30:05.168875 volume_loading.go:316 updating leveldb index /tmp/TestLDBIndexCompaction255107421/001/1.ldb volume_vacuum_test.go:110: watermark from levelDB: 20000, realWatermark: 20000, nm.recordCount: 29705, realRecordCount:29705, fileCount=29705, deletedcount:9705 I0720 21:30:05.184469 volume_loading.go:319 loading leveldb index /tmp/TestLDBIndexCompaction255107421/001/1.ldb --- PASS: TestLDBIndexCompaction (0.78s) === RUN TestCommitCompactDeletionTailKeepsWritable I0720 21:30:05.258150 volume_loading.go:319 loading leveldb index /tmp/TestCommitCompactDeletionTailKeepsWritable3711146412/001/1.ldb I0720 21:30:05.262628 needle_map_leveldb.go:345 loading idx to leveldb from offset 0 for file: /tmp/TestCommitCompactDeletionTailKeepsWritable3711146412/001/1.cpx I0720 21:30:05.263951 volume_vacuum.go:166 Committing volume 1 vacuuming... I0720 21:30:05.264043 needle_map_leveldb.go:345 loading idx to leveldb from offset 5 for file: /tmp/TestCommitCompactDeletionTailKeepsWritable3711146412/001/1.cpx I0720 21:30:05.265834 volume_loading.go:316 updating leveldb index /tmp/TestCommitCompactDeletionTailKeepsWritable3711146412/001/1.ldb I0720 21:30:05.266810 volume_loading.go:319 loading leveldb index /tmp/TestCommitCompactDeletionTailKeepsWritable3711146412/001/1.ldb --- PASS: TestCommitCompactDeletionTailKeepsWritable (0.01s) === RUN TestCompactVolumeFilesOffline I0720 21:30:05.270394 needle_map_memory.go:113 loading idx from offset 0 for file: /tmp/TestCompactVolumeFilesOffline2588220902/001/1.cpx I0720 21:30:05.270446 volume_vacuum.go:166 Committing volume 1 vacuuming... --- PASS: TestCompactVolumeFilesOffline (0.00s) === RUN TestCleanupCompactRemovesTempFiles I0720 21:30:05.271101 volume_vacuum.go:349 Cleaning up volume 1 vacuuming... --- PASS: TestCleanupCompactRemovesTempFiles (0.00s) === RUN TestCompactByIndex_DropsDanglingNeedle E0720 21:30:05.271898 needle_read.go:52 /tmp/TestCompactByIndex_DropsDanglingNeedle3816087397/001/1.dat read 0 dataSize 2080 offset 1051616 fileSize 3040: EOF W0720 21:30:05.271914 volume_vacuum.go:705 vacuum volume 1: dropping unreadable needle key=108 offset=1051616 size=2048: EOF W0720 21:30:05.271921 volume_vacuum.go:731 vacuum volume 1: dropped 1 unreadable index entries (2048 data bytes) during compaction I0720 21:30:05.271965 needle_map_memory.go:113 loading idx from offset 0 for file: /tmp/TestCompactByIndex_DropsDanglingNeedle3816087397/001/1.cpx --- PASS: TestCompactByIndex_DropsDanglingNeedle (0.00s) === RUN TestSearchVolumesWithDeletedNeedles offset: 8792, isLast: false --- PASS: TestSearchVolumesWithDeletedNeedles (0.00s) === RUN TestDestroyEmptyVolumeWithOnlyEmpty I0720 21:30:05.272556 volume_write.go:132 removed volume file /tmp/TestDestroyEmptyVolumeWithOnlyEmpty2081443858/001/1.dat (size=8) I0720 21:30:05.272570 volume_write.go:132 removed volume file /tmp/TestDestroyEmptyVolumeWithOnlyEmpty2081443858/001/1.idx (size=0) --- PASS: TestDestroyEmptyVolumeWithOnlyEmpty (0.00s) === RUN TestDestroyEmptyVolumeWithoutOnlyEmpty I0720 21:30:05.272913 volume_write.go:132 removed volume file /tmp/TestDestroyEmptyVolumeWithoutOnlyEmpty2183023554/001/1.dat (size=8) I0720 21:30:05.272927 volume_write.go:132 removed volume file /tmp/TestDestroyEmptyVolumeWithoutOnlyEmpty2183023554/001/1.idx (size=0) --- PASS: TestDestroyEmptyVolumeWithoutOnlyEmpty (0.00s) === RUN TestDestroyNonemptyVolumeWithOnlyEmpty --- PASS: TestDestroyNonemptyVolumeWithOnlyEmpty (0.00s) === RUN TestDestroyNonemptyVolumeWithoutOnlyEmpty I0720 21:30:05.273462 volume_write.go:132 removed volume file /tmp/TestDestroyNonemptyVolumeWithoutOnlyEmpty1412128961/001/1.dat (size=480) I0720 21:30:05.273479 volume_write.go:132 removed volume file /tmp/TestDestroyNonemptyVolumeWithoutOnlyEmpty1412128961/001/1.idx (size=16) --- PASS: TestDestroyNonemptyVolumeWithoutOnlyEmpty (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage 2.975s ? github.com/seaweedfs/seaweedfs/weed/storage/backend [no test files] ? github.com/seaweedfs/seaweedfs/weed/storage/backend/all [no test files] === RUN TestMemoryMapMaxSizeReadWrite --- PASS: TestMemoryMapMaxSizeReadWrite (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map 0.027s ? github.com/seaweedfs/seaweedfs/weed/storage/backend/rclone_backend [no test files] === RUN TestReadAtReturnsEOFForShortRead --- PASS: TestReadAtReturnsEOFForShortRead (0.00s) === RUN TestReadAtClearsEOFOnFullRead --- PASS: TestReadAtClearsEOFOnFullRead (0.00s) === RUN TestReadAtRejectsInvalidRequestsLocally === RUN TestReadAtRejectsInvalidRequestsLocally/empty_buffer === RUN TestReadAtRejectsInvalidRequestsLocally/negative_offset --- PASS: TestReadAtRejectsInvalidRequestsLocally (0.00s) --- PASS: TestReadAtRejectsInvalidRequestsLocally/empty_buffer (0.00s) --- PASS: TestReadAtRejectsInvalidRequestsLocally/negative_offset (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend 0.084s === RUN TestBitrotSidecarBytes_RustInterop --- PASS: TestBitrotSidecarBytes_RustInterop (0.00s) === RUN TestBitrotSidecarRoundTripAndSelfIntegrity --- PASS: TestBitrotSidecarRoundTripAndSelfIntegrity (0.00s) === RUN TestShardChecksumBuilderCrossesBoundaries --- PASS: TestShardChecksumBuilderCrossesBoundaries (0.02s) === RUN TestValidateBitrotManifest --- PASS: TestValidateBitrotManifest (0.00s) === RUN TestEncodeProducesVerifiableSidecar I0720 21:30:02.451861 ec_encoder.go:138 encodeDatFile /tmp/TestEncodeProducesVerifiableSidecar640428960/001/7.dat size:6291456 with EC context 10+4 (total: 14) --- PASS: TestEncodeProducesVerifiableSidecar (0.08s) === RUN TestRebuildExcludesCorruptPresentShard I0720 21:30:02.517126 ec_encoder.go:138 encodeDatFile /tmp/TestRebuildExcludesCorruptPresentShard3910963097/001/9.dat size:6291456 with EC context 10+4 (total: 14) W0720 21:30:02.559063 ec_encoder.go:238 bitrot: present shard 3 for /tmp/TestRebuildExcludesCorruptPresentShard3910963097/001/9 fails checksum; excluding from rebuild inputs and regenerating I0720 21:30:02.559310 ec_encoder.go:254 rebuilding /tmp/TestRebuildExcludesCorruptPresentShard3910963097/001/9: 12 shards present, 2 missing [0 3], config 10+4 (total: 14) --- PASS: TestRebuildExcludesCorruptPresentShard (0.10s) === RUN TestComputeProtectionFromShardsBackfill I0720 21:30:02.620058 ec_encoder.go:138 encodeDatFile /tmp/TestComputeProtectionFromShardsBackfill1658178853/001/11.dat size:6291456 with EC context 10+4 (total: 14) --- PASS: TestComputeProtectionFromShardsBackfill (0.08s) === RUN TestRebuildFailClosedCleansUpGeneratedShard I0720 21:30:02.683533 ec_encoder.go:138 encodeDatFile /tmp/TestRebuildFailClosedCleansUpGeneratedShard2979437370/001/13.dat size:6291456 with EC context 10+4 (total: 14) I0720 21:30:02.700225 ec_encoder.go:254 rebuilding /tmp/TestRebuildFailClosedCleansUpGeneratedShard2979437370/001/13: 13 shards present, 1 missing [0], config 10+4 (total: 14) I0720 21:30:02.719794 ec_encoder.go:254 rebuilding /tmp/TestRebuildFailClosedCleansUpGeneratedShard2979437370/001/13: 13 shards present, 1 missing [0], config 10+4 (total: 14) --- PASS: TestRebuildFailClosedCleansUpGeneratedShard (0.09s) === RUN TestRemoveBitrotSidecars --- PASS: TestRemoveBitrotSidecars (0.00s) === RUN TestEcConsistency_WritesBetweenEncodeAndEcx I0720 21:30:02.755041 ec_encoder.go:138 encodeDatFile /tmp/TestEcConsistency_WritesBetweenEncodeAndEcx3361371614/001/consistency.dat size:103000 with EC context 10+4 (total: 14) ec_consistency_test.go:91: Trying to read needle 2 at offset 103000 size 5032 from EC shards (shardSize=10300) ec_consistency_test.go:92: Intervals: [{BlockIndex:30 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:31 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:32 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:33 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:34 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:35 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:36 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:37 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:38 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:39 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:40 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:41 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:42 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:43 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:44 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:45 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:46 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:47 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:48 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:49 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:50 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:51 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:52 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:53 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:54 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:55 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:56 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:57 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:58 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:59 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:60 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:61 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:62 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:63 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:64 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:65 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:66 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:67 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:68 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:69 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:70 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:71 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:72 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:73 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:74 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:75 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:76 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:77 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:78 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:79 InnerBlockOffset:0 Size:100 IsLargeBlock:false LargeBlockRowsCount:1} {BlockIndex:80 InnerBlockOffset:0 Size:32 IsLargeBlock:false LargeBlockRowsCount:1}] ec_consistency_test.go:99: CONFIRMED: Read error for needle written after EC encoding: read past end of shard 0: offset 10300 + size 100 > fileSize 10300 ec_consistency_test.go:129: Original data reads correctly from EC shards --- PASS: TestEcConsistency_WritesBetweenEncodeAndEcx (0.02s) === RUN TestEcConsistency_ExactLargeRowEncoding I0720 21:30:02.770266 ec_encoder.go:138 encodeDatFile /tmp/TestEcConsistency_ExactLargeRowEncoding2928376172/001/exact.dat size:100000 with EC context 10+4 (total: 14) --- PASS: TestEcConsistency_ExactLargeRowEncoding (0.01s) === RUN TestMinusParityShardsRatio === RUN TestMinusParityShardsRatio/default_10+4_all_present === RUN TestMinusParityShardsRatio/zero_ratio_falls_back_to_default === RUN TestMinusParityShardsRatio/9+3_all_present === RUN TestMinusParityShardsRatio/9+3_drops_parity_id_9 === RUN TestMinusParityShardsRatio/12+4_keeps_data_ids_>=_10 === RUN TestMinusParityShardsRatio/16+6_keeps_data_ids_>=_10 --- PASS: TestMinusParityShardsRatio (0.00s) --- PASS: TestMinusParityShardsRatio/default_10+4_all_present (0.00s) --- PASS: TestMinusParityShardsRatio/zero_ratio_falls_back_to_default (0.00s) --- PASS: TestMinusParityShardsRatio/9+3_all_present (0.00s) --- PASS: TestMinusParityShardsRatio/9+3_drops_parity_id_9 (0.00s) --- PASS: TestMinusParityShardsRatio/12+4_keeps_data_ids_>=_10 (0.00s) --- PASS: TestMinusParityShardsRatio/16+6_keeps_data_ids_>=_10 (0.00s) === RUN TestRebuildEcxFile_AppliesTombstonesAndRemovesJournal --- PASS: TestRebuildEcxFile_AppliesTombstonesAndRemovesJournal (0.00s) === RUN TestRebuildEcxFile_TornJournalAborts --- PASS: TestRebuildEcxFile_TornJournalAborts (0.00s) === RUN TestRebuildEcxFile_NoJournalNoop --- PASS: TestRebuildEcxFile_NoJournalNoop (0.00s) === RUN TestRebuildEcFiles_TruncatedInputErrorsWithoutPublishing --- PASS: TestRebuildEcFiles_TruncatedInputErrorsWithoutPublishing (0.00s) === RUN TestRebuildEcFiles_ZeroSizeInputsErrors --- PASS: TestRebuildEcFiles_ZeroSizeInputsErrors (0.00s) === RUN TestRebuildEcFiles_HappyPathRebuildsByteIdentical I0720 21:30:02.786044 ec_encoder.go:138 encodeDatFile /tmp/TestRebuildEcFiles_HappyPathRebuildsByteIdentical2768403002/001/vol.dat size:7000 with EC context 10+4 (total: 14) I0720 21:30:02.799420 ec_encoder.go:254 rebuilding /tmp/TestRebuildEcFiles_HappyPathRebuildsByteIdentical2768403002/001/vol: 13 shards present, 1 missing [5], config 10+4 (total: 14) --- PASS: TestRebuildEcFiles_HappyPathRebuildsByteIdentical (0.03s) === RUN TestRebuildEcFiles_CustomRatioRebuildsByteIdentical I0720 21:30:02.820832 ec_encoder.go:138 encodeDatFile /tmp/TestRebuildEcFiles_CustomRatioRebuildsByteIdentical861809631/001/vol.dat size:9000 with EC context 9+3 (total: 12) I0720 21:30:02.846528 ec_encoder.go:254 rebuilding /tmp/TestRebuildEcFiles_CustomRatioRebuildsByteIdentical861809631/001/vol: 11 shards present, 1 missing [7], config 9+3 (total: 12) --- PASS: TestRebuildEcFiles_CustomRatioRebuildsByteIdentical (0.04s) === RUN TestEcReadRoundTrip === RUN TestEcReadRoundTrip/1_large_row_exact I0720 21:30:02.859339 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTrip1_large_row_exact1999284847/001/rt_100000.dat size:100000 with EC context 10+4 (total: 14) ec_roundtrip_test.go:59: WARN: EC read mismatch at offset 50000 (fallback path, shardFileSize=10000) === RUN TestEcReadRoundTrip/2_large_rows_exact I0720 21:30:02.862519 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTrip2_large_rows_exact1532305911/001/rt_200000.dat size:200000 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/3_large_rows_exact I0720 21:30:02.886810 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTrip3_large_rows_exact1447692915/001/rt_300000.dat size:300000 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/1_large_row_plus_1 I0720 21:30:02.895323 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTrip1_large_row_plus_11993836372/001/rt_100001.dat size:100001 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/2_large_rows_plus_small I0720 21:30:02.898956 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTrip2_large_rows_plus_small1050970572/001/rt_201000.dat size:201000 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/1_large_row_plus_half_small I0720 21:30:02.905657 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTrip1_large_row_plus_half_small2787906740/001/rt_100500.dat size:100500 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/just_under_1_large_row I0720 21:30:02.909191 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTripjust_under_1_large_row3602182981/001/rt_99999.dat size:99999 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/just_under_2_large_rows I0720 21:30:02.916067 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTripjust_under_2_large_rows1392618746/001/rt_199999.dat size:199999 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/small_only I0720 21:30:02.928338 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTripsmall_only1002745780/001/rt_3000.dat size:3000 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/small_single_row I0720 21:30:02.929015 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTripsmall_single_row3496334210/001/rt_1000.dat size:1000 with EC context 10+4 (total: 14) === RUN TestEcReadRoundTrip/boundary_spanning I0720 21:30:02.929810 ec_encoder.go:138 encodeDatFile /tmp/TestEcReadRoundTripboundary_spanning324382048/001/rt_105050.dat size:105050 with EC context 10+4 (total: 14) --- PASS: TestEcReadRoundTrip (0.07s) --- PASS: TestEcReadRoundTrip/1_large_row_exact (0.00s) --- PASS: TestEcReadRoundTrip/2_large_rows_exact (0.02s) --- PASS: TestEcReadRoundTrip/3_large_rows_exact (0.01s) --- PASS: TestEcReadRoundTrip/1_large_row_plus_1 (0.00s) --- PASS: TestEcReadRoundTrip/2_large_rows_plus_small (0.01s) --- PASS: TestEcReadRoundTrip/1_large_row_plus_half_small (0.00s) --- PASS: TestEcReadRoundTrip/just_under_1_large_row (0.01s) --- PASS: TestEcReadRoundTrip/just_under_2_large_rows (0.01s) --- PASS: TestEcReadRoundTrip/small_only (0.00s) --- PASS: TestEcReadRoundTrip/small_single_row (0.00s) --- PASS: TestEcReadRoundTrip/boundary_spanning (0.00s) === RUN TestEcOffByOneBug_Issue8947 I0720 21:30:02.935908 ec_encoder.go:138 encodeDatFile /tmp/TestEcOffByOneBug_Issue89472504261907/001/bug_200000.dat size:200000 with EC context 10+4 (total: 14) ec_roundtrip_test.go:269: Buggy formula returned wrong data: expected first bytes bbc0b4892a086ba5, got 4db40816265dc5be ec_roundtrip_test.go:279: Fixed: nLargeBlockRows=2, interval={BlockIndex:11 InnerBlockOffset:50 Size:50 IsLargeBlock:true LargeBlockRowsCount:2} ec_roundtrip_test.go:280: Buggy: nLargeBlockRows=1, interval={BlockIndex:100 InnerBlockOffset:50 Size:50 IsLargeBlock:false LargeBlockRowsCount:1} --- PASS: TestEcOffByOneBug_Issue8947 (0.01s) === RUN TestEcDecodeDatRoundTrip === RUN TestEcDecodeDatRoundTrip/size_1000 I0720 21:30:02.941675 ec_encoder.go:138 encodeDatFile /tmp/TestEcDecodeDatRoundTripsize_10002891213938/001/dec_1000.dat size:1000 with EC context 10+4 (total: 14) === RUN TestEcDecodeDatRoundTrip/size_10485760 I0720 21:30:02.997675 ec_encoder.go:138 encodeDatFile /tmp/TestEcDecodeDatRoundTripsize_104857602962593878/001/dec_10485760.dat size:10485760 with EC context 10+4 (total: 14) === RUN TestEcDecodeDatRoundTrip/size_10486260 I0720 21:30:03.048944 ec_encoder.go:138 encodeDatFile /tmp/TestEcDecodeDatRoundTripsize_104862604088180191/001/dec_10486260.dat size:10486260 with EC context 10+4 (total: 14) --- PASS: TestEcDecodeDatRoundTrip (0.15s) --- PASS: TestEcDecodeDatRoundTrip/size_1000 (0.02s) --- PASS: TestEcDecodeDatRoundTrip/size_10485760 (0.06s) --- PASS: TestEcDecodeDatRoundTrip/size_10486260 (0.07s) === RUN TestShardsInfo_SetAndGet --- PASS: TestShardsInfo_SetAndGet (0.00s) === RUN TestShardsInfo_SortedOrder --- PASS: TestShardsInfo_SortedOrder (0.00s) === RUN TestShardsInfo_Delete --- PASS: TestShardsInfo_Delete (0.00s) === RUN TestShardsInfo_Update --- PASS: TestShardsInfo_Update (0.00s) === RUN TestShardsInfo_TotalSize --- PASS: TestShardsInfo_TotalSize (0.00s) === RUN TestShardsInfo_Sizes --- PASS: TestShardsInfo_Sizes (0.00s) === RUN TestShardsInfo_Copy --- PASS: TestShardsInfo_Copy (0.00s) === RUN TestShardsInfo_AddSubtract --- PASS: TestShardsInfo_AddSubtract (0.00s) === RUN TestShardsInfo_PlusMinus --- PASS: TestShardsInfo_PlusMinus (0.00s) === RUN TestShardsInfo_DeleteParityShards --- PASS: TestShardsInfo_DeleteParityShards (0.00s) === RUN TestShardsInfo_FromVolumeEcShardInformationMessage === RUN TestShardsInfo_FromVolumeEcShardInformationMessage/nil_message === RUN TestShardsInfo_FromVolumeEcShardInformationMessage/single_shard === RUN TestShardsInfo_FromVolumeEcShardInformationMessage/multiple_shards === RUN TestShardsInfo_FromVolumeEcShardInformationMessage/missing_sizes --- PASS: TestShardsInfo_FromVolumeEcShardInformationMessage (0.00s) --- PASS: TestShardsInfo_FromVolumeEcShardInformationMessage/nil_message (0.00s) --- PASS: TestShardsInfo_FromVolumeEcShardInformationMessage/single_shard (0.00s) --- PASS: TestShardsInfo_FromVolumeEcShardInformationMessage/multiple_shards (0.00s) --- PASS: TestShardsInfo_FromVolumeEcShardInformationMessage/missing_sizes (0.00s) === RUN TestEcShardsTotalSize === RUN TestEcShardsTotalSize/nil_message === RUN TestEcShardsTotalSize/all_shards_present === RUN TestEcShardsTotalSize/only_parity_shards --- PASS: TestEcShardsTotalSize (0.00s) --- PASS: TestEcShardsTotalSize/nil_message (0.00s) --- PASS: TestEcShardsTotalSize/all_shards_present (0.00s) --- PASS: TestEcShardsTotalSize/only_parity_shards (0.00s) === RUN TestEcShardsDataSize === RUN TestEcShardsDataSize/nil_message === RUN TestEcShardsDataSize/data_shards_only === RUN TestEcShardsDataSize/mixed_data_and_parity_(default_10+4) === RUN TestEcShardsDataSize/only_parity_shards_(excluded) === RUN TestEcShardsDataSize/missing_sizes_tolerated === RUN TestEcShardsDataSize/custom_6+3_ratio:_shards_0..5_are_data,_6..8_are_parity === RUN TestEcShardsDataSize/custom_16+6_ratio:_shard_id_12_is_data,_not_parity === RUN TestEcShardsDataSize/negative_dataShards_falls_back_to_default --- PASS: TestEcShardsDataSize (0.00s) --- PASS: TestEcShardsDataSize/nil_message (0.00s) --- PASS: TestEcShardsDataSize/data_shards_only (0.00s) --- PASS: TestEcShardsDataSize/mixed_data_and_parity_(default_10+4) (0.00s) --- PASS: TestEcShardsDataSize/only_parity_shards_(excluded) (0.00s) --- PASS: TestEcShardsDataSize/missing_sizes_tolerated (0.00s) --- PASS: TestEcShardsDataSize/custom_6+3_ratio:_shards_0..5_are_data,_6..8_are_parity (0.00s) --- PASS: TestEcShardsDataSize/custom_16+6_ratio:_shard_id_12_is_data,_not_parity (0.00s) --- PASS: TestEcShardsDataSize/negative_dataShards_falls_back_to_default (0.00s) === RUN TestShardsInfo_String --- PASS: TestShardsInfo_String (0.00s) === RUN TestShardsInfo_ConcurrentAccess --- PASS: TestShardsInfo_ConcurrentAccess (0.00s) === RUN TestEncodingDecoding I0720 21:30:03.093109 ec_encoder.go:138 encodeDatFile 1.dat size:2590912 with EC context 10+4 (total: 14) --- PASS: TestEncodingDecoding (0.46s) === RUN TestLocateData [{BlockIndex:5 InnerBlockOffset:100 Size:9900 IsLargeBlock:true LargeBlockRowsCount:1} {BlockIndex:6 InnerBlockOffset:0 Size:10000 IsLargeBlock:true LargeBlockRowsCount:1} {BlockIndex:7 InnerBlockOffset:0 Size:10000 IsLargeBlock:true LargeBlockRowsCount:1} {BlockIndex:8 InnerBlockOffset:0 Size:10000 IsLargeBlock:true LargeBlockRowsCount:1} {BlockIndex:9 InnerBlockOffset:0 Size:10000 IsLargeBlock:true LargeBlockRowsCount:1} {BlockIndex:0 InnerBlockOffset:0 Size:1 IsLargeBlock:false LargeBlockRowsCount:1}] --- PASS: TestLocateData (0.00s) === RUN TestLocateData2 --- PASS: TestLocateData2 (0.00s) === RUN TestLocateData3 {BlockIndex:8876 InnerBlockOffset:912752 Size:112568 IsLargeBlock:false LargeBlockRowsCount:2} --- PASS: TestLocateData3 (0.00s) === RUN TestLocateData_ExactMultiple_Issue8947 --- PASS: TestLocateData_ExactMultiple_Issue8947 (0.00s) === RUN TestLocateData_Issue8179 --- PASS: TestLocateData_Issue8179 (0.00s) === RUN TestPositioning offset: 31300679656 size: 1167 offset: 11513014944 size: 66044 offset: 26311863528 size: 26823 interval: {BlockIndex:14852 InnerBlockOffset:994536 Size:26856 IsLargeBlock:false LargeBlockRowsCount:1}, shardId: 2, shardOffset: 2631871720 --- PASS: TestPositioning (0.00s) === RUN TestNewEcVolumeLoadsEncodeTsNs --- PASS: TestNewEcVolumeLoadsEncodeTsNs (0.00s) === RUN TestNewEcVolumeDoesNotWriteStubVif W0720 21:30:03.554368 ec_volume.go:182 vif file not found, using defaults, volumeId:124, filename:/tmp/TestNewEcVolumeDoesNotWriteStubVif1543071712/001/124.vif --- PASS: TestNewEcVolumeDoesNotWriteStubVif (0.00s) === RUN TestNewEcVolumeLoadsCustomRatio --- PASS: TestNewEcVolumeLoadsCustomRatio (0.00s) === RUN TestRequireRecoverableShardSet_AllPresent --- PASS: TestRequireRecoverableShardSet_AllPresent (0.00s) === RUN TestRequireRecoverableShardSet_DegradedButRecoverable --- PASS: TestRequireRecoverableShardSet_DegradedButRecoverable (0.00s) === RUN TestRequireRecoverableShardSet_BelowDataShards --- PASS: TestRequireRecoverableShardSet_BelowDataShards (0.00s) === RUN TestRequireRecoverableShardSet_EmptyBitmap --- PASS: TestRequireRecoverableShardSet_EmptyBitmap (0.00s) === RUN TestRequireRecoverableShardSet_CustomRatio --- PASS: TestRequireRecoverableShardSet_CustomRatio (0.00s) === RUN TestRequireRecoverableShardSet_RejectsInvalidParams --- PASS: TestRequireRecoverableShardSet_RejectsInvalidParams (0.00s) === RUN TestSummarizeShardInventory_Deterministic --- PASS: TestSummarizeShardInventory_Deterministic (0.00s) === RUN TestSummarizeShardInventory_IncludesError --- PASS: TestSummarizeShardInventory_IncludesError (0.00s) === RUN TestHasLiveNeedles_AllDeletedIsFalse --- PASS: TestHasLiveNeedles_AllDeletedIsFalse (0.00s) === RUN TestHasLiveNeedles_WithLiveEntryIsTrue --- PASS: TestHasLiveNeedles_WithLiveEntryIsTrue (0.00s) === RUN TestHasLiveNeedles_EmptyFileIsFalse --- PASS: TestHasLiveNeedles_EmptyFileIsFalse (0.00s) === RUN TestHasLiveNeedles_TruncatedEcxErrors --- PASS: TestHasLiveNeedles_TruncatedEcxErrors (0.00s) === RUN TestWriteIdxFileFromEcIndex_PreservesDeletedNeedles --- PASS: TestWriteIdxFileFromEcIndex_PreservesDeletedNeedles (0.00s) === RUN TestWriteIdxFileFromEcIndex_ProcessesEcjJournal --- PASS: TestWriteIdxFileFromEcIndex_ProcessesEcjJournal (0.00s) === RUN TestDecodeAtomicPublish --- PASS: TestDecodeAtomicPublish (0.00s) === RUN TestDecodeWithNonEmptyEcj_AllDeleted --- PASS: TestDecodeWithNonEmptyEcj_AllDeleted (0.00s) === RUN TestDecodeWithNonEmptyEcj_PartiallyDeleted --- PASS: TestDecodeWithNonEmptyEcj_PartiallyDeleted (0.00s) === RUN TestDecodeWithEmptyEcj --- PASS: TestDecodeWithEmptyEcj (0.00s) === RUN TestDecodeWithNoEcjFile --- PASS: TestDecodeWithNoEcjFile (0.00s) === RUN TestEcxFileDeletionVisibleAfterSync --- PASS: TestEcxFileDeletionVisibleAfterSync (0.00s) === RUN TestEcxFileDeletionWithSeparateHandles --- PASS: TestEcxFileDeletionWithSeparateHandles (0.00s) === RUN TestEcVolumeDeleteDurableToJournal W0720 21:30:03.556415 volume_info.go:46 empty volume info file /tmp/TestEcVolumeDeleteDurableToJournal1924732705/001/test_1.vif, treating as non-existent W0720 21:30:03.556424 ec_volume.go:182 vif file not found, using defaults, volumeId:1, filename:/tmp/TestEcVolumeDeleteDurableToJournal1924732705/001/test_1.vif --- PASS: TestEcVolumeDeleteDurableToJournal (0.00s) === RUN TestScrubLocal_SuppressesDeleteStateDisagreement W0720 21:30:03.556729 volume_info.go:46 empty volume info file /tmp/TestScrubLocal_SuppressesDeleteStateDisagreement4149227686/001/test_1.vif, treating as non-existent W0720 21:30:03.556749 ec_volume.go:182 vif file not found, using defaults, volumeId:1, filename:/tmp/TestScrubLocal_SuppressesDeleteStateDisagreement4149227686/001/test_1.vif --- PASS: TestScrubLocal_SuppressesDeleteStateDisagreement (0.00s) === RUN TestScrubLocal_ReportsGenuineSizeCorruption W0720 21:30:03.557017 volume_info.go:46 empty volume info file /tmp/TestScrubLocal_ReportsGenuineSizeCorruption4255428118/001/test_1.vif, treating as non-existent W0720 21:30:03.557035 ec_volume.go:182 vif file not found, using defaults, volumeId:1, filename:/tmp/TestScrubLocal_ReportsGenuineSizeCorruption4255428118/001/test_1.vif --- PASS: TestScrubLocal_ReportsGenuineSizeCorruption (0.00s) === RUN TestEcVolumeFileAndDeleteCountInitial W0720 21:30:03.557306 volume_info.go:46 empty volume info file /tmp/TestEcVolumeFileAndDeleteCountInitial499973517/001/test_1.vif, treating as non-existent W0720 21:30:03.557325 ec_volume.go:182 vif file not found, using defaults, volumeId:1, filename:/tmp/TestEcVolumeFileAndDeleteCountInitial499973517/001/test_1.vif --- PASS: TestEcVolumeFileAndDeleteCountInitial (0.00s) === RUN TestEcVolumeFileAndDeleteCountAfterDelete W0720 21:30:03.557582 volume_info.go:46 empty volume info file /tmp/TestEcVolumeFileAndDeleteCountAfterDelete1281485732/001/test_1.vif, treating as non-existent W0720 21:30:03.557606 ec_volume.go:182 vif file not found, using defaults, volumeId:1, filename:/tmp/TestEcVolumeFileAndDeleteCountAfterDelete1281485732/001/test_1.vif --- PASS: TestEcVolumeFileAndDeleteCountAfterDelete (0.00s) === RUN TestShardsInfoDeleteParityShards --- PASS: TestShardsInfoDeleteParityShards (0.00s) === RUN TestShardsInfoAsSlice --- PASS: TestShardsInfoAsSlice (0.00s) === RUN TestShardsInfoSerialize === RUN TestShardsInfoSerialize/no_bits === RUN TestShardsInfoSerialize/single_shard,_first === RUN TestShardsInfoSerialize/single_shard,_5th === RUN TestShardsInfoSerialize/multiple_shards --- PASS: TestShardsInfoSerialize (0.00s) --- PASS: TestShardsInfoSerialize/no_bits (0.00s) --- PASS: TestShardsInfoSerialize/single_shard,_first (0.00s) --- PASS: TestShardsInfoSerialize/single_shard,_5th (0.00s) --- PASS: TestShardsInfoSerialize/multiple_shards (0.00s) === RUN TestShardsInfoFromVolumeEcShardInformationMessage === RUN TestShardsInfoFromVolumeEcShardInformationMessage/no_msg === RUN TestShardsInfoFromVolumeEcShardInformationMessage/no_shards === RUN TestShardsInfoFromVolumeEcShardInformationMessage/single_shard === RUN TestShardsInfoFromVolumeEcShardInformationMessage/multiple_shards === RUN TestShardsInfoFromVolumeEcShardInformationMessage/multiple_shards_with_missing_sizes --- PASS: TestShardsInfoFromVolumeEcShardInformationMessage (0.00s) --- PASS: TestShardsInfoFromVolumeEcShardInformationMessage/no_msg (0.00s) --- PASS: TestShardsInfoFromVolumeEcShardInformationMessage/no_shards (0.00s) --- PASS: TestShardsInfoFromVolumeEcShardInformationMessage/single_shard (0.00s) --- PASS: TestShardsInfoFromVolumeEcShardInformationMessage/multiple_shards (0.00s) --- PASS: TestShardsInfoFromVolumeEcShardInformationMessage/multiple_shards_with_missing_sizes (0.00s) === RUN TestShardsInfoCombine --- PASS: TestShardsInfoCombine (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding 1.239s ? github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding/distribution [no test files] === RUN TestPickBestDiskOnNode === RUN TestPickBestDiskOnNode/skips_disks_with_no_free_slots === RUN TestPickBestDiskOnNode/spreads_a_volume's_shards_across_disks === RUN TestPickBestDiskOnNode/data_shard_avoids_disk_holding_parity === RUN TestPickBestDiskOnNode/anti-affinity_follows_the_supplied_ratio_boundary === RUN TestPickBestDiskOnNode/only_matching_disk_type_when_set --- PASS: TestPickBestDiskOnNode (0.00s) --- PASS: TestPickBestDiskOnNode/skips_disks_with_no_free_slots (0.00s) --- PASS: TestPickBestDiskOnNode/spreads_a_volume's_shards_across_disks (0.00s) --- PASS: TestPickBestDiskOnNode/data_shard_avoids_disk_holding_parity (0.00s) --- PASS: TestPickBestDiskOnNode/anti-affinity_follows_the_supplied_ratio_boundary (0.00s) --- PASS: TestPickBestDiskOnNode/only_matching_disk_type_when_set (0.00s) === RUN TestPlanSourceDiskAttribution --- PASS: TestPlanSourceDiskAttribution (0.00s) === RUN TestPlanSpreadsAcrossDestinationDisks --- PASS: TestPlanSpreadsAcrossDestinationDisks (0.00s) === RUN TestPlanCrossRackParityAntiAffinity --- PASS: TestPlanCrossRackParityAntiAffinity (0.00s) === RUN TestWithinRackParityAntiAffinity --- PASS: TestWithinRackParityAntiAffinity (0.00s) === RUN TestPlanReplicaPlacementCapsPerRack --- PASS: TestPlanReplicaPlacementCapsPerRack (0.00s) === RUN TestPlanDedup --- PASS: TestPlanDedup (0.00s) === RUN TestCeilDivide --- PASS: TestCeilDivide (0.00s) === RUN TestPlanConvergesToBalancedLoadOverRounds balancer_test.go:335: converged after 13 rounds: map[10.0.0.1:140 10.0.0.2:140 10.0.0.3:140 10.0.0.4:140 10.0.0.5:140 10.0.0.6:140] --- PASS: TestPlanConvergesToBalancedLoadOverRounds (0.02s) === RUN TestGlobalImbalanceMovesFromFullToEmpty --- PASS: TestGlobalImbalanceMovesFromFullToEmpty (0.00s) === RUN TestGlobalImbalanceHeterogeneousCapacity --- PASS: TestGlobalImbalanceHeterogeneousCapacity (0.00s) === RUN TestGlobalImbalanceSkipsFullNodes --- PASS: TestGlobalImbalanceSkipsFullNodes (0.00s) === RUN TestPlanBalancesSkewedDataParityWithEvenTotals --- PASS: TestPlanBalancesSkewedDataParityWithEvenTotals (0.00s) === RUN TestPlanVolumeRatioOverridesCollection --- PASS: TestPlanVolumeRatioOverridesCollection (0.00s) === RUN TestGlobalPrefersVolumeAbsentFromDestination --- PASS: TestGlobalPrefersVolumeAbsentFromDestination (0.00s) === RUN TestPlanKeepsCollectionsWithSameVolumeIdDistinct --- PASS: TestPlanKeepsCollectionsWithSameVolumeIdDistinct (0.00s) === RUN TestDedupFreesCapacityForLaterPhases --- PASS: TestDedupFreesCapacityForLaterPhases (0.00s) === RUN TestWithinRackSpreadsAcrossMachines --- PASS: TestWithinRackSpreadsAcrossMachines (0.00s) === RUN TestWithinRackMachineSpreadBalancesCombinedOccupancy --- PASS: TestWithinRackMachineSpreadBalancesCombinedOccupancy (0.00s) === RUN TestWithinRackMachineSpreadActsOnExactlyHalfSkew --- PASS: TestWithinRackMachineSpreadActsOnExactlyHalfSkew (0.00s) === RUN TestWithinRackSpreadUsesRackLocalShardCount --- PASS: TestWithinRackSpreadUsesRackLocalShardCount (0.00s) === RUN TestWithinRackNodeFallbackHonorsThreshold --- PASS: TestWithinRackNodeFallbackHonorsThreshold (0.00s) === RUN TestWithinRackSpreadDefaultsToNodes --- PASS: TestWithinRackSpreadDefaultsToNodes (0.00s) === RUN TestGlobalDoesNotConcentrateVolumeAcrossMachines --- PASS: TestGlobalDoesNotConcentrateVolumeAcrossMachines (0.00s) === RUN TestWithinRackMachineSkipsCappedMachine --- PASS: TestWithinRackMachineSkipsCappedMachine (0.00s) === RUN TestGlobalPrefersVolumeAbsentFromDestinationMachine --- PASS: TestGlobalPrefersVolumeAbsentFromDestinationMachine (0.00s) === RUN TestPlaceStrictSpreadAndCaps --- PASS: TestPlaceStrictSpreadAndCaps (0.00s) === RUN TestPlaceDistributesEvenlyAcrossManyVolumes --- PASS: TestPlaceDistributesEvenlyAcrossManyVolumes (0.02s) === RUN TestPlaceSpreadsAcrossMachines --- PASS: TestPlaceSpreadsAcrossMachines (0.00s) === RUN TestPlaceStrictFailsAndRollsBack --- PASS: TestPlaceStrictFailsAndRollsBack (0.00s) === RUN TestPlaceDurabilityFirstRelaxesRP --- PASS: TestPlaceDurabilityFirstRelaxesRP (0.00s) === RUN TestPlaceSameRackCountIsDirectPerNodeCap --- PASS: TestPlaceSameRackCountIsDirectPerNodeCap (0.00s) === RUN TestPlaceDiskTypeHardFilter --- PASS: TestPlaceDiskTypeHardFilter (0.00s) === RUN TestPlaceDiskTypeUnavailableFails --- PASS: TestPlaceDiskTypeUnavailableFails (0.00s) === RUN TestPlaceHDDRequestMatchesEmptyTypeDisks --- PASS: TestPlaceHDDRequestMatchesEmptyTypeDisks (0.00s) === RUN TestPlaceDurabilityCapRejectsSkewed --- PASS: TestPlaceDurabilityCapRejectsSkewed (0.00s) === RUN TestReleaseVolumeShards --- PASS: TestReleaseVolumeShards (0.00s) === RUN TestClearShardAccounting --- PASS: TestClearShardAccounting (0.00s) === RUN TestPlaceDiskTypePreferSpills --- PASS: TestPlaceDiskTypePreferSpills (0.00s) === RUN TestPlacePreferredTagsUseTaggedDisks --- PASS: TestPlacePreferredTagsUseTaggedDisks (0.00s) === RUN TestPlacePreferredTagsSpillWhenInsufficient --- PASS: TestPlacePreferredTagsSpillWhenInsufficient (0.00s) === RUN TestPlaceStrictCapsCountEligibleRacks --- PASS: TestPlaceStrictCapsCountEligibleRacks (0.00s) === RUN TestFromActiveTopology --- PASS: TestFromActiveTopology (0.00s) === RUN TestFromActiveTopologySkipsPhysicallyFullDisk --- PASS: TestFromActiveTopologySkipsPhysicallyFullDisk (0.00s) === RUN TestFromActiveTopologyGroupsByAddressHost --- PASS: TestFromActiveTopologyGroupsByAddressHost (0.00s) === RUN TestEcShardSlotsOnDiskRoundsUp --- PASS: TestEcShardSlotsOnDiskRoundsUp (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding/ecbalancer 0.107s === RUN TestCheckIndexFile_IgnoresOffset0Tombstone --- PASS: TestCheckIndexFile_IgnoresOffset0Tombstone (0.00s) === RUN TestCheckIndexFile === RUN TestCheckIndexFile/healthy_index === RUN TestCheckIndexFile/healthy_index_with_deleted_files === RUN TestCheckIndexFile/damaged_index_(bitrot) === RUN TestCheckIndexFile/damaged_index_(truncated) === RUN TestCheckIndexFile/healthy_EC_index_with_deleted_files === RUN TestCheckIndexFile/damaged_EC_index_(bitrot) --- PASS: TestCheckIndexFile (0.00s) --- PASS: TestCheckIndexFile/healthy_index (0.00s) --- PASS: TestCheckIndexFile/healthy_index_with_deleted_files (0.00s) --- PASS: TestCheckIndexFile/damaged_index_(bitrot) (0.00s) --- PASS: TestCheckIndexFile/damaged_index_(truncated) (0.00s) --- PASS: TestCheckIndexFile/healthy_EC_index_with_deleted_files (0.00s) --- PASS: TestCheckIndexFile/damaged_EC_index_(bitrot) (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/idx 0.007s === RUN TestParseFileIdFromString --- PASS: TestParseFileIdFromString (0.00s) === RUN TestNeedleIdFileId --- PASS: TestNeedleIdFileId (0.00s) === RUN TestEagerPreGrow === RUN TestEagerPreGrow/small_content-length_grows_exactly === RUN TestEagerPreGrow/content-length_at_cap_grows_to_cap === RUN TestEagerPreGrow/content-length_above_cap_is_clamped_to_cap === RUN TestEagerPreGrow/content-length_above_sizeLimit_is_rejected === RUN TestEagerPreGrow/zero_content-length_is_a_no-op === RUN TestEagerPreGrow/negative_content-length_(chunked_encoding)_is_a_no-op === RUN TestEagerPreGrow/buffer_with_sufficient_cap_is_left_alone --- PASS: TestEagerPreGrow (0.01s) --- PASS: TestEagerPreGrow/small_content-length_grows_exactly (0.00s) --- PASS: TestEagerPreGrow/content-length_at_cap_grows_to_cap (0.00s) --- PASS: TestEagerPreGrow/content-length_above_cap_is_clamped_to_cap (0.00s) --- PASS: TestEagerPreGrow/content-length_above_sizeLimit_is_rejected (0.00s) --- PASS: TestEagerPreGrow/zero_content-length_is_a_no-op (0.00s) --- PASS: TestEagerPreGrow/negative_content-length_(chunked_encoding)_is_a_no-op (0.00s) --- PASS: TestEagerPreGrow/buffer_with_sufficient_cap_is_left_alone (0.00s) === RUN TestParseUpload_GzipStreamCount === RUN TestParseUpload_GzipStreamCount/no_MD5:_stream-count needle_parse_upload_stream_test.go:73: allocated=31280 bytes (gzipped=16611, uncompressed=4194304) === RUN TestParseUpload_GzipStreamCount/MD5_on_part:_materialize needle_parse_upload_stream_test.go:73: allocated=16850200 bytes (gzipped=16611, uncompressed=4194304) === RUN TestParseUpload_GzipStreamCount/MD5_on_request:_materialize needle_parse_upload_stream_test.go:73: allocated=16849936 bytes (gzipped=16611, uncompressed=4194304) --- PASS: TestParseUpload_GzipStreamCount (0.21s) --- PASS: TestParseUpload_GzipStreamCount/no_MD5:_stream-count (0.00s) --- PASS: TestParseUpload_GzipStreamCount/MD5_on_part:_materialize (0.08s) --- PASS: TestParseUpload_GzipStreamCount/MD5_on_request:_materialize (0.06s) === RUN TestParseUploadUsesDiscoveredFilePartContentType --- PASS: TestParseUploadUsesDiscoveredFilePartContentType (0.00s) === RUN TestReadNeedleTailLegacyChecksum --- PASS: TestReadNeedleTailLegacyChecksum (0.00s) === RUN TestReadNeedleTailRawChecksum --- PASS: TestReadNeedleTailRawChecksum (0.00s) === RUN TestReadNeedleTailCorrupted --- PASS: TestReadNeedleTailCorrupted (0.00s) === RUN TestParseKeyHash --- PASS: TestParseKeyHash (0.00s) === RUN TestNeedleParsePathRejectsDeltaOverflow --- PASS: TestNeedleParsePathRejectsDeltaOverflow (0.00s) === RUN TestAppend --- PASS: TestAppend (0.00s) === RUN TestWriteNeedle_CompatibilityWithLegacy === RUN TestWriteNeedle_CompatibilityWithLegacy/Version1 === RUN TestWriteNeedle_CompatibilityWithLegacy/Version2 === RUN TestWriteNeedle_CompatibilityWithLegacy/Version3 --- PASS: TestWriteNeedle_CompatibilityWithLegacy (0.00s) --- PASS: TestWriteNeedle_CompatibilityWithLegacy/Version1 (0.00s) --- PASS: TestWriteNeedle_CompatibilityWithLegacy/Version2 (0.00s) --- PASS: TestWriteNeedle_CompatibilityWithLegacy/Version3 (0.00s) === RUN TestNewVolumeId volume_id_test.go:11: a is not legal volume id, strconv.ParseUint: parsing "a": invalid syntax --- PASS: TestNewVolumeId (0.00s) === RUN TestVolumeId_String --- PASS: TestVolumeId_String (0.00s) === RUN TestVolumeId_Next --- PASS: TestVolumeId_Next (0.00s) === RUN TestTTLReadWrite --- PASS: TestTTLReadWrite (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/needle 0.251s === RUN TestMemoryUsage Each 13.12 Bytes Alloc = 21 MiB TotalAlloc = 46 MiB Sys = 42 MiB NumGC = 8 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.686668677s Each 12.80 Bytes Alloc = 41 MiB TotalAlloc = 92 MiB Sys = 66 MiB NumGC = 10 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.393865803s Each 12.70 Bytes Alloc = 61 MiB TotalAlloc = 138 MiB Sys = 95 MiB NumGC = 12 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.109606095s Each 12.65 Bytes Alloc = 81 MiB TotalAlloc = 183 MiB Sys = 119 MiB NumGC = 13 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.091125025s Each 12.61 Bytes Alloc = 102 MiB TotalAlloc = 229 MiB Sys = 143 MiB NumGC = 14 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.055284252s Each 12.59 Bytes Alloc = 122 MiB TotalAlloc = 274 MiB Sys = 159 MiB NumGC = 15 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.089992137s Each 12.58 Bytes Alloc = 142 MiB TotalAlloc = 320 MiB Sys = 184 MiB NumGC = 16 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.118072123s Each 12.57 Bytes Alloc = 162 MiB TotalAlloc = 365 MiB Sys = 204 MiB NumGC = 17 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.282923693s Each 12.56 Bytes Alloc = 182 MiB TotalAlloc = 411 MiB Sys = 224 MiB NumGC = 18 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.669047467s Each 12.55 Bytes Alloc = 203 MiB TotalAlloc = 456 MiB Sys = 244 MiB NumGC = 19 CompactMap = 1539351/1749659 elements on 2198 segments, 87.98% efficiency Taken = 1.492585727s --- PASS: TestMemoryUsage (12.99s) === RUN TestSegmentBsearchKey === RUN TestSegmentBsearchKey/empty_segment === RUN TestSegmentBsearchKey/new_key,_insert_at_beggining === RUN TestSegmentBsearchKey/new_key,_insert_at_end === RUN TestSegmentBsearchKey/new_key,_insert_second === RUN TestSegmentBsearchKey/new_key,_insert_in_middle === RUN TestSegmentBsearchKey/key_#1 === RUN TestSegmentBsearchKey/key_#2 === RUN TestSegmentBsearchKey/key_#3 === RUN TestSegmentBsearchKey/key_#4 === RUN TestSegmentBsearchKey/key_#5 --- PASS: TestSegmentBsearchKey (0.00s) --- PASS: TestSegmentBsearchKey/empty_segment (0.00s) --- PASS: TestSegmentBsearchKey/new_key,_insert_at_beggining (0.00s) --- PASS: TestSegmentBsearchKey/new_key,_insert_at_end (0.00s) --- PASS: TestSegmentBsearchKey/new_key,_insert_second (0.00s) --- PASS: TestSegmentBsearchKey/new_key,_insert_in_middle (0.00s) --- PASS: TestSegmentBsearchKey/key_#1 (0.00s) --- PASS: TestSegmentBsearchKey/key_#2 (0.00s) --- PASS: TestSegmentBsearchKey/key_#3 (0.00s) --- PASS: TestSegmentBsearchKey/key_#4 (0.00s) --- PASS: TestSegmentBsearchKey/key_#5 (0.00s) === RUN TestSegmentSet --- PASS: TestSegmentSet (0.00s) === RUN TestSegmentSetOrdering --- PASS: TestSegmentSetOrdering (0.06s) === RUN TestSegmentGet === RUN TestSegmentGet/invalid_key === RUN TestSegmentGet/key_#1 === RUN TestSegmentGet/key_#2 === RUN TestSegmentGet/key_#3 --- PASS: TestSegmentGet (0.00s) --- PASS: TestSegmentGet/invalid_key (0.00s) --- PASS: TestSegmentGet/key_#1 (0.00s) --- PASS: TestSegmentGet/key_#2 (0.00s) --- PASS: TestSegmentGet/key_#3 (0.00s) === RUN TestSegmentDelete --- PASS: TestSegmentDelete (0.00s) === RUN TestSegmentForKey === RUN TestSegmentForKey/first_segment === RUN TestSegmentForKey/second_segment,_gapless === RUN TestSegmentForKey/gapped_segment --- PASS: TestSegmentForKey (0.00s) --- PASS: TestSegmentForKey/first_segment (0.00s) --- PASS: TestSegmentForKey/second_segment,_gapless (0.00s) --- PASS: TestSegmentForKey/gapped_segment (0.00s) === RUN TestAscendingVisit --- PASS: TestAscendingVisit (0.00s) === RUN TestRandomInsert --- PASS: TestRandomInsert (0.57s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/needle_map 13.664s === RUN TestMemoryUsage Each 15.32 Bytes Alloc = 24 MiB TotalAlloc = 114 MiB Sys = 50 MiB NumGC = 15 Taken = 2.008807433s Each 15.00 Bytes Alloc = 48 MiB TotalAlloc = 228 MiB Sys = 86 MiB NumGC = 19 Taken = 1.251622026s Each 14.89 Bytes Alloc = 72 MiB TotalAlloc = 341 MiB Sys = 127 MiB NumGC = 22 Taken = 1.070336533s Each 14.84 Bytes Alloc = 96 MiB TotalAlloc = 455 MiB Sys = 159 MiB NumGC = 24 Taken = 1.053686062s Each 14.81 Bytes Alloc = 119 MiB TotalAlloc = 568 MiB Sys = 203 MiB NumGC = 26 Taken = 1.041619459s Each 14.79 Bytes Alloc = 143 MiB TotalAlloc = 682 MiB Sys = 251 MiB NumGC = 27 Taken = 1.066945133s Each 14.77 Bytes Alloc = 167 MiB TotalAlloc = 795 MiB Sys = 275 MiB NumGC = 28 Taken = 1.126164653s Each 14.76 Bytes Alloc = 190 MiB TotalAlloc = 908 MiB Sys = 299 MiB NumGC = 29 Taken = 1.286202758s Each 14.75 Bytes Alloc = 214 MiB TotalAlloc = 1022 MiB Sys = 323 MiB NumGC = 30 Taken = 1.683340941s Each 14.74 Bytes Alloc = 238 MiB TotalAlloc = 1135 MiB Sys = 351 MiB NumGC = 31 Taken = 1.478943386s --- PASS: TestMemoryUsage (13.07s) === RUN TestSnowflakeSequencer I0720 21:30:15.429644 snowflake_sequencer.go:24 use snowflake seq id generator, nodeid:for_test hex_of_nodeid: 1 --- PASS: TestSnowflakeSequencer (0.05s) === RUN TestOverflow2 needle key: 150073 needle key: 150076 needle key: 150088 needle key: 150089 needle key: 150124 needle key: 150137 needle key: 150145 needle key: 150147 needle key: 150158 needle key: 150162 --- PASS: TestOverflow2 (0.00s) === RUN TestIssue52 key 10002 ok true 10002, 1250, 10002 key 10002 ok true 10002, 1250, 10002 --- PASS: TestIssue52 (0.00s) === RUN TestCompactMap --- PASS: TestCompactMap (0.04s) === RUN TestOverflow overflow[ 0 ]: 1 overflow[ 1 ]: 2 overflow[ 2 ]: 3 overflow[ 3 ]: 4 overflow[ 4 ]: 5 overflow[ 0 ]: 1 size -12 overflow[ 1 ]: 2 size 12 overflow[ 2 ]: 3 size 24 overflow[ 3 ]: 4 size -12 overflow[ 4 ]: 5 size 12 overflow[ 0 ]: 1 overflow[ 1 ]: 2 overflow[ 2 ]: 3 overflow[ 3 ]: 4 overflow[ 4 ]: 5 overflow[ 0 ]: 1 overflow[ 1 ]: 2 overflow[ 2 ]: 3 overflow[ 3 ]: 4 overflow[ 4 ]: 5 --- PASS: TestOverflow (0.00s) === RUN TestCompactSection_Get compact_map_test.go:204: 1574318345753513987 compact_map_test.go:215: 1574318350048481283 --- PASS: TestCompactSection_Get (1.02s) === RUN TestCompactSection_PutOutOfOrderItemBeyondLookBackWindow --- PASS: TestCompactSection_PutOutOfOrderItemBeyondLookBackWindow (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/needle_map/old 14.199s === RUN TestReplicaPlacementSerialDeserial --- PASS: TestReplicaPlacementSerialDeserial (0.00s) === RUN TestReplicaPlacementHasReplication === RUN TestReplicaPlacementHasReplication/empty_replica_placement === RUN TestReplicaPlacementHasReplication/no_replication === RUN TestReplicaPlacementHasReplication/same_rack_replication === RUN TestReplicaPlacementHasReplication/diff_rack_replication === RUN TestReplicaPlacementHasReplication/DC_replication === RUN TestReplicaPlacementHasReplication/full_replication --- PASS: TestReplicaPlacementHasReplication (0.00s) --- PASS: TestReplicaPlacementHasReplication/empty_replica_placement (0.00s) --- PASS: TestReplicaPlacementHasReplication/no_replication (0.00s) --- PASS: TestReplicaPlacementHasReplication/same_rack_replication (0.00s) --- PASS: TestReplicaPlacementHasReplication/diff_rack_replication (0.00s) --- PASS: TestReplicaPlacementHasReplication/DC_replication (0.00s) --- PASS: TestReplicaPlacementHasReplication/full_replication (0.00s) === RUN TestSuperBlockReadWrite --- PASS: TestSuperBlockReadWrite (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/super_block 0.025s ? github.com/seaweedfs/seaweedfs/weed/storage/types [no test files] ? github.com/seaweedfs/seaweedfs/weed/storage/volume_info [no test files] === RUN TestBuildUnionFromMultipleIndexDatabases --- PASS: TestBuildUnionFromMultipleIndexDatabases (0.00s) === RUN TestFindLargestReplica === RUN TestFindLargestReplica/single_replica === RUN TestFindLargestReplica/first_is_largest === RUN TestFindLargestReplica/last_is_largest === RUN TestFindLargestReplica/middle_is_largest === RUN TestFindLargestReplica/all_equal_-_pick_first --- PASS: TestFindLargestReplica (0.00s) --- PASS: TestFindLargestReplica/single_replica (0.00s) --- PASS: TestFindLargestReplica/first_is_largest (0.00s) --- PASS: TestFindLargestReplica/last_is_largest (0.00s) --- PASS: TestFindLargestReplica/middle_is_largest (0.00s) --- PASS: TestFindLargestReplica/all_equal_-_pick_first (0.00s) === RUN TestDeletedEntriesAreSkipped --- PASS: TestDeletedEntriesAreSkipped (0.00s) === RUN TestReplicaUnionBuilder_EmptyLocations --- PASS: TestReplicaUnionBuilder_EmptyLocations (0.00s) === RUN TestAvoidDuplicateCopies --- PASS: TestAvoidDuplicateCopies (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/storage/volume_replica 0.041s ? github.com/seaweedfs/seaweedfs/weed/telemetry [no test files] === RUN TestEcShardsAcrossMultipleDisksOnSameNode I0720 21:30:02.384492 node.go:471 weedfs adds child dc1 I0720 21:30:02.384782 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.384796 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:02.384803 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestEcShardsAcrossMultipleDisksOnSameNode (0.00s) === RUN TestEcShardsAfterRestartHeartbeat I0720 21:30:02.384856 node.go:471 weedfs adds child dc1 I0720 21:30:02.384860 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.384864 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:02.384868 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestEcShardsAfterRestartHeartbeat (0.00s) === RUN TestToDataNodeInfoReportsEmptyPhysicalDisks I0720 21:30:02.384886 node.go:471 weedfs adds child dc1 I0720 21:30:02.384889 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.384893 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:02.384916 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestToDataNodeInfoReportsEmptyPhysicalDisks (0.00s) === RUN TestToDataNodeInfoKeepsZeroCapacityDisk I0720 21:30:02.384945 node.go:471 weedfs adds child dc1 I0720 21:30:02.384948 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.384952 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:02.384963 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestToDataNodeInfoKeepsZeroCapacityDisk (0.00s) === RUN TestToDataNodeInfoFallsBackWhenNoCapacityReported I0720 21:30:02.384979 node.go:471 weedfs adds child dc1 I0720 21:30:02.384983 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.384988 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:02.384994 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestToDataNodeInfoFallsBackWhenNoCapacityReported (0.00s) === RUN TestRaceConditionStress I0720 21:30:02.385020 node.go:471 weedfs adds child dc1 I0720 21:30:02.385024 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.385030 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:02.385034 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:02.385047 node.go:471 weedfs:dc1:rack1 adds child server2 I0720 21:30:02.385052 node.go:471 weedfs:dc1:rack1:server2 adds child I0720 21:30:02.385056 node.go:471 weedfs:dc1:rack1 adds child server3 I0720 21:30:02.385061 node.go:471 weedfs:dc1:rack1:server3 adds child race_condition_stress_test.go:108: Test completed in 52.572887ms race_condition_stress_test.go:109: Successful allocations: 50 race_condition_stress_test.go:110: Failed allocations: 0 race_condition_stress_test.go:111: Total volumes created: 50 race_condition_stress_test.go:120: Server 1: 17 volumes (max: 40) race_condition_stress_test.go:120: Server 2: 17 volumes (max: 40) race_condition_stress_test.go:120: Server 3: 16 volumes (max: 40) race_condition_stress_test.go:148: Race condition test passed: Capacity limits respected with 50 concurrent requests --- PASS: TestRaceConditionStress (0.05s) === RUN TestCapacityJudgmentAccuracy I0720 21:30:02.437788 node.go:471 weedfs adds child dc1 I0720 21:30:02.437798 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.437805 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:02.437809 node.go:471 weedfs:dc1:rack1:server1 adds child race_condition_stress_test.go:226: Step 0: Volume count after update: 1 race_condition_stress_test.go:226: Step 1: Volume count after update: 2 race_condition_stress_test.go:226: Step 2: Volume count after update: 3 race_condition_stress_test.go:226: Step 3: Volume count after update: 4 race_condition_stress_test.go:226: Step 4: Volume count after update: 5 race_condition_stress_test.go:226: Step 5: Volume count after update: 6 race_condition_stress_test.go:226: Step 6: Volume count after update: 7 race_condition_stress_test.go:226: Step 7: Volume count after update: 8 race_condition_stress_test.go:226: Step 8: Volume count after update: 9 race_condition_stress_test.go:226: Step 9: Volume count after update: 10 I0720 21:30:02.437913 node.go:220 weedfs failed to pick 1 from 0 node candidates race_condition_stress_test.go:252: Capacity judgment accuracy test passed --- PASS: TestCapacityJudgmentAccuracy (0.00s) === RUN TestReservationSystemPerformance I0720 21:30:02.437932 node.go:471 weedfs adds child dc1 I0720 21:30:02.437936 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:02.437940 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:02.437944 node.go:471 weedfs:dc1:rack1:server1 adds child race_condition_stress_test.go:299: Performance: 1000 reservations in 2.277893ms (avg: 2.277µs per reservation) race_condition_stress_test.go:306: Performance test passed: 2.277µs per reservation --- PASS: TestReservationSystemPerformance (0.00s) === RUN TestDisk_GetEcShards_Race --- PASS: TestDisk_GetEcShards_Race (0.66s) === RUN TestClassifyReplicationError === RUN TestClassifyReplicationError/deadline_exceeded === RUN TestClassifyReplicationError/context_cancelled === RUN TestClassifyReplicationError/connection_refused === RUN TestClassifyReplicationError/generic_error === RUN TestClassifyReplicationError/nil_error --- PASS: TestClassifyReplicationError (0.00s) --- PASS: TestClassifyReplicationError/deadline_exceeded (0.00s) --- PASS: TestClassifyReplicationError/context_cancelled (0.00s) --- PASS: TestClassifyReplicationError/connection_refused (0.00s) --- PASS: TestClassifyReplicationError/generic_error (0.00s) --- PASS: TestClassifyReplicationError/nil_error (0.00s) === RUN TestDistributedOperationCancelsSiblingsOnFirstError --- PASS: TestDistributedOperationCancelsSiblingsOnFirstError (0.00s) === RUN TestDistributedOperationEmpty --- PASS: TestDistributedOperationEmpty (0.00s) === RUN TestRegisterEcShardsAdvancesMaxVolumeId I0720 21:30:03.101988 node.go:471 weedfs adds child dc1 I0720 21:30:03.101999 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.102016 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.102024 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestRegisterEcShardsAdvancesMaxVolumeId (0.00s) === RUN TestIncrementalSyncDataNodeEcShardsAdvancesMaxVolumeId I0720 21:30:03.102055 node.go:471 weedfs adds child dc1 I0720 21:30:03.102060 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.102064 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.102069 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestIncrementalSyncDataNodeEcShardsAdvancesMaxVolumeId (0.00s) === RUN TestCollectionVolumeStats I0720 21:30:03.102104 node.go:471 weedfs adds child dc1 I0720 21:30:03.102107 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.102111 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.102125 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child I0720 21:30:03.102130 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child ssd I0720 21:30:03.102167 volume_layout.go:345 volume 4 does not have enough copies I0720 21:30:03.102180 volume_layout.go:353 volume 4 remove from writable --- PASS: TestCollectionVolumeStats (0.00s) === RUN TestRemoveDataCenter data: map[dc1:map[rack1:map[server111:map[limit:3 volumes:[map[id:1 size:12312] map[id:2 size:12312] map[id:3 size:12312]]] server112:map[limit:10 volumes:[map[id:4 size:12312] map[id:5 size:12312] map[id:6 size:12312]]]] rack2:map[server121:map[limit:4 volumes:[map[id:4 size:12312] map[id:5 size:12312] map[id:6 size:12312]]] server122:map[limit:4 volumes:[]] server123:map[limit:5 volumes:[map[id:2 size:12312] map[id:3 size:12312] map[id:4 size:12312]]]]] dc2:map[] dc3:map[rack2:map[server321:map[limit:4 volumes:[map[id:1 size:12312] map[id:3 size:12312] map[id:5 size:12312]]]]]] I0720 21:30:03.102318 node.go:471 weedfs adds child dc2 I0720 21:30:03.102322 node.go:471 weedfs adds child dc3 I0720 21:30:03.102325 node.go:471 weedfs:dc3 adds child rack2 I0720 21:30:03.102329 node.go:471 weedfs:dc3:rack2 adds child server321 I0720 21:30:03.102333 node.go:471 weedfs:dc3:rack2:server321 adds child I0720 21:30:03.102339 node.go:471 weedfs adds child dc1 I0720 21:30:03.102342 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.102345 node.go:471 weedfs:dc1:rack1 adds child server111 I0720 21:30:03.102348 node.go:471 weedfs:dc1:rack1:server111 adds child I0720 21:30:03.102353 node.go:471 weedfs:dc1:rack1 adds child server112 I0720 21:30:03.102356 node.go:471 weedfs:dc1:rack1:server112 adds child I0720 21:30:03.102361 node.go:471 weedfs:dc1 adds child rack2 I0720 21:30:03.102365 node.go:471 weedfs:dc1:rack2 adds child server121 I0720 21:30:03.102368 node.go:471 weedfs:dc1:rack2:server121 adds child I0720 21:30:03.102373 node.go:471 weedfs:dc1:rack2 adds child server122 I0720 21:30:03.102377 node.go:471 weedfs:dc1:rack2:server122 adds child I0720 21:30:03.102389 node.go:471 weedfs:dc1:rack2 adds child server123 I0720 21:30:03.102394 node.go:471 weedfs:dc1:rack2:server123 adds child I0720 21:30:03.102401 node.go:492 weedfs removes dc2 I0720 21:30:03.102404 node.go:492 weedfs removes dc3 --- PASS: TestRemoveDataCenter (0.00s) === RUN TestHandlingVolumeServerHeartbeat I0720 21:30:03.102421 node.go:471 weedfs adds child dc1 I0720 21:30:03.102424 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.102428 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.102432 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child I0720 21:30:03.102437 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child ssd I0720 21:30:03.102523 data_node.go:94 Deleting volume id: 12 I0720 21:30:03.102528 data_node.go:94 Deleting volume id: 13 I0720 21:30:03.102531 data_node.go:94 Deleting volume id: 14 I0720 21:30:03.102533 data_node.go:94 Deleting volume id: 8 I0720 21:30:03.102536 data_node.go:94 Deleting volume id: 9 I0720 21:30:03.102539 data_node.go:94 Deleting volume id: 10 I0720 21:30:03.102541 data_node.go:94 Deleting volume id: 11 I0720 21:30:03.102544 data_node.go:94 Deleting volume id: 7 I0720 21:30:03.102549 topology.go:515 removing volume info: Id:12, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102573 volume_layout.go:345 volume 12 does not have enough copies I0720 21:30:03.102576 volume_layout.go:353 volume 12 remove from writable I0720 21:30:03.102580 volume_layout.go:818 Volume 12 becomes unwritable I0720 21:30:03.102584 topology.go:515 removing volume info: Id:13, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102589 volume_layout.go:345 volume 13 does not have enough copies I0720 21:30:03.102593 volume_layout.go:353 volume 13 remove from writable I0720 21:30:03.102595 volume_layout.go:818 Volume 13 becomes unwritable I0720 21:30:03.102598 topology.go:515 removing volume info: Id:14, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102602 volume_layout.go:345 volume 14 does not have enough copies I0720 21:30:03.102605 volume_layout.go:353 volume 14 remove from writable I0720 21:30:03.102609 volume_layout.go:818 Volume 14 becomes unwritable I0720 21:30:03.102612 topology.go:515 removing volume info: Id:8, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102616 volume_layout.go:345 volume 8 does not have enough copies I0720 21:30:03.102619 volume_layout.go:353 volume 8 remove from writable I0720 21:30:03.102621 volume_layout.go:818 Volume 8 becomes unwritable I0720 21:30:03.102624 topology.go:515 removing volume info: Id:9, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102628 volume_layout.go:345 volume 9 does not have enough copies I0720 21:30:03.102631 volume_layout.go:353 volume 9 remove from writable I0720 21:30:03.102633 volume_layout.go:818 Volume 9 becomes unwritable I0720 21:30:03.102636 topology.go:515 removing volume info: Id:10, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102640 volume_layout.go:345 volume 10 does not have enough copies I0720 21:30:03.102643 volume_layout.go:353 volume 10 remove from writable I0720 21:30:03.102646 volume_layout.go:818 Volume 10 becomes unwritable I0720 21:30:03.102648 topology.go:515 removing volume info: Id:11, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102654 volume_layout.go:345 volume 11 does not have enough copies I0720 21:30:03.102657 volume_layout.go:353 volume 11 remove from writable I0720 21:30:03.102659 volume_layout.go:818 Volume 11 becomes unwritable I0720 21:30:03.102662 topology.go:515 removing volume info: Id:7, Size:25432, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:2343, DeleteCount:345, DeletedByteCount:34524, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102667 volume_layout.go:345 volume 7 does not have enough copies I0720 21:30:03.102670 volume_layout.go:353 volume 7 remove from writable I0720 21:30:03.102672 volume_layout.go:818 Volume 7 becomes unwritable I0720 21:30:03.102681 topology.go:515 removing volume info: Id:3, Size:0, ReplicaPlacement:000, Collection:, Version:3, Ttl:, FileCount:0, DeleteCount:0, DeletedByteCount:0, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.102686 volume_layout.go:345 volume 3 does not have enough copies I0720 21:30:03.102799 volume_layout.go:353 volume 3 remove from writable I0720 21:30:03.102803 volume_layout.go:818 Volume 3 becomes unwritable after add volume id 1 after add volume id 2 after add volume id 3 after add volume id 4 after add volume id 5 after add volume id 6 after add writable volume id 1 after add writable volume id 2 after add writable volume id 4 after add writable volume id 5 after add writable volume id 6 after add writable volume id 3 I0720 21:30:03.102844 topology_event_handling.go:94 Removing Volume 1 from the dead volume server 127.0.0.1:34534 I0720 21:30:03.102850 volume_layout.go:870 Volume 1 has 0 replica, less than required 1 I0720 21:30:03.102855 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:03.102858 topology_event_handling.go:94 Removing Volume 2 from the dead volume server 127.0.0.1:34534 I0720 21:30:03.102862 volume_layout.go:870 Volume 2 has 0 replica, less than required 1 I0720 21:30:03.102864 volume_layout.go:818 Volume 2 becomes unwritable I0720 21:30:03.102867 topology_event_handling.go:94 Removing Volume 3 from the dead volume server 127.0.0.1:34534 I0720 21:30:03.102870 volume_layout.go:870 Volume 3 has 0 replica, less than required 1 I0720 21:30:03.102875 volume_layout.go:818 Volume 3 becomes unwritable I0720 21:30:03.102878 topology_event_handling.go:94 Removing Volume 4 from the dead volume server 127.0.0.1:34534 I0720 21:30:03.102882 volume_layout.go:870 Volume 4 has 0 replica, less than required 1 I0720 21:30:03.102885 volume_layout.go:818 Volume 4 becomes unwritable I0720 21:30:03.102888 topology_event_handling.go:94 Removing Volume 5 from the dead volume server 127.0.0.1:34534 I0720 21:30:03.102892 volume_layout.go:870 Volume 5 has 0 replica, less than required 1 I0720 21:30:03.102895 volume_layout.go:818 Volume 5 becomes unwritable I0720 21:30:03.102897 topology_event_handling.go:94 Removing Volume 6 from the dead volume server 127.0.0.1:34534 I0720 21:30:03.102901 volume_layout.go:870 Volume 6 has 0 replica, less than required 1 I0720 21:30:03.102913 volume_layout.go:818 Volume 6 becomes unwritable I0720 21:30:03.102930 node.go:492 weedfs:dc1:rack1 removes 127.0.0.1:34534 --- PASS: TestHandlingVolumeServerHeartbeat (0.00s) === RUN TestDataNodeToDataNodeInfo_IncludeEmptyDiskFromUsage --- PASS: TestDataNodeToDataNodeInfo_IncludeEmptyDiskFromUsage (0.00s) === RUN TestAddRemoveVolume I0720 21:30:03.102969 node.go:471 weedfs adds child dc1 I0720 21:30:03.102975 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.102979 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.102985 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child I0720 21:30:03.103002 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child ssd I0720 21:30:03.103011 topology.go:515 removing volume info: Id:1, Size:100, ReplicaPlacement:000, Collection:xcollection, Version:3, Ttl:, FileCount:123, DeleteCount:23, DeletedByteCount:45, ReadOnly:false, ModifiedAtSecond:0 from 127.0.0.1:34534 I0720 21:30:03.103017 volume_layout.go:345 volume 1 does not have enough copies I0720 21:30:03.103021 volume_layout.go:353 volume 1 remove from writable I0720 21:30:03.103023 volume_layout.go:818 Volume 1 becomes unwritable --- PASS: TestAddRemoveVolume (0.00s) === RUN TestVolumeReadOnlyStatusChange I0720 21:30:03.103042 node.go:471 weedfs adds child dc1 I0720 21:30:03.103046 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.103051 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.103055 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestVolumeReadOnlyStatusChange (0.00s) === RUN TestVolumeReadOnlyAndRemoteStatusChange I0720 21:30:03.103086 node.go:471 weedfs adds child dc1 I0720 21:30:03.103090 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.103095 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.103109 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child --- PASS: TestVolumeReadOnlyAndRemoteStatusChange (0.00s) === RUN TestListCollections I0720 21:30:03.103141 node.go:471 weedfs adds child dc1 I0720 21:30:03.103145 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.103150 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.103157 volume_layout.go:345 volume 1111 does not have enough copies I0720 21:30:03.103161 volume_layout.go:353 volume 1111 remove from writable I0720 21:30:03.103167 volume_layout.go:345 volume 2222 does not have enough copies I0720 21:30:03.103170 volume_layout.go:353 volume 2222 remove from writable I0720 21:30:03.103174 volume_layout.go:345 volume 3333 does not have enough copies I0720 21:30:03.103177 volume_layout.go:353 volume 3333 remove from writable === RUN TestListCollections/no_volume_types_selected === RUN TestListCollections/normal_volumes === RUN TestListCollections/EC_volumes === RUN TestListCollections/normal_+_EC_volumes --- PASS: TestListCollections (0.00s) --- PASS: TestListCollections/no_volume_types_selected (0.00s) --- PASS: TestListCollections/normal_volumes (0.00s) --- PASS: TestListCollections/EC_volumes (0.00s) --- PASS: TestListCollections/normal_+_EC_volumes (0.00s) === RUN TestDataNodeIdBasedIdentification I0720 21:30:03.103276 node.go:471 weedfs adds child dc1 I0720 21:30:03.105334 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105342 node.go:471 weedfs:dc1:rack1 adds child node-1 I0720 21:30:03.105347 node.go:471 weedfs:dc1:rack1:node-1 adds child I0720 21:30:03.105352 rack.go:68 DataNode node-1 address changed from 10.0.0.1:8080 to 10.0.0.2:8080 I0720 21:30:03.105357 node.go:471 weedfs:dc1:rack1 adds child node-2 I0720 21:30:03.105361 node.go:471 weedfs:dc1:rack1:node-2 adds child I0720 21:30:03.105364 node.go:471 weedfs:dc1:rack1 adds child 10.0.0.4:8080 I0720 21:30:03.105367 node.go:471 weedfs:dc1:rack1:10.0.0.4:8080 adds child I0720 21:30:03.105373 rack.go:100 Volume server 10.0.0.4:8080 transitioning id from 10.0.0.4:8080 to node-4-explicit I0720 21:30:03.105378 node.go:471 weedfs:dc1:rack1 adds child node-99 I0720 21:30:03.105382 node.go:471 weedfs:dc1:rack1:node-99 adds child --- PASS: TestDataNodeIdBasedIdentification (0.00s) === RUN TestLookupDataNodeByAddress I0720 21:30:03.105411 node.go:471 weedfs adds child dc1 I0720 21:30:03.105416 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105421 node.go:471 weedfs:dc1:rack1 adds child n1 I0720 21:30:03.105425 node.go:471 weedfs:dc1:rack1:n1 adds child I0720 21:30:03.105430 rack.go:68 DataNode n1 address changed from 10.1.2.3:8080 to 10.9.9.9:8080 I0720 21:30:03.105437 node.go:492 weedfs:dc1:rack1 removes n1 --- PASS: TestLookupDataNodeByAddress (0.00s) === RUN TestSyncDataNodeRegistrationReRegistersMissingVolume I0720 21:30:03.105455 node.go:471 weedfs adds child dc1 I0720 21:30:03.105458 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105463 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.105467 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child I0720 21:30:03.105485 volume_layout.go:870 Volume 18994 has 0 replica, less than required 1 I0720 21:30:03.105489 volume_layout.go:818 Volume 18994 becomes unwritable --- PASS: TestSyncDataNodeRegistrationReRegistersMissingVolume (0.00s) === RUN TestSetVolumeAvailableRepairsMissingVolume I0720 21:30:03.105508 node.go:471 weedfs adds child dc1 I0720 21:30:03.105512 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105515 node.go:471 weedfs:dc1:rack1 adds child 127.0.0.1:34534 I0720 21:30:03.105519 node.go:471 weedfs:dc1:rack1:127.0.0.1:34534 adds child I0720 21:30:03.105530 volume_layout.go:870 Volume 2640 has 0 replica, less than required 1 I0720 21:30:03.105533 volume_layout.go:818 Volume 2640 becomes unwritable --- PASS: TestSetVolumeAvailableRepairsMissingVolume (0.00s) === RUN TestVolumeGrowth_ReservationBasedAllocation I0720 21:30:03.105555 node.go:471 weedfs adds child dc1 I0720 21:30:03.105560 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105563 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.105567 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.105608 node.go:220 weedfs failed to pick 1 from 0 node candidates --- PASS: TestVolumeGrowth_ReservationBasedAllocation (0.00s) === RUN TestVolumeGrowth_ConcurrentAllocationPreventsRaceCondition I0720 21:30:03.105626 node.go:471 weedfs adds child dc1 I0720 21:30:03.105630 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105633 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.105636 node.go:471 weedfs:dc1:rack1:server1 adds child volume_growth_reservation_test.go:148: Request 9 succeeded, got reservation volume_growth_reservation_test.go:148: Request 0 succeeded, got reservation volume_growth_reservation_test.go:148: Request 1 succeeded, got reservation volume_growth_reservation_test.go:148: Request 2 succeeded, got reservation volume_growth_reservation_test.go:148: Request 3 succeeded, got reservation I0720 21:30:03.105726 node.go:220 weedfs failed to pick 1 from 0 node candidates volume_growth_reservation_test.go:145: Request 4 failed as expected: Not enough data nodes found! I0720 21:30:03.105736 node.go:220 weedfs failed to pick 1 from 0 node candidates volume_growth_reservation_test.go:145: Request 5 failed as expected: Not enough data nodes found! I0720 21:30:03.105747 node.go:220 weedfs failed to pick 1 from 0 node candidates volume_growth_reservation_test.go:145: Request 6 failed as expected: Not enough data nodes found! I0720 21:30:03.105756 node.go:220 weedfs failed to pick 1 from 0 node candidates volume_growth_reservation_test.go:145: Request 7 failed as expected: Not enough data nodes found! I0720 21:30:03.105764 node.go:220 weedfs failed to pick 1 from 0 node candidates volume_growth_reservation_test.go:145: Request 8 failed as expected: Not enough data nodes found! volume_growth_reservation_test.go:204: Concurrent test completed: 5 successes, 5 failures --- PASS: TestVolumeGrowth_ConcurrentAllocationPreventsRaceCondition (0.00s) === RUN TestVolumeGrowth_ReservationFailureRollback I0720 21:30:03.105792 node.go:471 weedfs adds child dc1 I0720 21:30:03.105797 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105800 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.105803 node.go:471 weedfs:dc1:rack1 adds child server2 I0720 21:30:03.105807 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.105811 node.go:471 weedfs:dc1:rack1:server2 adds child --- PASS: TestVolumeGrowth_ReservationFailureRollback (0.00s) === RUN TestVolumeGrowth_ReservationTimeout --- PASS: TestVolumeGrowth_ReservationTimeout (0.00s) === RUN TestFindEmptySlotsForOneVolume data: map[dc1:map[rack1:map[server111:map[limit:3 volumes:[map[id:1 size:12312] map[id:2 size:12312] map[id:3 size:12312]]] server112:map[limit:10 volumes:[map[id:4 size:12312] map[id:5 size:12312] map[id:6 size:12312]]]] rack2:map[server121:map[limit:4 volumes:[map[id:4 size:12312] map[id:5 size:12312] map[id:6 size:12312]]] server122:map[limit:4 volumes:[]] server123:map[limit:5 volumes:[map[id:2 size:12312] map[id:3 size:12312] map[id:4 size:12312]]]]] dc2:map[] dc3:map[rack2:map[server321:map[limit:4 volumes:[map[id:1 size:12312] map[id:3 size:12312] map[id:5 size:12312]]]]]] I0720 21:30:03.105926 node.go:471 weedfs adds child dc2 I0720 21:30:03.105929 node.go:471 weedfs adds child dc3 I0720 21:30:03.105932 node.go:471 weedfs:dc3 adds child rack2 I0720 21:30:03.105935 node.go:471 weedfs:dc3:rack2 adds child server321 I0720 21:30:03.105939 node.go:471 weedfs:dc3:rack2:server321 adds child I0720 21:30:03.105945 node.go:471 weedfs adds child dc1 I0720 21:30:03.105948 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.105951 node.go:471 weedfs:dc1:rack1 adds child server112 I0720 21:30:03.105954 node.go:471 weedfs:dc1:rack1:server112 adds child I0720 21:30:03.105961 node.go:471 weedfs:dc1:rack1 adds child server111 I0720 21:30:03.105964 node.go:471 weedfs:dc1:rack1:server111 adds child I0720 21:30:03.105969 node.go:471 weedfs:dc1 adds child rack2 I0720 21:30:03.105972 node.go:471 weedfs:dc1:rack2 adds child server123 I0720 21:30:03.105974 node.go:471 weedfs:dc1:rack2:server123 adds child I0720 21:30:03.105980 node.go:471 weedfs:dc1:rack2 adds child server121 I0720 21:30:03.105983 node.go:471 weedfs:dc1:rack2:server121 adds child I0720 21:30:03.105988 node.go:471 weedfs:dc1:rack2 adds child server122 I0720 21:30:03.105991 node.go:471 weedfs:dc1:rack2:server122 adds child assigned node : server122 assigned node : server123 assigned node : server121 --- PASS: TestFindEmptySlotsForOneVolume (0.00s) === RUN TestReplication011 data: map[dc1:map[rack1:map[server111:map[limit:300 volumes:[map[id:1 size:12312] map[id:2 size:12312] map[id:3 size:12312]]] server112:map[limit:300 volumes:[map[id:4 size:12312] map[id:5 size:12312] map[id:6 size:12312]]] server113:map[limit:300 volumes:[]] server114:map[limit:300 volumes:[]] server115:map[limit:300 volumes:[]] server116:map[limit:300 volumes:[]]] rack2:map[server121:map[limit:300 volumes:[map[id:4 size:12312] map[id:5 size:12312] map[id:6 size:12312]]] server122:map[limit:300 volumes:[]] server123:map[limit:300 volumes:[map[id:2 size:12312] map[id:3 size:12312] map[id:4 size:12312]]] server124:map[limit:300 volumes:[]] server125:map[limit:300 volumes:[]] server126:map[limit:300 volumes:[]]] rack3:map[server131:map[limit:300 volumes:[]] server132:map[limit:300 volumes:[]] server133:map[limit:300 volumes:[]] server134:map[limit:300 volumes:[]] server135:map[limit:300 volumes:[]] server136:map[limit:300 volumes:[]]]]] I0720 21:30:03.106100 node.go:471 weedfs adds child dc1 I0720 21:30:03.106103 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.106107 node.go:471 weedfs:dc1:rack1 adds child server116 I0720 21:30:03.106111 node.go:471 weedfs:dc1:rack1:server116 adds child I0720 21:30:03.106115 node.go:471 weedfs:dc1:rack1 adds child server111 I0720 21:30:03.106118 node.go:471 weedfs:dc1:rack1:server111 adds child I0720 21:30:03.106124 node.go:471 weedfs:dc1:rack1 adds child server112 I0720 21:30:03.106127 node.go:471 weedfs:dc1:rack1:server112 adds child I0720 21:30:03.106132 node.go:471 weedfs:dc1:rack1 adds child server113 I0720 21:30:03.106136 node.go:471 weedfs:dc1:rack1:server113 adds child I0720 21:30:03.106140 node.go:471 weedfs:dc1:rack1 adds child server114 I0720 21:30:03.106143 node.go:471 weedfs:dc1:rack1:server114 adds child I0720 21:30:03.106146 node.go:471 weedfs:dc1:rack1 adds child server115 I0720 21:30:03.106148 node.go:471 weedfs:dc1:rack1:server115 adds child I0720 21:30:03.106153 node.go:471 weedfs:dc1 adds child rack2 I0720 21:30:03.106156 node.go:471 weedfs:dc1:rack2 adds child server124 I0720 21:30:03.106158 node.go:471 weedfs:dc1:rack2:server124 adds child I0720 21:30:03.106161 node.go:471 weedfs:dc1:rack2 adds child server125 I0720 21:30:03.106165 node.go:471 weedfs:dc1:rack2:server125 adds child I0720 21:30:03.106168 node.go:471 weedfs:dc1:rack2 adds child server126 I0720 21:30:03.106171 node.go:471 weedfs:dc1:rack2:server126 adds child I0720 21:30:03.106174 node.go:471 weedfs:dc1:rack2 adds child server121 I0720 21:30:03.106177 node.go:471 weedfs:dc1:rack2:server121 adds child I0720 21:30:03.106182 node.go:471 weedfs:dc1:rack2 adds child server122 I0720 21:30:03.106184 node.go:471 weedfs:dc1:rack2:server122 adds child I0720 21:30:03.106188 node.go:471 weedfs:dc1:rack2 adds child server123 I0720 21:30:03.106191 node.go:471 weedfs:dc1:rack2:server123 adds child I0720 21:30:03.106196 node.go:471 weedfs:dc1 adds child rack3 I0720 21:30:03.106199 node.go:471 weedfs:dc1:rack3 adds child server134 I0720 21:30:03.106202 node.go:471 weedfs:dc1:rack3:server134 adds child I0720 21:30:03.106210 node.go:471 weedfs:dc1:rack3 adds child server135 I0720 21:30:03.106213 node.go:471 weedfs:dc1:rack3:server135 adds child I0720 21:30:03.106217 node.go:471 weedfs:dc1:rack3 adds child server136 I0720 21:30:03.106220 node.go:471 weedfs:dc1:rack3:server136 adds child I0720 21:30:03.106223 node.go:471 weedfs:dc1:rack3 adds child server131 I0720 21:30:03.106226 node.go:471 weedfs:dc1:rack3:server131 adds child I0720 21:30:03.106229 node.go:471 weedfs:dc1:rack3 adds child server132 I0720 21:30:03.106232 node.go:471 weedfs:dc1:rack3:server132 adds child I0720 21:30:03.106235 node.go:471 weedfs:dc1:rack3 adds child server133 I0720 21:30:03.106238 node.go:471 weedfs:dc1:rack3:server133 adds child assigned node : server132 assigned node : server131 assigned node : server113 --- PASS: TestReplication011 (0.00s) === RUN TestFindEmptySlotsForOneVolumeScheduleByWeight data: map[dc1:map[rack1:map[server111:map[limit:2000 volumes:[]]]] dc2:map[rack2:map[server222:map[limit:2000 volumes:[]]]] dc3:map[rack3:map[server333:map[limit:1000 volumes:[]]]] dc4:map[rack4:map[server444:map[limit:1000 volumes:[]]]] dc5:map[rack5:map[server555:map[limit:500 volumes:[]]]] dc6:map[rack6:map[server666:map[limit:500 volumes:[]]]]] I0720 21:30:03.106307 node.go:471 weedfs adds child dc3 I0720 21:30:03.106310 node.go:471 weedfs:dc3 adds child rack3 I0720 21:30:03.106314 node.go:471 weedfs:dc3:rack3 adds child server333 I0720 21:30:03.106317 node.go:471 weedfs:dc3:rack3:server333 adds child I0720 21:30:03.106321 node.go:471 weedfs adds child dc4 I0720 21:30:03.106324 node.go:471 weedfs:dc4 adds child rack4 I0720 21:30:03.106327 node.go:471 weedfs:dc4:rack4 adds child server444 I0720 21:30:03.106330 node.go:471 weedfs:dc4:rack4:server444 adds child I0720 21:30:03.106333 node.go:471 weedfs adds child dc5 I0720 21:30:03.106336 node.go:471 weedfs:dc5 adds child rack5 I0720 21:30:03.106339 node.go:471 weedfs:dc5:rack5 adds child server555 I0720 21:30:03.106342 node.go:471 weedfs:dc5:rack5:server555 adds child I0720 21:30:03.106349 node.go:471 weedfs adds child dc6 I0720 21:30:03.106352 node.go:471 weedfs:dc6 adds child rack6 I0720 21:30:03.106355 node.go:471 weedfs:dc6:rack6 adds child server666 I0720 21:30:03.106358 node.go:471 weedfs:dc6:rack6:server666 adds child I0720 21:30:03.106361 node.go:471 weedfs adds child dc1 I0720 21:30:03.106364 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.106367 node.go:471 weedfs:dc1:rack1 adds child server111 I0720 21:30:03.106370 node.go:471 weedfs:dc1:rack1:server111 adds child I0720 21:30:03.106375 node.go:471 weedfs adds child dc2 I0720 21:30:03.106377 node.go:471 weedfs:dc2 adds child rack2 I0720 21:30:03.106381 node.go:471 weedfs:dc2:rack2 adds child server222 I0720 21:30:03.106384 node.go:471 weedfs:dc2:rack2:server222 adds child server111 : 560 server666 : 168 server333 : 300 server444 : 295 server555 : 144 server222 : 533 --- PASS: TestFindEmptySlotsForOneVolumeScheduleByWeight (0.00s) === RUN TestPickForWrite data: map[dc1:map[rack1:map[serverdc111:map[ip:127.0.0.1 limit:100 volumes:[map[collection:test id:1 replication:001 size:12312] map[collection:test id:2 replication:100 size:12312] map[collection:test id:4 replication:100 size:12312] map[collection:test id:6 replication:010 size:12312]]]]] dc2:map[rack1:map[serverdc211:map[ip:127.0.0.2 limit:100 volumes:[map[collection:test id:2 replication:100 size:12312] map[collection:test id:3 replication:010 size:12312] map[collection:test id:5 replication:001 size:12312] map[collection:test id:6 replication:010 size:12312]]]]] dc3:map[rack1:map[serverdc311:map[ip:127.0.0.3 limit:100 volumes:[map[collection:test id:1 replication:001 size:12312] map[collection:test id:3 replication:010 size:12312] map[collection:test id:4 replication:100 size:12312] map[collection:test id:5 replication:001 size:12312]]]]]] I0720 21:30:03.109861 node.go:471 weedfs adds child dc1 I0720 21:30:03.109865 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.109869 node.go:471 weedfs:dc1:rack1 adds child serverdc111 I0720 21:30:03.109876 node.go:471 weedfs:dc1:rack1:serverdc111 adds child I0720 21:30:03.109886 node.go:471 weedfs adds child dc2 I0720 21:30:03.109888 node.go:471 weedfs:dc2 adds child rack1 I0720 21:30:03.109892 node.go:471 weedfs:dc2:rack1 adds child serverdc211 I0720 21:30:03.109896 volume_layout.go:818 Volume 2 becomes unwritable I0720 21:30:03.109899 node.go:471 weedfs:dc2:rack1:serverdc211 adds child I0720 21:30:03.109906 volume_layout.go:818 Volume 6 becomes unwritable I0720 21:30:03.109909 node.go:471 weedfs adds child dc3 I0720 21:30:03.109912 node.go:471 weedfs:dc3 adds child rack1 I0720 21:30:03.109915 node.go:471 weedfs:dc3:rack1 adds child serverdc311 I0720 21:30:03.109918 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:03.109921 node.go:471 weedfs:dc3:rack1:serverdc311 adds child I0720 21:30:03.109925 volume_layout.go:818 Volume 3 becomes unwritable I0720 21:30:03.109928 volume_layout.go:818 Volume 4 becomes unwritable I0720 21:30:03.109931 volume_layout.go:818 Volume 5 becomes unwritable --- PASS: TestPickForWrite (0.00s) === RUN TestPickNodesByWeightPrefersDistinctHosts I0720 21:30:03.109990 node.go:471 rack1 adds child s1a I0720 21:30:03.109994 node.go:471 rack1:s1a adds child I0720 21:30:03.109997 node.go:471 rack1 adds child s1b I0720 21:30:03.109999 node.go:471 rack1:s1b adds child I0720 21:30:03.110002 node.go:471 rack1 adds child s2a I0720 21:30:03.110005 node.go:471 rack1:s2a adds child I0720 21:30:03.110008 node.go:471 rack1 adds child s2b I0720 21:30:03.110010 node.go:471 rack1:s2b adds child --- PASS: TestPickNodesByWeightPrefersDistinctHosts (0.00s) === RUN TestSetVolumeCapacityFullStampsFullSinceAndRecovers I0720 21:30:03.110098 node.go:471 weedfs adds child dc1 I0720 21:30:03.110101 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110107 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110112 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.110136 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:03.110139 volume_layout.go:945 Volume 1 reaches full capacity. I0720 21:30:03.110146 volume_layout.go:300 Volume 1 recovered to writable (effective=4000, reported=4000, limit=10000). --- PASS: TestSetVolumeCapacityFullStampsFullSinceAndRecovers (0.00s) === RUN TestSetVolumeAvailableRestoresActiveCountForCapacityFullVolume I0720 21:30:03.110181 node.go:471 weedfs adds child dc1 I0720 21:30:03.110184 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110187 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110193 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.110197 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:03.110199 volume_layout.go:945 Volume 1 reaches full capacity. --- PASS: TestSetVolumeAvailableRestoresActiveCountForCapacityFullVolume (0.00s) === RUN TestGetPendingSize I0720 21:30:03.110233 node.go:471 weedfs adds child dc1 I0720 21:30:03.110236 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110240 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110245 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestGetPendingSize (0.00s) === RUN TestGetPendingSize_CompactionResets I0720 21:30:03.110294 node.go:471 weedfs adds child dc1 I0720 21:30:03.110297 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110301 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110306 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestGetPendingSize_CompactionResets (0.00s) === RUN TestDrainAndRemoveFromWritable_NoPending I0720 21:30:03.110339 node.go:471 weedfs adds child dc1 I0720 21:30:03.110343 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110346 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110351 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.110356 volume_layout.go:818 Volume 1 becomes unwritable --- PASS: TestDrainAndRemoveFromWritable_NoPending (0.00s) === RUN TestDrainAndRemoveFromWritable_WithPending I0720 21:30:03.110387 node.go:471 weedfs adds child dc1 I0720 21:30:03.110391 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110394 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110399 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.110407 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:03.110410 volume_layout.go:404 Volume 1 reaches full capacity (effective=2098151, limit=10000). --- PASS: TestDrainAndRemoveFromWritable_WithPending (0.00s) === RUN TestDrainAndRemoveFromWritable_DecaysViaConcurrentHeartbeat I0720 21:30:03.110461 node.go:471 weedfs adds child dc1 I0720 21:30:03.110465 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:03.110469 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:03.110474 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:03.110479 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:03.110482 volume_layout.go:404 Volume 1 reaches full capacity (effective=104858600, limit=10000). I0720 21:30:03.610696 volume_layout.go:960 Volume 1 becomes crowded --- PASS: TestDrainAndRemoveFromWritable_DecaysViaConcurrentHeartbeat (5.01s) === RUN TestSetVolumeReadOnly_PreservesPending I0720 21:30:08.116630 node.go:471 weedfs adds child dc1 I0720 21:30:08.116639 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.116646 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.116659 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.116668 volume_layout.go:818 Volume 1 becomes unwritable --- PASS: TestSetVolumeReadOnly_PreservesPending (0.00s) === RUN TestShouldGrowVolumesByDcAndRack_Issue8986 data: map[dc_edge:map[rack1:map[server1:map[ip:10.0.0.1 limit:30 volumes:[map[id:1 replication:010 size:12312] map[id:2 replication:010 size:12312]]]] rack2:map[server2:map[ip:10.0.0.2 limit:30 volumes:[map[id:1 replication:010 size:12312] map[id:2 replication:010 size:12312]]]] rack3:map[server3:map[ip:10.0.0.3 limit:30 volumes:[]]]]] I0720 21:30:08.116735 node.go:471 weedfs adds child dc_edge I0720 21:30:08.116739 node.go:471 weedfs:dc_edge adds child rack3 I0720 21:30:08.116742 node.go:471 weedfs:dc_edge:rack3 adds child server3 I0720 21:30:08.116745 node.go:471 weedfs:dc_edge:rack3:server3 adds child I0720 21:30:08.116749 node.go:471 weedfs:dc_edge adds child rack1 I0720 21:30:08.116753 node.go:471 weedfs:dc_edge:rack1 adds child server1 I0720 21:30:08.116758 node.go:471 weedfs:dc_edge:rack1:server1 adds child I0720 21:30:08.116763 node.go:471 weedfs:dc_edge adds child rack2 I0720 21:30:08.116766 node.go:471 weedfs:dc_edge:rack2 adds child server2 I0720 21:30:08.116769 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:08.116772 node.go:471 weedfs:dc_edge:rack2:server2 adds child I0720 21:30:08.116776 volume_layout.go:818 Volume 2 becomes unwritable --- PASS: TestShouldGrowVolumesByDcAndRack_Issue8986 (0.00s) === RUN TestPlanRackAwareGrowth_Issue9832 I0720 21:30:08.116832 node.go:471 weedfs adds child datacenter1 I0720 21:30:08.116838 node.go:471 weedfs:datacenter1 adds child rack1 I0720 21:30:08.116841 node.go:471 weedfs:datacenter1:rack1 adds child node-a I0720 21:30:08.116845 node.go:471 weedfs:datacenter1:rack1:node-a adds child I0720 21:30:08.116850 node.go:471 weedfs:datacenter1 adds child rack2 I0720 21:30:08.116853 node.go:471 weedfs:datacenter1:rack2 adds child node-b I0720 21:30:08.116856 volume_layout.go:818 Volume 12 becomes unwritable I0720 21:30:08.116859 node.go:471 weedfs:datacenter1:rack2:node-b adds child --- PASS: TestPlanRackAwareGrowth_Issue9832 (0.00s) === RUN TestPlanRackAwareGrowth_DefaultStepNotMultiplied I0720 21:30:08.116889 node.go:471 weedfs adds child datacenter1 I0720 21:30:08.116894 node.go:471 weedfs:datacenter1 adds child rack1 I0720 21:30:08.116897 node.go:471 weedfs:datacenter1:rack1 adds child node-a I0720 21:30:08.116902 node.go:471 weedfs:datacenter1:rack1:node-a adds child I0720 21:30:08.116906 node.go:471 weedfs:datacenter1 adds child rack2 I0720 21:30:08.116908 node.go:471 weedfs:datacenter1:rack2 adds child node-b I0720 21:30:08.116911 volume_layout.go:818 Volume 12 becomes unwritable I0720 21:30:08.116914 node.go:471 weedfs:datacenter1:rack2:node-b adds child --- PASS: TestPlanRackAwareGrowth_DefaultStepNotMultiplied (0.00s) === RUN TestPlanRackAwareGrowth_NotCrowdedNoGrowth I0720 21:30:08.116942 node.go:471 weedfs adds child datacenter1 I0720 21:30:08.116946 node.go:471 weedfs:datacenter1 adds child rack1 I0720 21:30:08.116949 node.go:471 weedfs:datacenter1:rack1 adds child node-a I0720 21:30:08.116961 node.go:471 weedfs:datacenter1:rack1:node-a adds child I0720 21:30:08.116965 node.go:471 weedfs:datacenter1 adds child rack2 I0720 21:30:08.116968 node.go:471 weedfs:datacenter1:rack2 adds child node-b I0720 21:30:08.116971 volume_layout.go:818 Volume 12 becomes unwritable I0720 21:30:08.116973 node.go:471 weedfs:datacenter1:rack2:node-b adds child --- PASS: TestPlanRackAwareGrowth_NotCrowdedNoGrowth (0.00s) === RUN TestPlanRackAwareGrowth_PerRackForNonRackSpanning I0720 21:30:08.117010 node.go:471 weedfs adds child datacenter1 I0720 21:30:08.117016 node.go:471 weedfs:datacenter1 adds child rack1 I0720 21:30:08.117019 node.go:471 weedfs:datacenter1:rack1 adds child node-a I0720 21:30:08.117023 node.go:471 weedfs:datacenter1:rack1:node-a adds child I0720 21:30:08.117027 node.go:471 weedfs:datacenter1 adds child rack2 I0720 21:30:08.117029 node.go:471 weedfs:datacenter1:rack2 adds child node-b I0720 21:30:08.117031 node.go:471 weedfs:datacenter1:rack2:node-b adds child --- PASS: TestPlanRackAwareGrowth_PerRackForNonRackSpanning (0.00s) === RUN TestPlanRackAwareGrowth_EvenDistributionAcrossUnevenDCs I0720 21:30:08.117064 node.go:471 weedfs adds child dc1 I0720 21:30:08.117067 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.117069 node.go:471 weedfs:dc1:rack1 adds child node-a I0720 21:30:08.117073 node.go:471 weedfs:dc1:rack1:node-a adds child I0720 21:30:08.117076 node.go:471 weedfs:dc1 adds child rack2 I0720 21:30:08.117079 node.go:471 weedfs:dc1:rack2 adds child node-b I0720 21:30:08.117082 node.go:471 weedfs:dc1:rack2:node-b adds child I0720 21:30:08.117086 node.go:471 weedfs adds child dc2 I0720 21:30:08.117088 node.go:471 weedfs:dc2 adds child rack3 I0720 21:30:08.117091 node.go:471 weedfs:dc2:rack3 adds child node-c I0720 21:30:08.117094 node.go:471 weedfs:dc2:rack3:node-c adds child I0720 21:30:08.117098 node.go:471 weedfs:dc2 adds child rack4 I0720 21:30:08.117100 node.go:471 weedfs:dc2:rack4 adds child node-d I0720 21:30:08.117103 node.go:471 weedfs:dc2:rack4:node-d adds child I0720 21:30:08.117106 node.go:471 weedfs:dc2 adds child rack5 I0720 21:30:08.117109 node.go:471 weedfs:dc2:rack5 adds child node-e I0720 21:30:08.117112 node.go:471 weedfs:dc2:rack5:node-e adds child --- PASS: TestPlanRackAwareGrowth_EvenDistributionAcrossUnevenDCs (0.00s) === RUN TestPickForWriteWeightedDistribution I0720 21:30:08.117142 node.go:471 weedfs adds child dc1 I0720 21:30:08.117144 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.117147 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.117150 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestPickForWriteWeightedDistribution (0.00s) === RUN TestPickForWriteWithPendingSize I0720 21:30:08.121674 node.go:471 weedfs adds child dc1 I0720 21:30:08.121679 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.121684 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.121718 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestPickForWriteWithPendingSize (0.00s) === RUN TestPickForWriteSingleWritable I0720 21:30:08.122497 node.go:471 weedfs adds child dc1 I0720 21:30:08.122501 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.122504 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.122509 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestPickForWriteSingleWritable (0.00s) === RUN TestPickForWriteAllNearFull I0720 21:30:08.122532 node.go:471 weedfs adds child dc1 I0720 21:30:08.122534 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.122536 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.122539 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestPickForWriteAllNearFull (0.00s) === RUN TestPickForWriteConstrainedWeighted I0720 21:30:08.122564 node.go:471 weedfs adds child dc1 I0720 21:30:08.122566 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.122568 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.122571 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.122575 node.go:471 weedfs:dc1 adds child rack2 I0720 21:30:08.122577 node.go:471 weedfs:dc1:rack2 adds child server2 I0720 21:30:08.122580 node.go:471 weedfs:dc1:rack2:server2 adds child --- PASS: TestPickForWriteConstrainedWeighted (0.00s) === RUN TestRecordAssignMarksCrowded I0720 21:30:08.124608 node.go:471 weedfs adds child dc1 I0720 21:30:08.124611 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.124614 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.124617 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.124621 volume_layout.go:960 Volume 1 becomes crowded --- PASS: TestRecordAssignMarksCrowded (0.00s) === RUN TestRecordAssignReachingCapacityRemovesFromWritable I0720 21:30:08.124637 node.go:471 weedfs adds child dc1 I0720 21:30:08.124639 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.124641 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.124644 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.124649 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:08.124652 volume_layout.go:404 Volume 1 reaches full capacity (effective=10000, limit=10000). --- PASS: TestRecordAssignReachingCapacityRemovesFromWritable (0.00s) === RUN TestUpdateVolumeSizeRecoversEagerlyRemovedVolume I0720 21:30:08.124668 node.go:471 weedfs adds child dc1 I0720 21:30:08.124670 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.124672 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.124676 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.124679 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:08.124681 volume_layout.go:404 Volume 1 reaches full capacity (effective=10000, limit=10000). I0720 21:30:08.124685 volume_layout.go:300 Volume 1 recovered to writable (effective=4375, reported=4000, limit=10000). --- PASS: TestUpdateVolumeSizeRecoversEagerlyRemovedVolume (0.00s) === RUN TestUpdateVolumeSizeNoRecoveryWhenDiskStillOversized I0720 21:30:08.124700 node.go:471 weedfs adds child dc1 I0720 21:30:08.124702 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.124704 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.124706 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.124709 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:08.124711 volume_layout.go:404 Volume 1 reaches full capacity (effective=10000, limit=10000). I0720 21:30:08.124713 volume_layout.go:960 Volume 1 becomes crowded --- PASS: TestUpdateVolumeSizeNoRecoveryWhenDiskStillOversized (0.00s) === RUN TestHeartbeatDecaysPendingSize I0720 21:30:08.124728 node.go:471 weedfs adds child dc1 I0720 21:30:08.124730 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.124732 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.124735 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestHeartbeatDecaysPendingSize (0.00s) === RUN TestHeartbeatDecayDedupReplicas I0720 21:30:08.125320 node.go:471 weedfs adds child dc1 I0720 21:30:08.125322 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.125324 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.125327 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.125330 node.go:471 weedfs:dc1:rack1 adds child server2 I0720 21:30:08.125332 volume_layout.go:818 Volume 1 becomes unwritable I0720 21:30:08.125334 node.go:471 weedfs:dc1:rack1:server2 adds child --- PASS: TestHeartbeatDecayDedupReplicas (0.00s) === RUN TestUpdateVolumeSize_DecaysEvenWhenReportedSizeUnchanged I0720 21:30:08.125349 node.go:471 weedfs adds child dc1 I0720 21:30:08.125351 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.125353 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.125356 node.go:471 weedfs:dc1:rack1:server1 adds child --- PASS: TestUpdateVolumeSize_DecaysEvenWhenReportedSizeUnchanged (0.00s) === RUN TestShouldGrowVolumesByDcAndRack_WithPendingSize I0720 21:30:08.125371 node.go:471 weedfs adds child dc1 I0720 21:30:08.125373 node.go:471 weedfs:dc1 adds child rack1 I0720 21:30:08.125375 node.go:471 weedfs:dc1:rack1 adds child server1 I0720 21:30:08.125378 node.go:471 weedfs:dc1:rack1:server1 adds child I0720 21:30:08.125381 volume_layout.go:960 Volume 1 becomes crowded --- PASS: TestShouldGrowVolumesByDcAndRack_WithPendingSize (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/topology 5.776s === RUN TestVolumeDensity --- PASS: TestVolumeDensity (0.00s) === RUN TestDensityRatio --- PASS: TestDensityRatio (0.00s) === RUN TestDensityNextRatio --- PASS: TestDensityNextRatio (0.00s) === RUN TestDensityRatio_OverfullSortsFullest --- PASS: TestDensityRatio_OverfullSortsFullest (0.00s) === RUN TestDiskTooFullAfter === RUN TestDiskTooFullAfter/disabled_at_0 === RUN TestDiskTooFullAfter/disabled_at_100 === RUN TestDiskTooFullAfter/not_reported_(total_0) === RUN TestDiskTooFullAfter/below_mark === RUN TestDiskTooFullAfter/at_mark === RUN TestDiskTooFullAfter/above_mark === RUN TestDiskTooFullAfter/below_but_crosses_with_incoming === RUN TestDiskTooFullAfter/below_and_stays_below_with_incoming === RUN TestDiskTooFullAfter/free_exceeds_total_(defensive) --- PASS: TestDiskTooFullAfter (0.00s) --- PASS: TestDiskTooFullAfter/disabled_at_0 (0.00s) --- PASS: TestDiskTooFullAfter/disabled_at_100 (0.00s) --- PASS: TestDiskTooFullAfter/not_reported_(total_0) (0.00s) --- PASS: TestDiskTooFullAfter/below_mark (0.00s) --- PASS: TestDiskTooFullAfter/at_mark (0.00s) --- PASS: TestDiskTooFullAfter/above_mark (0.00s) --- PASS: TestDiskTooFullAfter/below_but_crosses_with_incoming (0.00s) --- PASS: TestDiskTooFullAfter/below_and_stays_below_with_incoming (0.00s) --- PASS: TestDiskTooFullAfter/free_exceeds_total_(defensive) (0.00s) === RUN TestIsGoodMove_OverReplicatedSourceKeepsOtherReplica --- PASS: TestIsGoodMove_OverReplicatedSourceKeepsOtherReplica (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/topology/balancer 0.036s === RUN TestAsyncBatchWorkerProcessesInOrder --- PASS: TestAsyncBatchWorkerProcessesInOrder (0.00s) === RUN TestAsyncBatchWorkerDrainWaitsForEnqueued --- PASS: TestAsyncBatchWorkerDrainWaitsForEnqueued (0.00s) === RUN TestAsyncBatchWorkerEnqueueNeverBlocks --- PASS: TestAsyncBatchWorkerEnqueueNeverBlocks (0.00s) === RUN TestAsyncBatchWorkerDrainCompletesDuringShutdown --- PASS: TestAsyncBatchWorkerDrainCompletesDuringShutdown (0.10s) === RUN TestAsyncBatchWorkerShutdownDrainsThenStops --- PASS: TestAsyncBatchWorkerShutdownDrainsThenStops (0.00s) === RUN TestExtractBucketPath === RUN TestExtractBucketPath/child_paths_return_bucket === RUN TestExtractBucketPath/bucket_root_without_child_fails_when_required === RUN TestExtractBucketPath/bucket_root_allowed_when_not_required === RUN TestExtractBucketPath/path_outside_buckets_fails === RUN TestExtractBucketPath/trailing_slash_on_base_is_normalized --- PASS: TestExtractBucketPath (0.00s) --- PASS: TestExtractBucketPath/child_paths_return_bucket (0.00s) --- PASS: TestExtractBucketPath/bucket_root_without_child_fails_when_required (0.00s) --- PASS: TestExtractBucketPath/bucket_root_allowed_when_not_required (0.00s) --- PASS: TestExtractBucketPath/path_outside_buckets_fails (0.00s) --- PASS: TestExtractBucketPath/trailing_slash_on_base_is_normalized (0.00s) === RUN TestRandomUint64ReturnsFullRandomValue --- PASS: TestRandomUint64ReturnsFullRandomValue (0.00s) === RUN TestRandomUint64PanicsOnRandomReadError --- PASS: TestRandomUint64PanicsOnRandomReadError (0.00s) === RUN TestByteParsing --- PASS: TestByteParsing (0.00s) === RUN TestSameAsJavaImplementation Now we need to generate a 256-bit key for AES 256 GCM --- PASS: TestSameAsJavaImplementation (0.00s) === RUN TestDropOSPageCache --- PASS: TestDropOSPageCache (0.00s) === RUN TestToShortFileName --- PASS: TestToShortFileName (0.00s) === RUN TestResolvePath === RUN TestResolvePath/empty === RUN TestResolvePath/absolute === RUN TestResolvePath/relative === RUN TestResolvePath/tilde_mid_path_is_literal === RUN TestResolvePath/bare_tilde === RUN TestResolvePath/tilde_slash === RUN TestResolvePath/tilde_with_subpath === RUN TestResolvePath/tilde_with_current_user === RUN TestResolvePath/tilde_with_current_user_and_subpath === RUN TestResolvePath/tilde_unknown_user_falls_back_to_literal === RUN TestResolvePath/tilde_with_native_separator === RUN TestResolvePath/tilde_user_with_native_separator --- PASS: TestResolvePath (0.00s) --- PASS: TestResolvePath/empty (0.00s) --- PASS: TestResolvePath/absolute (0.00s) --- PASS: TestResolvePath/relative (0.00s) --- PASS: TestResolvePath/tilde_mid_path_is_literal (0.00s) --- PASS: TestResolvePath/bare_tilde (0.00s) --- PASS: TestResolvePath/tilde_slash (0.00s) --- PASS: TestResolvePath/tilde_with_subpath (0.00s) --- PASS: TestResolvePath/tilde_with_current_user (0.00s) --- PASS: TestResolvePath/tilde_with_current_user_and_subpath (0.00s) --- PASS: TestResolvePath/tilde_unknown_user_falls_back_to_literal (0.00s) --- PASS: TestResolvePath/tilde_with_native_separator (0.00s) --- PASS: TestResolvePath/tilde_user_with_native_separator (0.00s) === RUN TestResolveCommaSeparatedPaths === RUN TestResolveCommaSeparatedPaths/empty === RUN TestResolveCommaSeparatedPaths/single_absolute === RUN TestResolveCommaSeparatedPaths/single_tilde === RUN TestResolveCommaSeparatedPaths/two_absolutes === RUN TestResolveCommaSeparatedPaths/two_tildes === RUN TestResolveCommaSeparatedPaths/mixed === RUN TestResolveCommaSeparatedPaths/no_tilde_fast_path --- PASS: TestResolveCommaSeparatedPaths (0.00s) --- PASS: TestResolveCommaSeparatedPaths/empty (0.00s) --- PASS: TestResolveCommaSeparatedPaths/single_absolute (0.00s) --- PASS: TestResolveCommaSeparatedPaths/single_tilde (0.00s) --- PASS: TestResolveCommaSeparatedPaths/two_absolutes (0.00s) --- PASS: TestResolveCommaSeparatedPaths/two_tildes (0.00s) --- PASS: TestResolveCommaSeparatedPaths/mixed (0.00s) --- PASS: TestResolveCommaSeparatedPaths/no_tilde_fast_path (0.00s) === RUN TestSanitizeUTF8Name_ValidPassThrough --- PASS: TestSanitizeUTF8Name_ValidPassThrough (0.00s) === RUN TestSanitizeUTF8Name_InvalidBytes --- PASS: TestSanitizeUTF8Name_InvalidBytes (0.00s) === RUN TestFullPathSanitized_WholePath --- PASS: TestFullPathSanitized_WholePath (0.00s) === RUN TestFullPathDirAndName_OnlyNameSanitized --- PASS: TestFullPathDirAndName_OnlyNameSanitized (0.00s) === RUN TestHumanReadableIntsMax --- PASS: TestHumanReadableIntsMax (0.00s) === RUN TestHumanReadableInts --- PASS: TestHumanReadableInts (0.00s) === RUN TestOrderedLock ActiveLock 1 acquired lock 1 ActiveLock 1 released lock 1 ActiveLock 2 acquired lock 0 ActiveLock 3 acquired lock 0 ActiveLock 4 acquired lock 0 ActiveLock 4 released lock 0 ActiveLock 5 acquired lock 0 ActiveLock 2 released lock 0 ActiveLock 5 released lock 0 ActiveLock 3 released lock 0 ActiveLock 6 acquired lock 1 ActiveLock 6 released lock 1 ActiveLock 7 acquired lock 0 ActiveLock 8 acquired lock 0 ActiveLock 9 acquired lock 0 ActiveLock 10 acquired lock 0 ActiveLock 8 released lock 0 ActiveLock 7 released lock 0 ActiveLock 9 released lock 0 ActiveLock 10 released lock 0 ActiveLock 11 acquired lock 1 ActiveLock 11 released lock 1 ActiveLock 12 acquired lock 0 ActiveLock 13 acquired lock 0 ActiveLock 14 acquired lock 0 ActiveLock 15 acquired lock 0 ActiveLock 15 released lock 0 ActiveLock 14 released lock 0 ActiveLock 12 released lock 0 ActiveLock 13 released lock 0 ActiveLock 16 acquired lock 1 ActiveLock 16 released lock 1 ActiveLock 17 acquired lock 0 ActiveLock 18 acquired lock 0 ActiveLock 19 acquired lock 0 ActiveLock 20 acquired lock 0 ActiveLock 17 released lock 0 ActiveLock 20 released lock 0 ActiveLock 18 released lock 0 ActiveLock 19 released lock 0 ActiveLock 21 acquired lock 1 ActiveLock 21 released lock 1 ActiveLock 22 acquired lock 0 ActiveLock 23 acquired lock 0 ActiveLock 24 acquired lock 0 ActiveLock 24 released lock 0 ActiveLock 25 acquired lock 0 ActiveLock 25 released lock 0 ActiveLock 23 released lock 0 ActiveLock 22 released lock 0 ActiveLock 26 acquired lock 1 ActiveLock 26 released lock 1 ActiveLock 27 acquired lock 0 ActiveLock 28 acquired lock 0 ActiveLock 29 acquired lock 0 ActiveLock 30 acquired lock 0 ActiveLock 30 released lock 0 ActiveLock 29 released lock 0 ActiveLock 27 released lock 0 ActiveLock 28 released lock 0 ActiveLock 31 acquired lock 1 ActiveLock 31 released lock 1 ActiveLock 32 acquired lock 0 ActiveLock 33 acquired lock 0 ActiveLock 33 released lock 0 ActiveLock 34 acquired lock 0 ActiveLock 35 acquired lock 0 ActiveLock 35 released lock 0 ActiveLock 32 released lock 0 ActiveLock 34 released lock 0 ActiveLock 36 acquired lock 1 ActiveLock 36 released lock 1 ActiveLock 37 acquired lock 0 ActiveLock 38 acquired lock 0 ActiveLock 39 acquired lock 0 ActiveLock 40 acquired lock 0 ActiveLock 40 released lock 0 ActiveLock 38 released lock 0 ActiveLock 39 released lock 0 ActiveLock 37 released lock 0 ActiveLock 41 acquired lock 1 ActiveLock 41 released lock 1 ActiveLock 42 acquired lock 0 ActiveLock 43 acquired lock 0 ActiveLock 44 acquired lock 0 ActiveLock 45 acquired lock 0 ActiveLock 44 released lock 0 ActiveLock 43 released lock 0 ActiveLock 45 released lock 0 ActiveLock 42 released lock 0 ActiveLock 46 acquired lock 1 ActiveLock 46 released lock 1 ActiveLock 47 acquired lock 0 ActiveLock 48 acquired lock 0 ActiveLock 49 acquired lock 0 ActiveLock 50 acquired lock 0 ActiveLock 49 released lock 0 ActiveLock 50 released lock 0 ActiveLock 48 released lock 0 ActiveLock 47 released lock 0 --- PASS: TestOrderedLock (1.10s) === RUN TestShouldWaitForSharedLock --- PASS: TestShouldWaitForSharedLock (0.00s) === RUN TestShouldWaitForExclusiveLock --- PASS: TestShouldWaitForExclusiveLock (0.00s) === RUN TestParseMinFreeSpace --- PASS: TestParseMinFreeSpace (0.00s) === RUN TestSetOutboundLocalIP === RUN TestSetOutboundLocalIP/ipv4 === RUN TestSetOutboundLocalIP/ipv6 === RUN TestSetOutboundLocalIP/empty === RUN TestSetOutboundLocalIP/wildcard_v4 === RUN TestSetOutboundLocalIP/wildcard_v6 === RUN TestSetOutboundLocalIP/garbage --- PASS: TestSetOutboundLocalIP (0.00s) --- PASS: TestSetOutboundLocalIP/ipv4 (0.00s) --- PASS: TestSetOutboundLocalIP/ipv6 (0.00s) --- PASS: TestSetOutboundLocalIP/empty (0.00s) --- PASS: TestSetOutboundLocalIP/wildcard_v4 (0.00s) --- PASS: TestSetOutboundLocalIP/wildcard_v6 (0.00s) --- PASS: TestSetOutboundLocalIP/garbage (0.00s) === RUN TestSetOutboundLocalIPFirstWins --- PASS: TestSetOutboundLocalIPFirstWins (0.00s) === RUN TestOutboundLocalAddrForDial === RUN TestOutboundLocalAddrForDial/remote_tcp_binds === RUN TestOutboundLocalAddrForDial/tcp4_binds === RUN TestOutboundLocalAddrForDial/hostname_binds === RUN TestOutboundLocalAddrForDial/loopback_ip_skipped === RUN TestOutboundLocalAddrForDial/loopback_name_skipped === RUN TestOutboundLocalAddrForDial/ipv6_loopback_skipped === RUN TestOutboundLocalAddrForDial/ipv6_literal_target_skipped === RUN TestOutboundLocalAddrForDial/unix_network_skipped === RUN TestOutboundLocalAddrForDial/udp_network_skipped --- PASS: TestOutboundLocalAddrForDial (0.00s) --- PASS: TestOutboundLocalAddrForDial/remote_tcp_binds (0.00s) --- PASS: TestOutboundLocalAddrForDial/tcp4_binds (0.00s) --- PASS: TestOutboundLocalAddrForDial/hostname_binds (0.00s) --- PASS: TestOutboundLocalAddrForDial/loopback_ip_skipped (0.00s) --- PASS: TestOutboundLocalAddrForDial/loopback_name_skipped (0.00s) --- PASS: TestOutboundLocalAddrForDial/ipv6_loopback_skipped (0.00s) --- PASS: TestOutboundLocalAddrForDial/ipv6_literal_target_skipped (0.00s) --- PASS: TestOutboundLocalAddrForDial/unix_network_skipped (0.00s) --- PASS: TestOutboundLocalAddrForDial/udp_network_skipped (0.00s) === RUN TestOutboundLocalAddrFamilyMismatch --- PASS: TestOutboundLocalAddrFamilyMismatch (0.00s) === RUN TestParseHostPort === RUN TestParseHostPort/hostname === RUN TestParseHostPort/ipv4 === RUN TestParseHostPort/bracketed_ipv6 === RUN TestParseHostPort/missing_port === RUN TestParseHostPort/invalid_port --- PASS: TestParseHostPort (0.00s) --- PASS: TestParseHostPort/hostname (0.00s) --- PASS: TestParseHostPort/ipv4 (0.00s) --- PASS: TestParseHostPort/bracketed_ipv6 (0.00s) --- PASS: TestParseHostPort/missing_port (0.00s) --- PASS: TestParseHostPort/invalid_port (0.00s) === RUN TestPathPrefixFilter_Empty --- PASS: TestPathPrefixFilter_Empty (0.00s) === RUN TestPathPrefixFilter_ExcludeOnly --- PASS: TestPathPrefixFilter_ExcludeOnly (0.00s) === RUN TestPathPrefixFilter_IncludeOnly --- PASS: TestPathPrefixFilter_IncludeOnly (0.00s) === RUN TestPathPrefixFilter_DeeperPrefixWins --- PASS: TestPathPrefixFilter_DeeperPrefixWins (0.00s) === RUN TestPathPrefixFilter_DeeperExcludeWins --- PASS: TestPathPrefixFilter_DeeperExcludeWins (0.00s) === RUN TestPathPrefixFilter_MultipleOverlappingPrefixes --- PASS: TestPathPrefixFilter_MultipleOverlappingPrefixes (0.00s) === RUN TestPathPrefixFilter_InvalidPrefixes --- PASS: TestPathPrefixFilter_InvalidPrefixes (0.00s) === RUN TestPathPrefixFilter_TrailingSlashNormalization --- PASS: TestPathPrefixFilter_TrailingSlashNormalization (0.00s) === RUN TestPathPrefixFilter_BoundaryMatching --- PASS: TestPathPrefixFilter_BoundaryMatching (0.00s) === RUN TestNewQueue --- PASS: TestNewQueue (0.00s) === RUN TestEnqueueAndConsume 1 2 3 ----------------------- 4 5 6 7 ----------------------- --- PASS: TestEnqueueAndConsume (0.00s) === RUN TestRetryUntil === RUN TestRetryUntil/SucceedsImmediately === RUN TestRetryUntil/SucceedsAfterRetry === RUN TestRetryUntil/FailsNonRetryable --- PASS: TestRetryUntil (2.51s) --- PASS: TestRetryUntil/SucceedsImmediately (0.00s) --- PASS: TestRetryUntil/SucceedsAfterRetry (2.51s) --- PASS: TestRetryUntil/FailsNonRetryable (0.00s) === RUN TestRetryWithBackoff === RUN TestRetryWithBackoff/SucceedsAfterRetries === RUN TestRetryWithBackoff/StopsOnNonRetryableError === RUN TestRetryWithBackoff/StopsOnContextCancel === RUN TestRetryWithBackoff/StopsOnMaxDuration I0720 21:30:13.627307 retry.go:125 retry test: giving up after 3s: unavailable --- PASS: TestRetryWithBackoff (7.51s) --- PASS: TestRetryWithBackoff/SucceedsAfterRetries (2.51s) --- PASS: TestRetryWithBackoff/StopsOnNonRetryableError (0.00s) --- PASS: TestRetryWithBackoff/StopsOnContextCancel (2.00s) --- PASS: TestRetryWithBackoff/StopsOnMaxDuration (3.00s) === RUN TestSingleFlightGroupDedup --- PASS: TestSingleFlightGroupDedup (0.00s) === RUN TestSingleFlightGroupDifferentKeys --- PASS: TestSingleFlightGroupDifferentKeys (0.00s) === RUN TestSingleFlightGroupErrorPropagation --- PASS: TestSingleFlightGroupErrorPropagation (0.00s) === RUN TestSingleFlightGroupFreshCallAfterCompletion --- PASS: TestSingleFlightGroupFreshCallAfterCompletion (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util 11.243s === RUN TestSyncPoolPutBuffer_DropsOversized --- PASS: TestSyncPoolPutBuffer_DropsOversized (0.00s) === RUN TestSyncPoolPutBuffer_KeepsRightSized --- PASS: TestSyncPoolPutBuffer_KeepsRightSized (0.00s) === RUN TestSyncPoolPutBuffer_NilSafe --- PASS: TestSyncPoolPutBuffer_NilSafe (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/buffer_pool 0.006s === RUN TestJobQueue enqueued 5 items dequeue 1 dequeue 2 enqueue 6 enqueue 7 dequeue ... dequeued 3 dequeue ... dequeued 4 dequeue ... dequeued 5 dequeue ... dequeued 6 dequeue ... dequeued 7 enqueue 8 enqueue 9 enqueue 10 enqueue 11 enqueue 12 dequeued 8 dequeued 9 dequeued 10 dequeued 11 dequeued 12 --- PASS: TestJobQueue (0.00s) === RUN TestJobQueueClose dequeued 1 dequeued 2 dequeued 3 dequeued 4 dequeued 5 dequeued 6 dequeued 7 --- PASS: TestJobQueueClose (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/buffered_queue 0.011s ? github.com/seaweedfs/seaweedfs/weed/util/buffered_writer [no test files] === RUN TestOnDisk chunk_cache_on_disk_test.go:76: cache miss for entry 2 (acceptable with size constraints) chunk_cache_on_disk_test.go:129: cache miss for entry 2 after restart (acceptable) --- PASS: TestOnDisk (0.05s) === RUN TestLoadOrCreateChunkCacheVolumeClosesFilesOnError --- PASS: TestLoadOrCreateChunkCacheVolumeClosesFilesOnError (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/chunk_cache 0.071s ? github.com/seaweedfs/seaweedfs/weed/util/constants [no test files] ? github.com/seaweedfs/seaweedfs/weed/util/fla9 [no test files] ? github.com/seaweedfs/seaweedfs/weed/util/grace [no test files] === RUN TestAppendQueryParameter === RUN TestAppendQueryParameter/without_existing_query === RUN TestAppendQueryParameter/with_existing_query === RUN TestAppendQueryParameter/with_trailing_question_mark === RUN TestAppendQueryParameter/with_trailing_ampersand === RUN TestAppendQueryParameter/encodes_values === RUN TestAppendQueryParameter/preserves_fragment === RUN TestAppendQueryParameter/blank_url --- PASS: TestAppendQueryParameter (0.00s) --- PASS: TestAppendQueryParameter/without_existing_query (0.00s) --- PASS: TestAppendQueryParameter/with_existing_query (0.00s) --- PASS: TestAppendQueryParameter/with_trailing_question_mark (0.00s) --- PASS: TestAppendQueryParameter/with_trailing_ampersand (0.00s) --- PASS: TestAppendQueryParameter/encodes_values (0.00s) --- PASS: TestAppendQueryParameter/preserves_fragment (0.00s) --- PASS: TestAppendQueryParameter/blank_url (0.00s) === RUN TestReadUrlAsStreamReturnsGzipReaderError --- PASS: TestReadUrlAsStreamReturnsGzipReaderError (0.00s) === RUN TestDeleteReturnsInvalidRequestErrorBeforeAddingAuth --- PASS: TestDeleteReturnsInvalidRequestErrorBeforeAddingAuth (0.00s) === RUN TestDeleteTreatsNoContentAsSuccess --- PASS: TestDeleteTreatsNoContentAsSuccess (0.00s) === RUN TestDeleteProxiedReturnsInvalidRequestErrorBeforeAddingAuth --- PASS: TestDeleteProxiedReturnsInvalidRequestErrorBeforeAddingAuth (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/http 0.016s === RUN TestNewHttpClientSetsResponseTimeouts --- PASS: TestNewHttpClientSetsResponseTimeouts (0.00s) === RUN TestResponseHeaderTimeoutUnblocksUnresponsivePeer --- PASS: TestResponseHeaderTimeoutUnblocksUnresponsivePeer (0.20s) === RUN TestResponseHeaderTimeoutUnblocksUnresponsivePeerTLS --- PASS: TestResponseHeaderTimeoutUnblocksUnresponsivePeerTLS (0.21s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/http/client 0.422s ? github.com/seaweedfs/seaweedfs/weed/util/httpdown [no test files] === RUN TestReadTsCorruptedBuffer log_buffer_corruption_test.go:46: Error message: corrupted log buffer at pos 0, size 10: bad field tag at 0 log_buffer_corruption_test.go:53: ✓ readTs correctly returned error for corrupted buffer: corrupted log buffer at pos 0, size 10: bad field tag at 0 --- PASS: TestReadTsCorruptedBuffer (0.00s) === RUN TestReadTsValidBuffer log_buffer_corruption_test.go:92: ✓ readTs correctly parsed valid buffer: size=15, ts=123456789 --- PASS: TestReadTsValidBuffer (0.01s) === RUN TestReadFromBufferCorruption E0720 21:30:02.504982 log_buffer.go:914 ReadFromBuffer: buffer corruption at idx[0] pos 0: corrupted log buffer at pos 0, size 33: bad field tag at 0 log_buffer_corruption_test.go:134: ✓ ReadFromBuffer correctly detected corruption: log buffer is corrupted: corrupted log buffer at pos 0, size 33: bad field tag at 0 log_buffer_corruption_test.go:137: ReadFromBuffer result: buf=false, offset=-1, err=log buffer is corrupted: corrupted log buffer at pos 0, size 33: bad field tag at 0 --- PASS: TestReadFromBufferCorruption (0.03s) === RUN TestLocateByTsCorruption log_buffer_corruption_test.go:163: ✓ locateByTs correctly detected corruption: locateByTs: buffer corruption at pos 0: corrupted log buffer at pos 0, size 10: bad field tag at 0 --- PASS: TestLocateByTsCorruption (0.00s) === RUN TestErrorPropagationChain === RUN TestErrorPropagationChain/Corruption_in_readTs log_buffer_corruption_test.go:171: ✓ readTs error propagation tested === RUN TestErrorPropagationChain/Corruption_in_locateByTs log_buffer_corruption_test.go:176: ✓ locateByTs error propagation tested === RUN TestErrorPropagationChain/Corruption_in_ReadFromBuffer_binary_search log_buffer_corruption_test.go:181: ✓ ReadFromBuffer error propagation tested === NAME TestErrorPropagationChain log_buffer_corruption_test.go:184: ✓ Complete error propagation chain verified --- PASS: TestErrorPropagationChain (0.00s) --- PASS: TestErrorPropagationChain/Corruption_in_readTs (0.00s) --- PASS: TestErrorPropagationChain/Corruption_in_locateByTs (0.00s) --- PASS: TestErrorPropagationChain/Corruption_in_ReadFromBuffer_binary_search (0.00s) === RUN TestNoSilentCorruption === RUN TestNoSilentCorruption/Invalid_protobuf log_buffer_corruption_test.go:215: ✓ Correctly returned error instead of silent (0, 0): corrupted log buffer: entry size 167772160 at pos 0 exceeds buffer length 14 === RUN TestNoSilentCorruption/Truncated_data log_buffer_corruption_test.go:215: ✓ Correctly returned error instead of silent (0, 0): corrupted log buffer: entry size 1677721600 at pos 0 exceeds buffer length 7 --- PASS: TestNoSilentCorruption (0.00s) --- PASS: TestNoSilentCorruption/Invalid_protobuf (0.00s) --- PASS: TestNoSilentCorruption/Truncated_data (0.00s) === RUN TestFlushOffsetGap_ReproduceDataLoss log_buffer_flush_gap_test.go:69: Sending 100 messages... log_buffer_flush_gap_test.go:82: Forcing flush... log_buffer_flush_gap_test.go:33: FLUSH: minOffset=0 maxOffset=99 size=4434 bytes log_buffer_flush_gap_test.go:61: Parsed 100 messages from flush buffer log_buffer_flush_gap_test.go:33: FLUSH: minOffset=100 maxOffset=149 size=2352 bytes log_buffer_flush_gap_test.go:61: Parsed 50 messages from flush buffer log_buffer_flush_gap_test.go:117: BUFFER STATE AFTER FLUSH: log_buffer_flush_gap_test.go:118: bufferStartOffset: 150 log_buffer_flush_gap_test.go:119: currentOffset (HWM): 150 log_buffer_flush_gap_test.go:120: pos (bytes in buffer): 0 log_buffer_flush_gap_test.go:121: Messages sent: 150 (offsets 0-149) log_buffer_flush_gap_test.go:122: Messages flushed to disk: 150 (offsets 0-149) log_buffer_flush_gap_test.go:128: OFFSET CONTINUITY CHECK: log_buffer_flush_gap_test.go:129: Last flushed offset: 149 log_buffer_flush_gap_test.go:130: Buffer starts at: 150 log_buffer_flush_gap_test.go:131: Gap: 0 offsets log_buffer_flush_gap_test.go:143: PASS: No gap detected - offsets are continuous log_buffer_flush_gap_test.go:147: READABILITY CHECK: log_buffer_flush_gap_test.go:159: Offset 0: OK (buf=false, err=resumeFromDisk) log_buffer_flush_gap_test.go:159: Offset 10: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 20: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 30: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 40: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 50: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 60: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 70: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 80: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 90: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 100: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 110: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 120: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 130: OK (buf=true, err=) log_buffer_flush_gap_test.go:159: Offset 140: OK (buf=true, err=) log_buffer_flush_gap_test.go:177: MESSAGE ACCOUNTING: log_buffer_flush_gap_test.go:178: Expected: 150 messages log_buffer_flush_gap_test.go:179: Flushed to disk: 150 log_buffer_flush_gap_test.go:180: In memory buffer: 0 (offset range 150-149) log_buffer_flush_gap_test.go:181: Total accounted for: 150 log_buffer_flush_gap_test.go:182: Missing: 0 messages log_buffer_flush_gap_test.go:187: All messages accounted for --- PASS: TestFlushOffsetGap_ReproduceDataLoss (0.22s) === RUN TestFlushOffsetGap_CheckPrevBuffers log_buffer_flush_gap_test.go:211: Batch 0: log_buffer_flush_gap_test.go:203: FLUSH #1: minOffset=0 maxOffset=19 size=872 bytes log_buffer_flush_gap_test.go:242: Before flush: offset=20, bufferStartOffset=0 log_buffer_flush_gap_test.go:243: After flush: offset=20, bufferStartOffset=20, prevBuffers=4 log_buffer_flush_gap_test.go:249: prevBuffer[3]: offsets 0-19, size=872 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:211: Batch 1: log_buffer_flush_gap_test.go:203: FLUSH #2: minOffset=20 maxOffset=39 size=874 bytes log_buffer_flush_gap_test.go:242: Before flush: offset=40, bufferStartOffset=20 log_buffer_flush_gap_test.go:243: After flush: offset=40, bufferStartOffset=40, prevBuffers=4 log_buffer_flush_gap_test.go:249: prevBuffer[2]: offsets 0-19, size=872 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[3]: offsets 20-39, size=874 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:211: Batch 2: log_buffer_flush_gap_test.go:203: FLUSH #3: minOffset=40 maxOffset=59 size=879 bytes log_buffer_flush_gap_test.go:242: Before flush: offset=60, bufferStartOffset=40 log_buffer_flush_gap_test.go:243: After flush: offset=60, bufferStartOffset=60, prevBuffers=4 log_buffer_flush_gap_test.go:249: prevBuffer[1]: offsets 0-19, size=872 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[2]: offsets 20-39, size=874 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[3]: offsets 40-59, size=879 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:211: Batch 3: log_buffer_flush_gap_test.go:203: FLUSH #4: minOffset=60 maxOffset=79 size=904 bytes log_buffer_flush_gap_test.go:242: Before flush: offset=80, bufferStartOffset=60 log_buffer_flush_gap_test.go:243: After flush: offset=80, bufferStartOffset=80, prevBuffers=4 log_buffer_flush_gap_test.go:249: prevBuffer[0]: offsets 0-19, size=872 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[1]: offsets 20-39, size=874 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[2]: offsets 40-59, size=879 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[3]: offsets 60-79, size=904 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:211: Batch 4: log_buffer_flush_gap_test.go:203: FLUSH #5: minOffset=80 maxOffset=99 size=905 bytes log_buffer_flush_gap_test.go:242: Before flush: offset=100, bufferStartOffset=80 log_buffer_flush_gap_test.go:243: After flush: offset=100, bufferStartOffset=100, prevBuffers=4 log_buffer_flush_gap_test.go:249: prevBuffer[0]: offsets 20-39, size=874 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[1]: offsets 40-59, size=879 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[2]: offsets 60-79, size=904 bytes (NOT FLUSHED!) log_buffer_flush_gap_test.go:249: prevBuffer[3]: offsets 80-99, size=905 bytes (NOT FLUSHED!) --- PASS: TestFlushOffsetGap_CheckPrevBuffers (0.27s) === RUN TestFlushOffsetGap_ConcurrentWriteAndFlush log_buffer_flush_gap_test.go:273: FLUSH: offsets 0-100 (4478 bytes) log_buffer_flush_gap_test.go:273: FLUSH: offsets 101-199 (4669 bytes) log_buffer_flush_gap_test.go:335: FINAL STATE: log_buffer_flush_gap_test.go:336: Total messages sent: 200 (offsets 0-199) log_buffer_flush_gap_test.go:337: Flushed to disk: 200 log_buffer_flush_gap_test.go:338: In memory: 0 (offsets 200-199) log_buffer_flush_gap_test.go:339: Total accounted: 200 log_buffer_flush_gap_test.go:340: Missing: 0 --- PASS: TestFlushOffsetGap_ConcurrentWriteAndFlush (0.36s) === RUN TestFlushOffsetGap_ProductionScenario log_buffer_flush_gap_test.go:393: === ROUND 1: Adding messages 0-49 === log_buffer_flush_gap_test.go:412: Before flush: logBuffer.offset=50, bufferStartOffset=0, nextKafkaOffset=50 log_buffer_flush_gap_test.go:382: FLUSH: minOffset=0 maxOffset=49, parsed 50 messages log_buffer_flush_gap_test.go:424: After flush: logBuffer.offset=50, bufferStartOffset=50 log_buffer_flush_gap_test.go:428: === ROUND 2: Adding messages 50-99 === log_buffer_flush_gap_test.go:382: FLUSH: minOffset=50 maxOffset=99, parsed 50 messages log_buffer_flush_gap_test.go:447: === VERIFICATION === log_buffer_flush_gap_test.go:448: Expected Kafka offsets: 0-99 log_buffer_flush_gap_test.go:452: Flush #0: minOffset=0, maxOffset=49, messages=50 log_buffer_flush_gap_test.go:452: Flush #1: minOffset=50, maxOffset=99, messages=50 log_buffer_flush_gap_test.go:481: SUCCESS: All 100 Kafka offsets accounted for (0-99) log_buffer_flush_gap_test.go:490: Final buffer state: log_buffer_flush_gap_test.go:491: logBuffer.offset: 100 log_buffer_flush_gap_test.go:492: bufferStartOffset: 100 log_buffer_flush_gap_test.go:493: Expected (nextKafkaOffset): 100 --- PASS: TestFlushOffsetGap_ProductionScenario (0.21s) === RUN TestFlushOffsetGap_ConcurrentReadDuringFlush log_buffer_flush_gap_test.go:554: Adding 100 messages... log_buffer_flush_gap_test.go:568: Flushing... log_buffer_flush_gap_test.go:546: FLUSH: Stored 100 offsets to disk (minOffset=0, maxOffset=99) log_buffer_flush_gap_test.go:573: Reading messages from offset 0... E0720 21:30:03.664046 log_read_stateless.go:169 [StatelessRead] CASE 1: Historical data - offset 0 < bufferStart 100 log_buffer_flush_gap_test.go:576: Read result: messages=100, nextOffset=100, hwm=100, endOfPartition=true, err= log_buffer_flush_gap_test.go:600: All 100 offsets can be read after flush --- PASS: TestFlushOffsetGap_ConcurrentReadDuringFlush (0.10s) === RUN TestFlushOffsetGap_ForceFlushAdvancesBuffer log_buffer_flush_gap_test.go:622: === ROUND 0 === log_buffer_flush_gap_test.go:630: Before adding: offset=0, bufferStartOffset=0 log_buffer_flush_gap_test.go:649: After adding: offset=10, bufferStartOffset=0 log_buffer_flush_gap_test.go:652: Forcing flush... log_buffer_flush_gap_test.go:614: FLUSH: offsets 0-9 log_buffer_flush_gap_test.go:662: After flush: offset=10, bufferStartOffset=10 log_buffer_flush_gap_test.go:671: bufferStartOffset correctly advanced to 10 log_buffer_flush_gap_test.go:622: === ROUND 1 === log_buffer_flush_gap_test.go:630: Before adding: offset=10, bufferStartOffset=10 log_buffer_flush_gap_test.go:649: After adding: offset=20, bufferStartOffset=10 log_buffer_flush_gap_test.go:652: Forcing flush... log_buffer_flush_gap_test.go:614: FLUSH: offsets 10-19 log_buffer_flush_gap_test.go:662: After flush: offset=20, bufferStartOffset=20 log_buffer_flush_gap_test.go:671: bufferStartOffset correctly advanced to 20 log_buffer_flush_gap_test.go:622: === ROUND 2 === log_buffer_flush_gap_test.go:630: Before adding: offset=20, bufferStartOffset=20 log_buffer_flush_gap_test.go:649: After adding: offset=30, bufferStartOffset=20 log_buffer_flush_gap_test.go:652: Forcing flush... log_buffer_flush_gap_test.go:614: FLUSH: offsets 20-29 log_buffer_flush_gap_test.go:662: After flush: offset=30, bufferStartOffset=30 log_buffer_flush_gap_test.go:671: bufferStartOffset correctly advanced to 30 log_buffer_flush_gap_test.go:677: === FLUSHED RANGES === log_buffer_flush_gap_test.go:679: Flush #0: offsets 0-9 log_buffer_flush_gap_test.go:679: Flush #1: offsets 10-19 log_buffer_flush_gap_test.go:692: Continuous with previous flush log_buffer_flush_gap_test.go:679: Flush #2: offsets 20-29 log_buffer_flush_gap_test.go:692: Continuous with previous flush --- PASS: TestFlushOffsetGap_ForceFlushAdvancesBuffer (0.31s) === RUN TestBufferQueryability log_buffer_queryability_test.go:104: Buffer queryability test passed - data is immediately readable --- PASS: TestBufferQueryability (0.01s) === RUN TestMultipleEntriesQueryability log_buffer_queryability_test.go:168: Entry 1: Key=test-key-1, Data=test-data-1, Offset=1 log_buffer_queryability_test.go:168: Entry 2: Key=test-key-2, Data=test-data-2, Offset=2 log_buffer_queryability_test.go:168: Entry 3: Key=test-key-3, Data=test-data-3, Offset=3 log_buffer_queryability_test.go:175: Multiple entries queryability test passed - found 3 entries --- PASS: TestMultipleEntriesQueryability (0.00s) === RUN TestSchemaRegistryScenario log_buffer_queryability_test.go:243: Schema registry scenario test passed - schema value preserved: 109 bytes --- PASS: TestSchemaRegistryScenario (0.01s) === RUN TestTimeBasedFirstReadBeforeEarliest --- PASS: TestTimeBasedFirstReadBeforeEarliest (0.00s) === RUN TestEarliestTimeExactRead --- PASS: TestEarliestTimeExactRead (0.00s) === RUN TestNewLogBufferFirstBuffer processed all messages E0720 21:30:04.011745 log_read.go:290 LoopProcessLogData: test process log entry 1 key:"" ts_ns:1784583004011712755 offset:4999 size:1024: EOF before flush: sent 5000 received 5000 lastProcessedTime {2026-07-20 21:30:04.011712755 +0000 UTC 5000 false} isDone true err: EOF --- PASS: TestNewLogBufferFirstBuffer (0.02s) === RUN TestReadFromBufferTimestampBased_AfterFlushReturnsNewerData --- PASS: TestReadFromBufferTimestampBased_AfterFlushReturnsNewerData (0.00s) === RUN TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError === RUN TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_0_when_buffer_starts_at_4_(Schema_Registry_bug_scenario) log_buffer_test.go:211: ✓ When Schema Registry tries to read from offset 0, but data has been flushed to disk: correctly returned resumeFromDisk === RUN TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_before_buffer_start_with_empty_buffer log_buffer_test.go:211: ✓ Old offset with no data in memory should trigger disk read: correctly returned resumeFromDisk === RUN TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_before_buffer_start_with_data log_buffer_test.go:211: ✓ Old offset with current data in memory should still trigger disk read: correctly returned resumeFromDisk === RUN TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_current_offset_(no_disk_read_needed) log_buffer_test.go:218: ✓ Current offset should return data from memory without error: correctly returned data without error === RUN TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_within_buffer_range log_buffer_test.go:218: ✓ Offset within buffer range should return data without error: correctly returned data without error --- PASS: TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError (0.02s) --- PASS: TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_0_when_buffer_starts_at_4_(Schema_Registry_bug_scenario) (0.01s) --- PASS: TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_before_buffer_start_with_empty_buffer (0.01s) --- PASS: TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_before_buffer_start_with_data (0.00s) --- PASS: TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_current_offset_(no_disk_read_needed) (0.00s) --- PASS: TestReadFromBuffer_OldOffsetReturnsResumeFromDiskError/Request_offset_within_buffer_range (0.00s) === RUN TestReadFromBuffer_OldOffsetWithNoPrevBuffers log_buffer_test.go:262: DEBUG: ReadFromBuffer returned: buf=false, batchIdx=-2, err=resumeFromDisk log_buffer_test.go:263: DEBUG: Buffer state: bufferStartOffset=10, offset=15, pos=100 log_buffer_test.go:265: DEBUG: Requested offset 5, prevBuffers[0] range: [20-25] log_buffer_test.go:274: ✓ BUG FIX VERIFIED: Correctly returns ResumeFromDiskError when requesting old offset 5 log_buffer_test.go:275: This allows the subscriber to read from disk instead of waiting forever --- PASS: TestReadFromBuffer_OldOffsetWithNoPrevBuffers (0.00s) === RUN TestReadFromBuffer_EmptyBufferAtCurrentOffset log_buffer_test.go:300: DEBUG: ReadFromBuffer returned: buf=false, batchIdx=-2, err=resumeFromDisk log_buffer_test.go:301: DEBUG: Buffer state: bufferStartOffset=0, offset=4, pos=0 log_buffer_test.go:311: ✓ BUG #2 FIX VERIFIED: Empty buffer correctly returns ResumeFromDiskError to check disk --- PASS: TestReadFromBuffer_EmptyBufferAtCurrentOffset (0.00s) === RUN TestReadFromBuffer_OffsetRanges === RUN TestReadFromBuffer_OffsetRanges/Before_buffer_start log_buffer_test.go:371: ✓ Offset 5 < bufferStartOffset 10 → read from disk === RUN TestReadFromBuffer_OffsetRanges/At_buffer_start log_buffer_test.go:379: ✓ Offset 10 == bufferStartOffset 10 → read from buffer === RUN TestReadFromBuffer_OffsetRanges/Within_buffer_range log_buffer_test.go:379: ✓ Offset 15 is within [10, 20] → read from buffer === RUN TestReadFromBuffer_OffsetRanges/At_buffer_end log_buffer_test.go:379: ✓ Offset 20 == offset 20 → read from buffer === RUN TestReadFromBuffer_OffsetRanges/After_buffer_end log_buffer_test.go:379: ✓ Offset 25 > offset 20 → future data, return nil without error --- PASS: TestReadFromBuffer_OffsetRanges (0.00s) --- PASS: TestReadFromBuffer_OffsetRanges/Before_buffer_start (0.00s) --- PASS: TestReadFromBuffer_OffsetRanges/At_buffer_start (0.00s) --- PASS: TestReadFromBuffer_OffsetRanges/Within_buffer_range (0.00s) --- PASS: TestReadFromBuffer_OffsetRanges/At_buffer_end (0.00s) --- PASS: TestReadFromBuffer_OffsetRanges/After_buffer_end (0.00s) === RUN TestReadFromBuffer_InitializedFromDisk log_buffer_test.go:410: After InitializeOffsetFromExistingData(highestOffset=3): log_buffer_test.go:411: offset=4 (should be 4), bufferStartOffset=4 (FIX: should be 4, not 0) log_buffer_test.go:429: After writing new message: log_buffer_test.go:430: bufferStartOffset=4, offset=5, pos=56 log_buffer_test.go:431: Requested offset 0, got: buf=false, batchIdx=-2, err=resumeFromDisk log_buffer_test.go:449: ✓ BUG #3 FIX VERIFIED: Reading old offset 0 correctly returns ResumeFromDiskError log_buffer_test.go:450: This ensures Schema Registry reads correct data from disk instead of getting new messages --- PASS: TestReadFromBuffer_InitializedFromDisk (0.00s) === RUN TestLoopProcessLogDataWithOffset_DiskReadRetry log_buffer_test.go:481: DISK READ #1: startOffset=0, dataReady=false log_buffer_test.go:484: → No data on disk yet log_buffer_test.go:545: ➕ Marking data as ready on disk flush from 2026-07-20 21:30:03.995533457 +0000 UTC m=+1.549168286 to 2026-07-20 21:30:04.011712755 +0000 UTC m=+1.565347574 5274870 bytes log_buffer_test.go:481: DISK READ #2: startOffset=0, dataReady=true log_buffer_test.go:516: ✉️ RECEIVED: offset=0 key=key-0 log_buffer_test.go:526: 📋 Reader finished: isDone=true, err= log_buffer_test.go:561: RESULTS: log_buffer_test.go:562: Disk reads: 2 log_buffer_test.go:563: Received messages: 1 log_buffer_test.go:564: Loop iterations: 1 log_buffer_test.go:576: ✓ SUCCESS: Message received after 2 disk read attempts --- PASS: TestLoopProcessLogDataWithOffset_DiskReadRetry (2.00s) === RUN TestConcurrentProducerConsumer log_read_integration_test.go:108: Consumer 0 consumed 500 messages log_read_integration_test.go:108: Consumer 1 consumed 500 messages --- PASS: TestConcurrentProducerConsumer (1.18s) === RUN TestBackwardSeeksWhileProducing log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:183: Seeking backward from 100 to 80 log_read_integration_test.go:205: Total unique offsets read: 500 out of 500 --- PASS: TestBackwardSeeksWhileProducing (0.55s) === RUN TestHighConcurrencyReads --- PASS: TestHighConcurrencyReads (0.00s) === RUN TestRepeatedReadsAtSameOffset --- PASS: TestRepeatedReadsAtSameOffset (0.00s) === RUN TestEmptyPartitionPolling --- PASS: TestEmptyPartitionPolling (0.00s) === RUN TestReadMessagesAtOffset_EmptyBuffer --- PASS: TestReadMessagesAtOffset_EmptyBuffer (0.00s) === RUN TestReadMessagesAtOffset_SingleMessage --- PASS: TestReadMessagesAtOffset_SingleMessage (0.00s) === RUN TestReadMessagesAtOffset_MultipleMessages --- PASS: TestReadMessagesAtOffset_MultipleMessages (0.00s) === RUN TestReadMessagesAtOffset_StartFromMiddle --- PASS: TestReadMessagesAtOffset_StartFromMiddle (0.00s) === RUN TestReadMessagesAtOffset_MaxBytesLimit --- PASS: TestReadMessagesAtOffset_MaxBytesLimit (0.00s) === RUN TestReadMessagesAtOffset_ConcurrentReads --- PASS: TestReadMessagesAtOffset_ConcurrentReads (0.00s) === RUN TestReadMessagesAtOffset_FutureOffset --- PASS: TestReadMessagesAtOffset_FutureOffset (0.00s) === RUN TestWaitForDataWithTimeout_DataAvailable --- PASS: TestWaitForDataWithTimeout_DataAvailable (0.00s) === RUN TestWaitForDataWithTimeout_NoData log_read_stateless_test.go:314: Waited 50.351555ms for timeout --- PASS: TestWaitForDataWithTimeout_NoData (0.05s) === RUN TestWaitForDataWithTimeout_DataArrives --- PASS: TestWaitForDataWithTimeout_DataArrives (0.05s) === RUN TestGetHighWaterMark --- PASS: TestGetHighWaterMark (0.00s) === RUN TestGetLogStartOffset --- PASS: TestGetLogStartOffset (0.00s) === RUN TestLoopProcessLogDataWithOffset_ClientDisconnect log_read_test.go:53: Loop exited cleanly in 250.530787ms after client disconnect --- PASS: TestLoopProcessLogDataWithOffset_ClientDisconnect (0.25s) === RUN TestLoopProcessLogDataWithOffset_EmptyBuffer log_read_test.go:108: Loop exited cleanly in 751.254893ms after 4 iterations (no busy-waiting detected) --- PASS: TestLoopProcessLogDataWithOffset_EmptyBuffer (0.75s) === RUN TestLoopProcessLogDataWithOffset_NoDataResumeFromDisk log_read_test.go:154: Loop exited cleanly in 4.003177344s after 3 iterations (proper sleep detected) --- PASS: TestLoopProcessLogDataWithOffset_NoDataResumeFromDisk (4.00s) === RUN TestLoopFlush_NotifiesSubscribersAfterFlush log_read_test.go:205: flush notification delivered in 11.212µs --- PASS: TestLoopFlush_NotifiesSubscribersAfterFlush (0.00s) === RUN TestLoopProcessLogDataWithOffset_WakesOnDataArrival I0720 21:30:12.931104 log_read.go:567 LoopProcessLogDataWithOffset: wake-test process log entry 1 log_read_test.go:274: reader processed the entry in 346.601µs after AddToBuffer --- PASS: TestLoopProcessLogDataWithOffset_WakesOnDataArrival (0.05s) === RUN TestLoopProcessLogDataWithOffset_WakesOnShutdown log_read_test.go:335: reader exited in 23.175µs after ShutdownLogBuffer --- PASS: TestLoopProcessLogDataWithOffset_WakesOnShutdown (0.05s) === RUN TestLoopProcessLogDataWithOffset_WithData I0720 21:30:12.984592 log_read.go:567 LoopProcessLogDataWithOffset: test-client process log entry 1 log_read_test.go:401: Successfully processed 1 message(s) in 221.833µs --- PASS: TestLoopProcessLogDataWithOffset_WithData (0.00s) === RUN TestLoopProcessLogDataWithOffset_ConcurrentDisconnect log_read_test.go:449: All 10 concurrent clients exited cleanly --- PASS: TestLoopProcessLogDataWithOffset_ConcurrentDisconnect (0.25s) === RUN TestLoopProcessLogDataWithOffset_StopTime log_read_test.go:488: Loop correctly exited for past stopTsNs in 10.069µs (waitForDataFn called 0 times) --- PASS: TestLoopProcessLogDataWithOffset_StopTime (0.00s) === RUN TestLoopProcessLogData_SlowConsumerFallsBehind --- PASS: TestLoopProcessLogData_SlowConsumerFallsBehind (0.00s) === RUN TestReadTsNsMatchesUnmarshal --- PASS: TestReadTsNsMatchesUnmarshal (0.00s) === RUN TestReadTsNsFuzz --- PASS: TestReadTsNsFuzz (0.00s) === RUN TestReadTsNsCorrupted --- PASS: TestReadTsNsCorrupted (0.00s) === RUN TestUnmarshalLogEntryAliasedMatchesUnmarshal --- PASS: TestUnmarshalLogEntryAliasedMatchesUnmarshal (0.02s) === RUN TestUnmarshalLogEntryAliasedAliases --- PASS: TestUnmarshalLogEntryAliasedAliases (0.00s) === RUN TestSealedReadsShareOneSnapshot --- PASS: TestSealedReadsShareOneSnapshot (0.00s) === RUN TestSharedSnapshotSurvivesRecycle --- PASS: TestSharedSnapshotSurvivesRecycle (0.00s) === RUN TestSharedSnapshotConcurrentIntegrity --- PASS: TestSharedSnapshotConcurrentIntegrity (2.06s) === RUN TestCurrentWindowPrefixSharing --- PASS: TestCurrentWindowPrefixSharing (0.00s) === RUN TestSealHandsOffCompleteSnapshot --- PASS: TestSealHandsOffCompleteSnapshot (0.00s) === RUN TestSharedSnapshotContentMatches --- PASS: TestSharedSnapshotContentMatches (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/log_buffer 12.895s === RUN TestAllocateFree --- PASS: TestAllocateFree (0.00s) === RUN TestAllocateFreeEdgeCases --- PASS: TestAllocateFreeEdgeCases (0.00s) === RUN TestBitCount --- PASS: TestBitCount (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/mem 0.018s === RUN TestQuoteDSNValue --- PASS: TestQuoteDSNValue (0.00s) === RUN TestBuildDSN_QuotesProblematicValues --- PASS: TestBuildDSN_QuotesProblematicValues (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/pgxutil 0.019s === RUN TestNewUsesUppercaseHexFormat --- PASS: TestNewUsesUppercaseHexFormat (0.00s) === RUN TestNewIsUnique --- PASS: TestNewIsUnique (0.00s) === RUN TestEnsureIgnoresClientHeader --- PASS: TestEnsureIgnoresClientHeader (0.00s) === RUN TestEnsureReusesContextID --- PASS: TestEnsureReusesContextID (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/request_id 0.017s === RUN TestReverseInsert --- PASS: TestReverseInsert (0.00s) === RUN TestDeleteRekeyedNode --- PASS: TestDeleteRekeyedNode (0.00s) === RUN TestInsertByKeyRecoversInconsistentEnds --- PASS: TestInsertByKeyRecoversInconsistentEnds (0.00s) === RUN TestInsertAndFind --- PASS: TestInsertAndFind (0.08s) === RUN TestDelete --- PASS: TestDelete (0.10s) === RUN TestNext --- PASS: TestNext (0.02s) === RUN TestPrev --- PASS: TestPrev (0.01s) === RUN TestFindGreaterOrEqual --- PASS: TestFindGreaterOrEqual (0.08s) === RUN TestChangeValue --- PASS: TestChangeValue (0.03s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/skiplist 0.345s === RUN TestSplitStatements === RUN TestSplitStatements/Simple_single_statement === RUN TestSplitStatements/Multiple_statements === RUN TestSplitStatements/Semicolon_in_single_quotes === RUN TestSplitStatements/Semicolon_in_double_quotes === RUN TestSplitStatements/Escaped_quotes_in_strings === RUN TestSplitStatements/Escaped_quotes_in_identifiers === RUN TestSplitStatements/Single_line_comment === RUN TestSplitStatements/Single_line_comment_with_semicolon === RUN TestSplitStatements/Multi-line_comment === RUN TestSplitStatements/Multi-line_comment_with_semicolon === RUN TestSplitStatements/Complex_mixed_case === RUN TestSplitStatements/Empty_statements_filtered === RUN TestSplitStatements/Whitespace_handling === RUN TestSplitStatements/Single_statement_without_semicolon === RUN TestSplitStatements/Empty_query === RUN TestSplitStatements/Only_whitespace === RUN TestSplitStatements/Only_single_line_comment === RUN TestSplitStatements/Only_multi-line_comment --- PASS: TestSplitStatements (0.00s) --- PASS: TestSplitStatements/Simple_single_statement (0.00s) --- PASS: TestSplitStatements/Multiple_statements (0.00s) --- PASS: TestSplitStatements/Semicolon_in_single_quotes (0.00s) --- PASS: TestSplitStatements/Semicolon_in_double_quotes (0.00s) --- PASS: TestSplitStatements/Escaped_quotes_in_strings (0.00s) --- PASS: TestSplitStatements/Escaped_quotes_in_identifiers (0.00s) --- PASS: TestSplitStatements/Single_line_comment (0.00s) --- PASS: TestSplitStatements/Single_line_comment_with_semicolon (0.00s) --- PASS: TestSplitStatements/Multi-line_comment (0.00s) --- PASS: TestSplitStatements/Multi-line_comment_with_semicolon (0.00s) --- PASS: TestSplitStatements/Complex_mixed_case (0.00s) --- PASS: TestSplitStatements/Empty_statements_filtered (0.00s) --- PASS: TestSplitStatements/Whitespace_handling (0.00s) --- PASS: TestSplitStatements/Single_statement_without_semicolon (0.00s) --- PASS: TestSplitStatements/Empty_query (0.00s) --- PASS: TestSplitStatements/Only_whitespace (0.00s) --- PASS: TestSplitStatements/Only_single_line_comment (0.00s) --- PASS: TestSplitStatements/Only_multi-line_comment (0.00s) === RUN TestSplitStatements_EdgeCases === RUN TestSplitStatements_EdgeCases/Nested_comments_are_not_supported_but_handled_gracefully === RUN TestSplitStatements_EdgeCases/Unterminated_string_(malformed_SQL) === RUN TestSplitStatements_EdgeCases/Unterminated_comment_(malformed_SQL) === RUN TestSplitStatements_EdgeCases/Multiple_semicolons_in_quotes --- PASS: TestSplitStatements_EdgeCases (0.00s) --- PASS: TestSplitStatements_EdgeCases/Nested_comments_are_not_supported_but_handled_gracefully (0.00s) --- PASS: TestSplitStatements_EdgeCases/Unterminated_string_(malformed_SQL) (0.00s) --- PASS: TestSplitStatements_EdgeCases/Unterminated_comment_(malformed_SQL) (0.00s) --- PASS: TestSplitStatements_EdgeCases/Multiple_semicolons_in_quotes (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/sqlutil 0.013s ? github.com/seaweedfs/seaweedfs/weed/util/version [no test files] === RUN TestMatchesWildcard === RUN TestMatchesWildcard/Exact_match === RUN TestMatchesWildcard/Single_wildcard === RUN TestMatchesWildcard/Empty_string_with_wildcard === RUN TestMatchesWildcard/Prefix_wildcard === RUN TestMatchesWildcard/Suffix_wildcard === RUN TestMatchesWildcard/Middle_wildcard === RUN TestMatchesWildcard/Multiple_wildcards === RUN TestMatchesWildcard/No_match === RUN TestMatchesWildcard/Single_question_mark === RUN TestMatchesWildcard/Multiple_question_marks === RUN TestMatchesWildcard/Question_mark_no_match === RUN TestMatchesWildcard/Mixed_wildcards === RUN TestMatchesWildcard/Empty_pattern === RUN TestMatchesWildcard/Empty_pattern_with_string === RUN TestMatchesWildcard/Pattern_with_string_empty === RUN TestMatchesWildcard/Pattern_with_regex_special_chars === RUN TestMatchesWildcard/Pattern_with_dots === RUN TestMatchesWildcard/Pattern_with_dots_and_wildcard --- PASS: TestMatchesWildcard (0.00s) --- PASS: TestMatchesWildcard/Exact_match (0.00s) --- PASS: TestMatchesWildcard/Single_wildcard (0.00s) --- PASS: TestMatchesWildcard/Empty_string_with_wildcard (0.00s) --- PASS: TestMatchesWildcard/Prefix_wildcard (0.00s) --- PASS: TestMatchesWildcard/Suffix_wildcard (0.00s) --- PASS: TestMatchesWildcard/Middle_wildcard (0.00s) --- PASS: TestMatchesWildcard/Multiple_wildcards (0.00s) --- PASS: TestMatchesWildcard/No_match (0.00s) --- PASS: TestMatchesWildcard/Single_question_mark (0.00s) --- PASS: TestMatchesWildcard/Multiple_question_marks (0.00s) --- PASS: TestMatchesWildcard/Question_mark_no_match (0.00s) --- PASS: TestMatchesWildcard/Mixed_wildcards (0.00s) --- PASS: TestMatchesWildcard/Empty_pattern (0.00s) --- PASS: TestMatchesWildcard/Empty_pattern_with_string (0.00s) --- PASS: TestMatchesWildcard/Pattern_with_string_empty (0.00s) --- PASS: TestMatchesWildcard/Pattern_with_regex_special_chars (0.00s) --- PASS: TestMatchesWildcard/Pattern_with_dots (0.00s) --- PASS: TestMatchesWildcard/Pattern_with_dots_and_wildcard (0.00s) === RUN TestWildcardMatcherMatch === RUN TestWildcardMatcherMatch/Simple_star === RUN TestWildcardMatcherMatch/Question_mark === RUN TestWildcardMatcherMatch/Extension_filter --- PASS: TestWildcardMatcherMatch (0.00s) --- PASS: TestWildcardMatcherMatch/Simple_star (0.00s) --- PASS: TestWildcardMatcherMatch/Question_mark (0.00s) --- PASS: TestWildcardMatcherMatch/Extension_filter (0.00s) === RUN TestCompileWildcardPattern === RUN TestCompileWildcardPattern/s3:Get* === RUN TestCompileWildcardPattern/s3:Get?bject === RUN TestCompileWildcardPattern/s3:*Object* --- PASS: TestCompileWildcardPattern (0.00s) --- PASS: TestCompileWildcardPattern/s3:Get* (0.00s) --- PASS: TestCompileWildcardPattern/s3:Get?bject (0.00s) --- PASS: TestCompileWildcardPattern/s3:*Object* (0.00s) === RUN TestFastMatchesWildcard === RUN TestFastMatchesWildcard/s3:Get*_s3:GetObject === RUN TestFastMatchesWildcard/s3:Put*_s3:GetObject === RUN TestFastMatchesWildcard/arn:aws:s3:::bucket/*_arn:aws:s3:::bucket/file.txt === RUN TestFastMatchesWildcard/user:admin-*_user:admin-john === RUN TestFastMatchesWildcard/user:admin-*_user:guest-john --- PASS: TestFastMatchesWildcard (0.00s) --- PASS: TestFastMatchesWildcard/s3:Get*_s3:GetObject (0.00s) --- PASS: TestFastMatchesWildcard/s3:Put*_s3:GetObject (0.00s) --- PASS: TestFastMatchesWildcard/arn:aws:s3:::bucket/*_arn:aws:s3:::bucket/file.txt (0.00s) --- PASS: TestFastMatchesWildcard/user:admin-*_user:admin-john (0.00s) --- PASS: TestFastMatchesWildcard/user:admin-*_user:guest-john (0.00s) === RUN TestWildcardMatcherCaching --- PASS: TestWildcardMatcherCaching (0.00s) === RUN TestWildcardMatcherCacheBounding --- PASS: TestWildcardMatcherCacheBounding (0.00s) === RUN TestWildcardMatcherCacheLRU --- PASS: TestWildcardMatcherCacheLRU (0.00s) === RUN TestWildcardMatcherCacheClear --- PASS: TestWildcardMatcherCacheClear (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/util/wildcard 0.013s === RUN TestApplyDiscoveredFilersPrunesStaleAddress I0720 21:30:02.509226 filer_client.go:429 FilerClient: removed 1 filer(s) no longer in group '': [10.0.0.2:18888] I0720 21:30:02.509516 filer_client.go:432 FilerClient: discovered 1 new filer(s) in group '': [10.0.0.3:18888] --- PASS: TestApplyDiscoveredFilersPrunesStaleAddress (0.00s) === RUN TestApplyDiscoveredFilersKeepsIndexOnSurvivor I0720 21:30:02.509552 filer_client.go:429 FilerClient: removed 1 filer(s) no longer in group '': [10.0.0.1:18888] I0720 21:30:02.509573 filer_client.go:432 FilerClient: discovered 1 new filer(s) in group '': [10.0.0.3:18888] --- PASS: TestApplyDiscoveredFilersKeepsIndexOnSurvivor (0.00s) === RUN TestApplyDiscoveredFilersNoChangeIsNoop --- PASS: TestApplyDiscoveredFilersNoChangeIsNoop (0.00s) === RUN TestJitterBounds --- PASS: TestJitterBounds (0.00s) === RUN TestJitterZeroAndNegative --- PASS: TestJitterZeroAndNegative (0.00s) === RUN TestJitterTinyDuration --- PASS: TestJitterTinyDuration (0.00s) === RUN TestJitterDistribution --- PASS: TestJitterDistribution (0.00s) === RUN TestLookupVolumeIdsRetriesOnUnavailable --- PASS: TestLookupVolumeIdsRetriesOnUnavailable (4.78s) === RUN TestLookupVolumeIdsStopsOnContextCancel --- PASS: TestLookupVolumeIdsStopsOnContextCancel (2.00s) === RUN TestWaitUntilConnectedWithoutKeepConnected --- PASS: TestWaitUntilConnectedWithoutKeepConnected (0.10s) === RUN TestWaitUntilConnectedReturnsImmediatelyWhenConnected --- PASS: TestWaitUntilConnectedReturnsImmediatelyWhenConnected (0.00s) === RUN TestGetMasterRespectsContextCancellation --- PASS: TestGetMasterRespectsContextCancellation (0.10s) === RUN TestMasterClientFilerGroupLogging --- PASS: TestMasterClientFilerGroupLogging (0.00s) === RUN TestLookupFileId --- PASS: TestLookupFileId (0.00s) === RUN TestConcurrentGetLocations --- PASS: TestConcurrentGetLocations (0.79s) === RUN TestHasVolumeServer --- PASS: TestHasVolumeServer (0.00s) === RUN TestInvalidateCacheValidFileId --- PASS: TestInvalidateCacheValidFileId (0.00s) === RUN TestInvalidateCacheInvalidFileId === RUN TestInvalidateCacheInvalidFileId/empty_file_ID === RUN TestInvalidateCacheInvalidFileId/no_comma_separator === RUN TestInvalidateCacheInvalidFileId/non-numeric_vid === RUN TestInvalidateCacheInvalidFileId/negative_vid === RUN TestInvalidateCacheInvalidFileId/oversized_vid --- PASS: TestInvalidateCacheInvalidFileId (0.00s) --- PASS: TestInvalidateCacheInvalidFileId/empty_file_ID (0.00s) --- PASS: TestInvalidateCacheInvalidFileId/no_comma_separator (0.00s) --- PASS: TestInvalidateCacheInvalidFileId/non-numeric_vid (0.00s) --- PASS: TestInvalidateCacheInvalidFileId/negative_vid (0.00s) --- PASS: TestInvalidateCacheInvalidFileId/oversized_vid (0.00s) === RUN TestInvalidateCacheWithHistory --- PASS: TestInvalidateCacheWithHistory (0.00s) === RUN TestDeleteVidRecursion --- PASS: TestDeleteVidRecursion (0.00s) === RUN TestGetLocationsEmptyArrayNoFallback --- PASS: TestGetLocationsEmptyArrayNoFallback (0.00s) === RUN TestGetLocationsUnknownVolumeUsesCache --- PASS: TestGetLocationsUnknownVolumeUsesCache (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/wdclient 7.790s ? github.com/seaweedfs/seaweedfs/weed/wdclient/exclusive_locks [no test files] === RUN TestManagerLoopSendersAbortAfterStop I0720 21:30:02.519100 worker.go:148 Generated new worker ID: w-arch-n-53cc I0720 21:30:02.519372 worker.go:155 Saved worker ID to /tmp/TestManagerLoopSendersAbortAfterStop487074390/001/worker.id I0720 21:30:02.519421 worker.go:517 Worker w-arch-n-53cc stopped I0720 21:30:02.519456 worker.go:938 Worker w-arch-n-53cc received task cancellation for task t1 I0720 21:30:02.519464 worker.go:517 Worker w-arch-n-53cc stopped --- PASS: TestManagerLoopSendersAbortAfterStop (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker 0.023s ? github.com/seaweedfs/seaweedfs/weed/worker/tasks [no test files] === RUN TestIntegration_DCFilterWithVolumeState I0720 21:30:02.546853 detection.go:289 BALANCE [hdd]: Created 13 task(s), cluster now balanced. Imbalance=16.0% (threshold=20.0%) detection_integration_test.go:147: DC+state filter: 13 tasks, effective=map[node-a:27 node-b:23] --- PASS: TestIntegration_DCFilterWithVolumeState (0.00s) === RUN TestIntegration_NodeFilterWithCollections I0720 21:30:02.547369 detection.go:289 BALANCE [hdd]: Created 18 task(s), cluster now balanced. Imbalance=13.3% (threshold=20.0%) detection_integration_test.go:206: Node filter with mixed collections: 18 tasks --- PASS: TestIntegration_NodeFilterWithCollections (0.00s) === RUN TestIntegration_ReplicaPlacementWithDCFilter I0720 21:30:02.547566 detection.go:289 BALANCE [hdd]: Created 11 task(s), cluster now balanced. Imbalance=17.1% (threshold=20.0%) detection_integration_test.go:287: Replica placement + DC filter: 11 tasks --- PASS: TestIntegration_ReplicaPlacementWithDCFilter (0.00s) === RUN TestIntegration_RackFilterWithReplicaPlacement I0720 21:30:02.547686 detection.go:289 BALANCE [hdd]: Created 11 task(s), cluster now balanced. Imbalance=17.1% (threshold=20.0%) detection_integration_test.go:361: Rack filter + replica placement: 11 tasks --- PASS: TestIntegration_RackFilterWithReplicaPlacement (0.00s) === RUN TestIntegration_AllFactors I0720 21:30:02.547804 detection.go:289 BALANCE [hdd]: Created 9 task(s), cluster now balanced. Imbalance=13.3% (threshold=20.0%) detection_integration_test.go:469: All factors combined: 9 tasks, effective=map[node-a:16 node-b:14] --- PASS: TestIntegration_AllFactors (0.00s) === RUN TestIntegration_FullVolumesOnlyBalancing I0720 21:30:02.551344 detection.go:289 BALANCE [hdd]: Created 11 task(s), cluster now balanced. Imbalance=17.1% (threshold=20.0%) detection_integration_test.go:512: Full-only balancing: 11 tasks --- PASS: TestIntegration_FullVolumesOnlyBalancing (0.00s) === RUN TestDetection_MixedDiskTypes I0720 21:30:02.551710 detection.go:286 BALANCE [hdd]: No tasks created - cluster well balanced. Imbalance=0.0% (threshold=20.0%). MaxUtil=10.0% on hdd-server-1, MinUtil=10.0% on hdd-server-1 I0720 21:30:02.551722 detection.go:286 BALANCE [ssd]: No tasks created - cluster well balanced. Imbalance=0.0% (threshold=20.0%). MaxUtil=1.0% on ssd-server-1, MinUtil=1.0% on ssd-server-1 --- PASS: TestDetection_MixedDiskTypes (0.00s) === RUN TestDetection_ImbalancedDiskType I0720 21:30:02.552382 detection.go:289 BALANCE [ssd]: Created 40 task(s), cluster now balanced. Imbalance=18.2% (threshold=20.0%) --- PASS: TestDetection_ImbalancedDiskType (0.00s) === RUN TestDetection_SkipsPhysicallyFullDestination I0720 21:30:02.552498 detection.go:347 BALANCE [hdd]: Created 7 task(s), no more beneficial moves available. After move, dest disk-empty util would be 0.8% vs source src util 0.0% --- PASS: TestDetection_SkipsPhysicallyFullDestination (0.00s) === RUN TestDetection_NoDestinationWhenAllDisksFull --- PASS: TestDetection_NoDestinationWhenAllDisksFull (0.00s) === RUN TestDetection_ProjectsVolumeOntoNearFullDestination I0720 21:30:02.552576 detection.go:286 BALANCE [hdd]: No tasks created - cluster well balanced. Imbalance=0.0% (threshold=20.0%). MaxUtil=80.0% on src, MinUtil=80.0% on src --- PASS: TestDetection_ProjectsVolumeOntoNearFullDestination (0.00s) === RUN TestDetection_SkipsRemoteTieredVolumes --- PASS: TestDetection_SkipsRemoteTieredVolumes (0.00s) === RUN TestDetection_RespectsMaxResults I0720 21:30:02.553207 detection.go:289 BALANCE [ssd]: Created 40 task(s), cluster now balanced. Imbalance=18.2% (threshold=20.0%) --- PASS: TestDetection_RespectsMaxResults (0.00s) === RUN TestDetection_ThreeServers_ConvergesToBalance I0720 21:30:02.553625 detection.go:289 BALANCE [hdd]: Created 23 task(s), cluster now balanced. Imbalance=17.6% (threshold=20.0%) --- PASS: TestDetection_ThreeServers_ConvergesToBalance (0.00s) === RUN TestDetection_SkipsPreExistingPendingTasks I0720 21:30:02.553764 detection.go:289 BALANCE [hdd]: Created 5 task(s), cluster now balanced. Imbalance=0.0% (threshold=20.0%) --- PASS: TestDetection_SkipsPreExistingPendingTasks (0.00s) === RUN TestDetection_NoDuplicateVolumesAcrossIterations I0720 21:30:02.554080 detection.go:289 BALANCE [ssd]: Created 18 task(s), cluster now balanced. Imbalance=13.3% (threshold=20.0%) --- PASS: TestDetection_NoDuplicateVolumesAcrossIterations (0.00s) === RUN TestDetection_ThreeServers_MaxServerShifts I0720 21:30:02.554337 detection.go:289 BALANCE [hdd]: Created 16 task(s), cluster now balanced. Imbalance=17.5% (threshold=20.0%) --- PASS: TestDetection_ThreeServers_MaxServerShifts (0.00s) === RUN TestDetection_FourServers_DestinationSpreading I0720 21:30:02.554939 detection.go:289 BALANCE [ssd]: Created 54 task(s), cluster now balanced. Imbalance=12.2% (threshold=20.0%) --- PASS: TestDetection_FourServers_DestinationSpreading (0.00s) === RUN TestDetection_ConvergenceVerification === RUN TestDetection_ConvergenceVerification/2-server-big-gap I0720 21:30:02.555695 detection.go:289 BALANCE [hdd]: Created 40 task(s), cluster now balanced. Imbalance=18.2% (threshold=20.0%) detection_test.go:915: 2-server-big-gap: 40 moves, effective=map[node-0:60 node-1:50], imbalance=18.2% === RUN TestDetection_ConvergenceVerification/3-server-staircase I0720 21:30:02.556019 detection.go:289 BALANCE [hdd]: Created 36 task(s), cluster now balanced. Imbalance=16.0% (threshold=20.0%) detection_test.go:915: 3-server-staircase: 36 moves, effective=map[node-0:54 node-1:50 node-2:46], imbalance=16.0% === RUN TestDetection_ConvergenceVerification/4-server-one-hot I0720 21:30:02.556879 detection.go:289 BALANCE [hdd]: Created 126 task(s), cluster now balanced. Imbalance=17.6% (threshold=20.0%) detection_test.go:915: 4-server-one-hot: 126 moves, effective=map[node-0:74 node-1:62 node-2:62 node-3:62], imbalance=18.5% === RUN TestDetection_ConvergenceVerification/3-server-tight-threshold I0720 21:30:02.557029 detection.go:289 BALANCE [hdd]: Created 10 task(s), cluster now balanced. Imbalance=0.0% (threshold=10.0%) detection_test.go:915: 3-server-tight-threshold: 10 moves, effective=map[node-0:20 node-1:20 node-2:20], imbalance=0.0% --- PASS: TestDetection_ConvergenceVerification (0.00s) --- PASS: TestDetection_ConvergenceVerification/2-server-big-gap (0.00s) --- PASS: TestDetection_ConvergenceVerification/3-server-staircase (0.00s) --- PASS: TestDetection_ConvergenceVerification/4-server-one-hot (0.00s) --- PASS: TestDetection_ConvergenceVerification/3-server-tight-threshold (0.00s) === RUN TestDetection_ExhaustedServerFallsThrough I0720 21:30:02.557539 detection.go:289 BALANCE [hdd]: Created 23 task(s), cluster now balanced. Imbalance=16.0% (threshold=20.0%) detection_test.go:987: Created 23 tasks from node-b after node-a exhausted --- PASS: TestDetection_ExhaustedServerFallsThrough (0.00s) === RUN TestDetection_HeterogeneousCapacity detection_test.go:1052: First balance task: move from server-3 (correct: highest utilization) --- PASS: TestDetection_HeterogeneousCapacity (0.00s) === RUN TestDetection_ZeroVolumeServerIncludedInBalance I0720 21:30:02.559135 detection.go:316 BALANCE [hdd]: Created 5 task(s), cluster as balanced as possible. Imbalance=40.0% (threshold=20.0%), max-min diff=1 detection_test.go:1134: Distribution 8/2/1/0 → map[node-a:3 node-b:3 node-c:3 node-d:2] after 5 moves (imbalance=36.4%) --- PASS: TestDetection_ZeroVolumeServerIncludedInBalance (0.00s) === RUN TestDetection_DataCenterFilter I0720 21:30:02.559299 detection.go:289 BALANCE [hdd]: Created 18 task(s), cluster now balanced. Imbalance=13.3% (threshold=20.0%) detection_test.go:1191: Created 18 tasks within dc1 scope --- PASS: TestDetection_DataCenterFilter (0.00s) === RUN TestDetection_NodeFilter I0720 21:30:02.560246 detection.go:289 BALANCE [hdd]: Created 18 task(s), cluster now balanced. Imbalance=13.3% (threshold=20.0%) detection_test.go:1244: Created 18 tasks within node-a,node-b scope --- PASS: TestDetection_NodeFilter (0.00s) === RUN TestDetection_HeterogeneousMax_NoOvershootNoOscillation I0720 21:30:02.560300 detection.go:344 BALANCE [hdd]: No tasks created - no beneficial move available. After move, dest node-b util would be 60.0% vs source node-a util 50.0% detection_test.go:1353: tasks=0, effective=map[node-a:11 node-b:5], final diff=0.050 (initial=0.050) --- PASS: TestDetection_HeterogeneousMax_NoOvershootNoOscillation (0.00s) === RUN TestDetection_RespectsClusterIdealUtilization I0720 21:30:02.560371 detection.go:289 BALANCE [hdd]: Created 3 task(s), cluster now balanced. Imbalance=11.8% (threshold=20.0%) detection_test.go:1411: tasks=3, effective=map[node-a:18 node-b:9 node-c:4], ideal=0.443 --- PASS: TestDetection_RespectsClusterIdealUtilization (0.00s) === RUN TestResolveBalanceDestination_UsesEffectiveCapacity --- PASS: TestResolveBalanceDestination_UsesEffectiveCapacity (0.00s) === RUN TestDecodeVolumeBalanceTaskParamsFromPayload --- PASS: TestDecodeVolumeBalanceTaskParamsFromPayload (0.00s) === RUN TestDecodeVolumeBalanceTaskParamsFallback --- PASS: TestDecodeVolumeBalanceTaskParamsFallback (0.00s) === RUN TestDeriveBalanceWorkerConfig --- PASS: TestDeriveBalanceWorkerConfig (0.00s) === RUN TestDeriveBalanceWorkerConfigBatchFields --- PASS: TestDeriveBalanceWorkerConfigBatchFields (0.00s) === RUN TestDeriveBalanceWorkerConfigBatchClamping --- PASS: TestDeriveBalanceWorkerConfigBatchClamping (0.00s) === RUN TestBuildBatchVolumeBalanceProposals_SingleBatch --- PASS: TestBuildBatchVolumeBalanceProposals_SingleBatch (0.00s) === RUN TestBuildBatchVolumeBalanceProposals_MultipleBatches --- PASS: TestBuildBatchVolumeBalanceProposals_MultipleBatches (0.00s) === RUN TestBuildBatchVolumeBalanceProposals_BatchSizeOne --- PASS: TestBuildBatchVolumeBalanceProposals_BatchSizeOne (0.00s) === RUN TestVolumeBalanceDescriptorHasBatchFields --- PASS: TestVolumeBalanceDescriptorHasBatchFields (0.00s) === RUN TestBuildVolumeBalanceProposal --- PASS: TestBuildVolumeBalanceProposal (0.00s) === RUN TestVolumeBalanceHandlerRejectsUnsupportedJobType --- PASS: TestVolumeBalanceHandlerRejectsUnsupportedJobType (0.00s) === RUN TestEmitVolumeBalanceDetectionDecisionTraceNoTasks --- PASS: TestEmitVolumeBalanceDetectionDecisionTraceNoTasks (0.00s) === RUN TestVolumeBalanceDescriptorOmitsExecutionTuningFields --- PASS: TestVolumeBalanceDescriptorOmitsExecutionTuningFields (0.00s) === RUN TestBuildMoveTaskParams --- PASS: TestBuildMoveTaskParams (0.00s) === RUN TestBuildMoveTaskParamsDefaultTimeout --- PASS: TestBuildMoveTaskParamsDefaultTimeout (0.00s) === RUN TestExecuteDispatchesBatchPath I0720 21:30:02.561060 plugin_handler.go:857 batch volume balance: 2 moves, max concurrent 2 W0720 21:30:02.570960 plugin_handler.go:783 volume balance: no master addresses in cluster context, skipping pre-move check for volume 2 I0720 21:30:02.573193 task.go:63 Starting balance task - moving volume W0720 21:30:02.571021 plugin_handler.go:783 volume balance: no master addresses in cluster context, skipping pre-move check for volume 1 I0720 21:30:02.573225 task.go:63 Starting balance task - moving volume I0720 21:30:02.573239 task.go:63 Marking volume readonly for move W0720 21:30:02.573292 plugin_handler.go:970 batch balance move 1 failed: volume 2 s2:8080→t2:8080: failed to mark volume readonly: getOrCreateConnection s2:18080: fail to dial s2:18080: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) I0720 21:30:02.573340 task.go:63 Marking volume readonly for move W0720 21:30:02.573400 plugin_handler.go:970 batch balance move 0 failed: volume 1 s1:8080→t1:8080: failed to mark volume readonly: getOrCreateConnection s1:18080: fail to dial s1:18080: grpc: no transport security set (use grpc.WithTransportCredentials(insecure.NewCredentials()) explicitly or set credentials) --- PASS: TestExecuteDispatchesBatchPath (0.01s) === RUN TestExecuteSingleMovePathUnchanged W0720 21:30:02.573717 plugin_handler.go:783 volume balance: no master addresses in cluster context, skipping pre-move check for volume 99 I0720 21:30:02.573727 task.go:63 Starting balance task - moving volume I0720 21:30:02.573734 task.go:63 Marking volume readonly for move --- PASS: TestExecuteSingleMovePathUnchanged (0.00s) === RUN TestFilterMetricsByLocation --- PASS: TestFilterMetricsByLocation (0.00s) === RUN TestFilterMetricsByVolumeState === RUN TestFilterMetricsByVolumeState/ALL_returns_everything === RUN TestFilterMetricsByVolumeState/empty_string_returns_everything === RUN TestFilterMetricsByVolumeState/ACTIVE_keeps_FullnessRatio_below_1.01 === RUN TestFilterMetricsByVolumeState/FULL_keeps_FullnessRatio_at_or_above_1.01 === RUN TestFilterMetricsByVolumeState/unknown_value_returns_everything --- PASS: TestFilterMetricsByVolumeState (0.00s) --- PASS: TestFilterMetricsByVolumeState/ALL_returns_everything (0.00s) --- PASS: TestFilterMetricsByVolumeState/empty_string_returns_everything (0.00s) --- PASS: TestFilterMetricsByVolumeState/ACTIVE_keeps_FullnessRatio_below_1.01 (0.00s) --- PASS: TestFilterMetricsByVolumeState/FULL_keeps_FullnessRatio_at_or_above_1.01 (0.00s) --- PASS: TestFilterMetricsByVolumeState/unknown_value_returns_everything (0.00s) === RUN TestFilterMetricsByVolumeState_NilElement --- PASS: TestFilterMetricsByVolumeState_NilElement (0.00s) === RUN TestFilterMetricsByVolumeState_EmptyInput --- PASS: TestFilterMetricsByVolumeState_EmptyInput (0.00s) === RUN TestVolumeBalanceDescriptorHasVolumeStateField --- PASS: TestVolumeBalanceDescriptorHasVolumeStateField (0.00s) === RUN TestCheckMovePreconditions === RUN TestCheckMovePreconditions/valid_move === RUN TestCheckMovePreconditions/volume_left_the_source === RUN TestCheckMovePreconditions/volume_gone_entirely === RUN TestCheckMovePreconditions/target_already_has_a_replica === RUN TestCheckMovePreconditions/proposal_nodes_carry_grpc_suffix === RUN TestCheckMovePreconditions/grpc-suffixed_target_already_has_a_replica --- PASS: TestCheckMovePreconditions (0.00s) --- PASS: TestCheckMovePreconditions/valid_move (0.00s) --- PASS: TestCheckMovePreconditions/volume_left_the_source (0.00s) --- PASS: TestCheckMovePreconditions/volume_gone_entirely (0.00s) --- PASS: TestCheckMovePreconditions/target_already_has_a_replica (0.00s) --- PASS: TestCheckMovePreconditions/proposal_nodes_carry_grpc_suffix (0.00s) --- PASS: TestCheckMovePreconditions/grpc-suffixed_target_already_has_a_replica (0.00s) === RUN TestIsGoodMove_NoReplication --- PASS: TestIsGoodMove_NoReplication (0.00s) === RUN TestIsGoodMove_MachineAntiAffinity --- PASS: TestIsGoodMove_MachineAntiAffinity (0.00s) === RUN TestIsGoodMove_001_SameRack --- PASS: TestIsGoodMove_001_SameRack (0.00s) === RUN TestIsGoodMove_010_DiffRack --- PASS: TestIsGoodMove_010_DiffRack (0.00s) === RUN TestIsGoodMove_100_DiffDC --- PASS: TestIsGoodMove_100_DiffDC (0.00s) === RUN TestIsGoodMove_SameNode --- PASS: TestIsGoodMove_SameNode (0.00s) === RUN TestIsGoodMove_011_Composite --- PASS: TestIsGoodMove_011_Composite (0.00s) === RUN TestIsGoodMove_110_Composite --- PASS: TestIsGoodMove_110_Composite (0.00s) === RUN TestIsGoodMove_NilReplicaPlacement --- PASS: TestIsGoodMove_NilReplicaPlacement (0.00s) === RUN TestCalculateBalanceScore_ReplicationAware --- PASS: TestCalculateBalanceScore_ReplicationAware (0.00s) === RUN TestPlanBalanceDestination_ChoosesBestValidCompositeDestination --- PASS: TestPlanBalanceDestination_ChoosesBestValidCompositeDestination (0.00s) === RUN TestCreateBalanceTask_FallbackSelectsValidCompositeDestination --- PASS: TestCreateBalanceTask_FallbackSelectsValidCompositeDestination (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker/tasks/balance 0.077s ? github.com/seaweedfs/seaweedfs/weed/worker/tasks/base [no test files] === RUN TestBuildBalancerTopology --- PASS: TestBuildBalancerTopology (0.00s) === RUN TestBuildBalancerTopologyGroupsByHost --- PASS: TestBuildBalancerTopologyGroupsByHost (0.00s) === RUN TestBuildBalancerTopologyCollectionFilter --- PASS: TestBuildBalancerTopologyCollectionFilter (0.00s) === RUN TestDetectionDisabled --- PASS: TestDetectionDisabled (0.00s) === RUN TestDetectionNilTopology --- PASS: TestDetectionNilTopology (0.00s) === RUN TestMovePhasePriority --- PASS: TestMovePhasePriority (0.00s) === RUN TestMultiDiskBalanceNeverLosesShards === RUN TestMultiDiskBalanceNeverLosesShards/balanced_across_disks === RUN TestMultiDiskBalanceNeverLosesShards/concentrated_on_one_node === RUN TestMultiDiskBalanceNeverLosesShards/spread_across_racks --- PASS: TestMultiDiskBalanceNeverLosesShards (0.00s) --- PASS: TestMultiDiskBalanceNeverLosesShards/balanced_across_disks (0.00s) --- PASS: TestMultiDiskBalanceNeverLosesShards/concentrated_on_one_node (0.00s) --- PASS: TestMultiDiskBalanceNeverLosesShards/spread_across_racks (0.00s) === RUN TestConcentratedVolumeSpreadsAcrossNodesAndDisks --- PASS: TestConcentratedVolumeSpreadsAcrossNodesAndDisks (0.00s) === RUN TestBuildBalancerTopologyNormalizesHddDiskType --- PASS: TestBuildBalancerTopologyNormalizesHddDiskType (0.00s) === RUN TestResolveReplicaPlacementFallsBackToMasterDefault === RUN TestResolveReplicaPlacementFallsBackToMasterDefault/explicit_config_used === RUN TestResolveReplicaPlacementFallsBackToMasterDefault/explicit_config_wins_over_default === RUN TestResolveReplicaPlacementFallsBackToMasterDefault/falls_back_to_master_default === RUN TestResolveReplicaPlacementFallsBackToMasterDefault/zero_master_default_=_no_constraint === RUN TestResolveReplicaPlacementFallsBackToMasterDefault/empty_everywhere_=_no_constraint === RUN TestResolveReplicaPlacementFallsBackToMasterDefault/invalid_value_ignored W0720 21:30:02.553738 replica_placement.go:101 ignoring invalid replica placement "nonsense": unexpected replication type: nonsense --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault (0.00s) --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault/explicit_config_used (0.00s) --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault/explicit_config_wins_over_default (0.00s) --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault/falls_back_to_master_default (0.00s) --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault/zero_master_default_=_no_constraint (0.00s) --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault/empty_everywhere_=_no_constraint (0.00s) --- PASS: TestResolveReplicaPlacementFallsBackToMasterDefault/invalid_value_ignored (0.00s) === RUN TestValidateRejectsSameNodeCrossDiskMove --- PASS: TestValidateRejectsSameNodeCrossDiskMove (0.00s) === RUN TestIsDedupPhaseRequiresSameDisk --- PASS: TestIsDedupPhaseRequiresSameDisk (0.00s) === RUN TestDeriveECBalanceWorkerConfig === RUN TestDeriveECBalanceWorkerConfig/nil_values_uses_defaults === RUN TestDeriveECBalanceWorkerConfig/custom_threshold === RUN TestDeriveECBalanceWorkerConfig/threshold_clamped_to_min === RUN TestDeriveECBalanceWorkerConfig/threshold_clamped_to_max === RUN TestDeriveECBalanceWorkerConfig/min_server_count_clamped_to_2 --- PASS: TestDeriveECBalanceWorkerConfig (0.00s) --- PASS: TestDeriveECBalanceWorkerConfig/nil_values_uses_defaults (0.00s) --- PASS: TestDeriveECBalanceWorkerConfig/custom_threshold (0.00s) --- PASS: TestDeriveECBalanceWorkerConfig/threshold_clamped_to_min (0.00s) --- PASS: TestDeriveECBalanceWorkerConfig/threshold_clamped_to_max (0.00s) --- PASS: TestDeriveECBalanceWorkerConfig/min_server_count_clamped_to_2 (0.00s) === RUN TestBuildECBalanceProposal --- PASS: TestBuildECBalanceProposal (0.00s) === RUN TestBuildECBalanceProposalNilResult --- PASS: TestBuildECBalanceProposalNilResult (0.00s) === RUN TestDecodeECBalanceTaskParamsFromProtobuf --- PASS: TestDecodeECBalanceTaskParamsFromProtobuf (0.00s) === RUN TestDecodeECBalanceTaskParamsFallback --- PASS: TestDecodeECBalanceTaskParamsFallback (0.00s) === RUN TestDecodeECBalanceTaskParamsProtobufValidation --- PASS: TestDecodeECBalanceTaskParamsProtobufValidation (0.00s) === RUN TestDecodeECBalanceTaskParamsMissingShardID --- PASS: TestDecodeECBalanceTaskParamsMissingShardID (0.00s) === RUN TestECBalanceHandlerCapability --- PASS: TestECBalanceHandlerCapability (0.00s) === RUN TestECBalanceConfigRoundTrip --- PASS: TestECBalanceConfigRoundTrip (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker/tasks/ec_balance 0.037s === RUN TestPlanECDestinationsPrefersSourceDiskType_FullCluster --- PASS: TestPlanECDestinationsPrefersSourceDiskType_FullCluster (0.00s) === RUN TestPlanECDestinationsSpillsToOtherDiskType_WhenPreferredScarce W0720 21:30:02.591418 detection.go:520 EC volume 2: placed shards outside preferred disk type "ssd" --- PASS: TestPlanECDestinationsSpillsToOtherDiskType_WhenPreferredScarce (0.00s) === RUN TestCountExistingEcShards_DoesNotUnionAcrossGenerations --- PASS: TestCountExistingEcShards_DoesNotUnionAcrossGenerations (0.00s) === RUN TestCountExistingEcShards_SingleCompleteGeneration --- PASS: TestCountExistingEcShards_SingleCompleteGeneration (0.00s) === RUN TestCountExistingEcShards_PreUpgradeZeroGeneration --- PASS: TestCountExistingEcShards_PreUpgradeZeroGeneration (0.00s) === RUN TestPlanECDestinationsUsesPlanner --- PASS: TestPlanECDestinationsUsesPlanner (0.00s) === RUN TestDetectionSkipsWhenECShardsAlreadyExist W0720 21:30:02.592279 detection.go:155 EC Detection: Volume 42 has all 14 EC shards in topology; source replica on 127.0.0.1:8080 is orphaned (#9448). W0720 21:30:02.592285 detection.go:173 EC Detection: no gRPC dial option available to auto-clean orphaned source for volume 42; to clean up by hand, send a targeted VolumeDelete RPC to 127.0.0.1:8080 only — DO NOT use the cluster-wide `volume.delete` shell command, which would also delete the EC shards. --- PASS: TestDetectionSkipsWhenECShardsAlreadyExist (0.00s) === RUN TestDetectionAllowsRegularReplicaWhenShardsPartial I0720 21:30:02.592335 detection.go:216 EC Detection: Volume 43 meets all criteria, attempting to create task I0720 21:30:02.592345 detection.go:242 EC Detection: ActiveTopology available, planning destinations for volume 43 I0720 21:30:02.592405 detection.go:258 EC Detection: Successfully planned 14 destinations for volume 43 I0720 21:30:02.592410 detection.go:275 EC Detection: Looking for replica locations for volume 43 I0720 21:30:02.592415 detection.go:281 EC Detection: Found 1 replica locations for volume 43 I0720 21:30:02.592442 detection.go:400 EC Detection: Successfully created EC task for volume 43, adding to results --- PASS: TestDetectionAllowsRegularReplicaWhenShardsPartial (0.00s) === RUN TestCountExistingEcShardsForVolume --- PASS: TestCountExistingEcShardsForVolume (0.00s) === RUN TestDetectionContextCancellation --- PASS: TestDetectionContextCancellation (0.00s) === RUN TestDetectionMaxResultsHonorsLimit I0720 21:30:02.595766 detection.go:216 EC Detection: Volume 1 meets all criteria, attempting to create task I0720 21:30:02.595783 detection.go:242 EC Detection: ActiveTopology available, planning destinations for volume 1 I0720 21:30:02.595944 detection.go:258 EC Detection: Successfully planned 14 destinations for volume 1 I0720 21:30:02.595952 detection.go:275 EC Detection: Looking for replica locations for volume 1 I0720 21:30:02.595967 detection.go:281 EC Detection: Found 14 replica locations for volume 1 I0720 21:30:02.596051 detection.go:400 EC Detection: Successfully created EC task for volume 1, adding to results --- PASS: TestDetectionMaxResultsHonorsLimit (0.00s) === RUN TestPlanECDestinationsSpreadsAcrossPhysicalDisks --- PASS: TestPlanECDestinationsSpreadsAcrossPhysicalDisks (0.00s) === RUN TestPlanECDestinationsFailsWithInsufficientCapacity --- PASS: TestPlanECDestinationsFailsWithInsufficientCapacity (0.00s) === RUN TestPlanECDestinationsPacksWhenFewerDisksThanShards --- PASS: TestPlanECDestinationsPacksWhenFewerDisksThanShards (0.00s) === RUN TestDeleteOriginalVolumeSurfacesReplicaFailures I0720 21:30:12.437630 task.go:63 Starting original volume deletion from replica servers I0720 21:30:12.437645 task.go:63 Deleting volume from replica server I0720 21:30:12.444943 task.go:63 Successfully deleted volume from replica server I0720 21:30:12.444955 task.go:63 Deleting volume from replica server E0720 21:30:12.445407 task.go:79 Failed to delete volume from replica server E0720 21:30:12.445637 task.go:79 Failed to delete some original volume replicas after EC encoding --- PASS: TestDeleteOriginalVolumeSurfacesReplicaFailures (19.87s) === RUN TestDeleteOriginalVolumeSucceedsWhenAllReplicasReachable I0720 21:30:22.898345 task.go:63 Starting original volume deletion from replica servers I0720 21:30:22.898356 task.go:63 Deleting volume from replica server I0720 21:30:22.900148 task.go:63 Successfully deleted volume from replica server I0720 21:30:22.900158 task.go:63 Successfully deleted volume from all replica servers --- PASS: TestDeleteOriginalVolumeSucceedsWhenAllReplicasReachable (10.47s) === RUN TestReplicasPendingDelete --- PASS: TestReplicasPendingDelete (0.00s) === RUN TestIsExpectedSweepSkip --- PASS: TestIsExpectedSweepSkip (0.00s) === RUN TestSweepEmptyReplicasDeletesStubKeepsData I0720 21:30:33.364301 ec_task.go:877 EC volume 9479: removed empty stub replica on 127.0.0.1:42703.42401 before distribute --- PASS: TestSweepEmptyReplicasDeletesStubKeepsData (10.45s) === RUN TestEcEncodeLeavesRightFilesAndRemovesStubAndSource I0720 21:30:44.240345 task.go:63 Starting erasure coding task I0720 21:30:44.240357 task.go:63 Target server shard assignment I0720 21:30:44.240360 task.go:63 Target server shard assignment I0720 21:30:44.240363 task.go:63 Source server information I0720 21:30:44.240366 task.go:63 Source server information I0720 21:30:44.240383 task.go:63 Marking volume readonly I0720 21:30:44.284481 ec_task.go:372 EC encode replica sync for volume 9490: volume 9490: replicas are inconsistent, building union... volume 9490: selected 127.0.0.1:40925.37145 as best replica (file count: 8) I0720 21:30:44.284495 task.go:63 Copying volume files to worker I0720 21:30:44.284892 task.go:63 Starting volume file copy from source server I0720 21:30:44.285232 task.go:63 Volume index file copied successfully I0720 21:30:44.285787 task.go:63 Volume data file copied successfully I0720 21:30:44.285793 task.go:63 Generating EC shards locally I0720 21:30:44.288073 ec_encoder.go:138 encodeDatFile /tmp/TestEcEncodeLeavesRightFilesAndRemovesStubAndSource967724833/001/vol_9490_1784583044/9490.dat size:33480 with EC context 10+4 (total: 14) I0720 21:30:44.303747 task.go:63 EC shard generated I0720 21:30:44.303764 task.go:63 EC shard generated I0720 21:30:44.303770 task.go:63 EC shard generated I0720 21:30:44.303775 task.go:63 EC shard generated I0720 21:30:44.303780 task.go:63 EC shard generated I0720 21:30:44.303790 task.go:63 EC shard generated I0720 21:30:44.303795 task.go:63 EC shard generated I0720 21:30:44.303800 task.go:63 EC shard generated I0720 21:30:44.303808 task.go:63 EC shard generated I0720 21:30:44.303814 task.go:63 EC shard generated I0720 21:30:44.303819 task.go:63 EC shard generated I0720 21:30:44.303824 task.go:63 EC shard generated I0720 21:30:44.303829 task.go:63 EC shard generated I0720 21:30:44.303833 task.go:63 EC shard generated I0720 21:30:44.304343 task.go:63 EC index file generated I0720 21:30:44.304429 task.go:63 Volume info file generated I0720 21:30:44.304435 task.go:63 EC bitrot checksum sidecar generated I0720 21:30:44.304438 task.go:63 EC shard generation completed I0720 21:30:44.304441 task.go:63 Clearing stale EC shards on destinations I0720 21:30:44.304445 task.go:63 Clearing stale EC shards on destination before re-distribute I0720 21:30:44.310715 task.go:63 Clearing stale EC shards on destination before re-distribute I0720 21:30:44.311819 task.go:63 Removing empty stub replicas before distribute I0720 21:30:44.313239 ec_task.go:877 EC volume 9490: removed empty stub replica on 127.0.0.1:35271.42723 before distribute I0720 21:30:44.313247 task.go:63 Distributing EC shards to destinations I0720 21:30:44.313345 task.go:63 Starting shard distribution to destination server I0720 21:30:44.313359 task.go:63 Starting shard file transfer I0720 21:30:44.320731 task.go:63 Shard file transfer completed I0720 21:30:44.320759 task.go:63 Starting shard file transfer I0720 21:30:44.330704 task.go:63 Shard file transfer completed I0720 21:30:44.330736 task.go:63 Starting shard file transfer I0720 21:30:44.334715 task.go:63 Shard file transfer completed I0720 21:30:44.334745 task.go:63 Starting shard file transfer I0720 21:30:44.342899 task.go:63 Shard file transfer completed I0720 21:30:44.342976 task.go:63 Starting shard file transfer I0720 21:30:44.349163 task.go:63 Shard file transfer completed I0720 21:30:44.349187 task.go:63 Starting shard file transfer I0720 21:30:44.352263 task.go:63 Shard file transfer completed I0720 21:30:44.352283 task.go:63 Starting shard file transfer I0720 21:30:44.355322 task.go:63 Shard file transfer completed I0720 21:30:44.355401 task.go:63 Starting shard file transfer I0720 21:30:44.358497 task.go:63 Shard file transfer completed I0720 21:30:44.358516 task.go:63 Starting shard file transfer I0720 21:30:44.364184 task.go:63 Shard file transfer completed I0720 21:30:44.364213 task.go:63 Starting shard file transfer I0720 21:30:44.366352 task.go:63 Shard file transfer completed I0720 21:30:44.366371 task.go:63 Starting shard file transfer I0720 21:30:44.374075 task.go:63 Shard file transfer completed I0720 21:30:44.374124 task.go:63 Starting shard file transfer I0720 21:30:44.380718 task.go:63 Shard file transfer completed I0720 21:30:44.380809 task.go:63 Starting shard file transfer I0720 21:30:44.384145 task.go:63 Shard file transfer completed I0720 21:30:44.384156 task.go:63 All shards distributed to destination server I0720 21:30:44.384161 task.go:63 Starting shard distribution to destination server I0720 21:30:44.384171 task.go:63 Starting shard file transfer I0720 21:30:44.394014 task.go:63 Shard file transfer completed I0720 21:30:44.394077 task.go:63 Starting shard file transfer I0720 21:30:44.405745 task.go:63 Shard file transfer completed I0720 21:30:44.405777 task.go:63 Starting shard file transfer I0720 21:30:44.412360 task.go:63 Shard file transfer completed I0720 21:30:44.412385 task.go:63 Starting shard file transfer I0720 21:30:44.415756 task.go:63 Shard file transfer completed I0720 21:30:44.415778 task.go:63 Starting shard file transfer I0720 21:30:44.420824 task.go:63 Shard file transfer completed I0720 21:30:44.420915 task.go:63 Starting shard file transfer I0720 21:30:44.421170 task.go:63 Shard file transfer completed I0720 21:30:44.421183 task.go:63 Starting shard file transfer I0720 21:30:44.421402 task.go:63 Shard file transfer completed I0720 21:30:44.421408 task.go:63 All shards distributed to destination server I0720 21:30:44.421413 task.go:63 Mounting EC shards I0720 21:30:44.421435 task.go:63 Starting EC shard mount operation I0720 21:30:44.438140 task.go:63 Successfully mounted EC shards I0720 21:30:44.438164 task.go:63 Starting EC shard mount operation I0720 21:30:44.444013 task.go:63 Successfully mounted EC shards I0720 21:30:44.444031 task.go:63 Verifying EC shards across destinations I0720 21:30:44.448117 task.go:63 EC shard inventory before source deletion I0720 21:30:44.448128 task.go:63 Deleting original volume I0720 21:30:44.448143 task.go:63 Starting original volume deletion from replica servers I0720 21:30:44.448147 task.go:63 Deleting volume from replica server I0720 21:30:44.448935 task.go:63 Successfully deleted volume from replica server I0720 21:30:44.448944 task.go:63 Successfully deleted volume from all replica servers I0720 21:30:44.448956 ec_task.go:252 EC task completed successfully: volume 9490 from 127.0.0.1:40925.37145 with 17 shards distributed --- PASS: TestEcEncodeLeavesRightFilesAndRemovesStubAndSource (21.09s) === RUN TestEcEncodeJulorLayoutConverges I0720 21:31:05.315185 task.go:63 Starting erasure coding task I0720 21:31:05.315196 task.go:63 Target server shard assignment I0720 21:31:05.315199 task.go:63 Target server shard assignment I0720 21:31:05.315201 task.go:63 Target server shard assignment I0720 21:31:05.315204 task.go:63 Target server shard assignment I0720 21:31:05.315206 task.go:63 Target server shard assignment I0720 21:31:05.315209 task.go:63 Target server shard assignment I0720 21:31:05.315212 task.go:63 Source server information I0720 21:31:05.315214 task.go:63 Source server information I0720 21:31:05.315217 task.go:63 Source server information I0720 21:31:05.315233 task.go:63 Marking volume readonly I0720 21:31:05.337255 ec_task.go:372 EC encode replica sync for volume 9500: volume 9500: replicas are inconsistent, building union... volume 9500: selected 127.0.0.1:46261.46477 as best replica (file count: 8) I0720 21:31:05.337272 task.go:63 Copying volume files to worker I0720 21:31:05.337878 task.go:63 Starting volume file copy from source server I0720 21:31:05.338564 task.go:63 Volume index file copied successfully I0720 21:31:05.339461 task.go:63 Volume data file copied successfully I0720 21:31:05.339558 task.go:63 Generating EC shards locally I0720 21:31:05.342268 ec_encoder.go:138 encodeDatFile /tmp/TestEcEncodeJulorLayoutConverges934248148/001/vol_9500_1784583065/9500.dat size:33480 with EC context 10+4 (total: 14) I0720 21:31:05.361054 task.go:63 EC shard generated I0720 21:31:05.361132 task.go:63 EC shard generated I0720 21:31:05.361151 task.go:63 EC shard generated I0720 21:31:05.361169 task.go:63 EC shard generated I0720 21:31:05.361185 task.go:63 EC shard generated I0720 21:31:05.361209 task.go:63 EC shard generated I0720 21:31:05.361231 task.go:63 EC shard generated I0720 21:31:05.361247 task.go:63 EC shard generated I0720 21:31:05.361268 task.go:63 EC shard generated I0720 21:31:05.361284 task.go:63 EC shard generated I0720 21:31:05.361307 task.go:63 EC shard generated I0720 21:31:05.361331 task.go:63 EC shard generated I0720 21:31:05.361337 task.go:63 EC shard generated I0720 21:31:05.361343 task.go:63 EC shard generated I0720 21:31:05.361418 task.go:63 EC index file generated I0720 21:31:05.361499 task.go:63 Volume info file generated I0720 21:31:05.361504 task.go:63 EC bitrot checksum sidecar generated I0720 21:31:05.361508 task.go:63 EC shard generation completed I0720 21:31:05.361512 task.go:63 Clearing stale EC shards on destinations I0720 21:31:05.361517 task.go:63 Clearing stale EC shards on destination before re-distribute I0720 21:31:05.364984 task.go:63 Clearing stale EC shards on destination before re-distribute I0720 21:31:05.371557 task.go:63 Clearing stale EC shards on destination before re-distribute I0720 21:31:05.373515 task.go:63 Removing empty stub replicas before distribute I0720 21:31:05.378622 ec_task.go:877 EC volume 9500: removed empty stub replica on 127.0.0.1:37155.37661 before distribute I0720 21:31:05.378640 task.go:63 Distributing EC shards to destinations I0720 21:31:05.378671 task.go:63 Starting shard distribution to destination server I0720 21:31:05.378695 task.go:63 Starting shard file transfer I0720 21:31:05.382192 task.go:63 Shard file transfer completed I0720 21:31:05.382217 task.go:63 Starting shard file transfer I0720 21:31:05.385464 task.go:63 Shard file transfer completed I0720 21:31:05.385496 task.go:63 Starting shard file transfer I0720 21:31:05.394862 task.go:63 Shard file transfer completed I0720 21:31:05.394900 task.go:63 Starting shard file transfer I0720 21:31:05.395397 task.go:63 Shard file transfer completed I0720 21:31:05.395415 task.go:63 Starting shard file transfer I0720 21:31:05.396383 task.go:63 Shard file transfer completed I0720 21:31:05.396401 task.go:63 Starting shard file transfer I0720 21:31:05.398761 task.go:63 Shard file transfer completed I0720 21:31:05.398779 task.go:63 Starting shard file transfer I0720 21:31:05.401843 task.go:63 Shard file transfer completed I0720 21:31:05.401864 task.go:63 Starting shard file transfer I0720 21:31:05.404529 task.go:63 Shard file transfer completed I0720 21:31:05.404541 task.go:63 All shards distributed to destination server I0720 21:31:05.404547 task.go:63 Starting shard distribution to destination server I0720 21:31:05.404560 task.go:63 Starting shard file transfer I0720 21:31:05.410781 task.go:63 Shard file transfer completed I0720 21:31:05.410855 task.go:63 Starting shard file transfer I0720 21:31:05.413620 task.go:63 Shard file transfer completed I0720 21:31:05.413689 task.go:63 Starting shard file transfer I0720 21:31:05.417290 task.go:63 Shard file transfer completed I0720 21:31:05.417318 task.go:63 Starting shard file transfer I0720 21:31:05.417698 task.go:63 Shard file transfer completed I0720 21:31:05.417711 task.go:63 Starting shard file transfer I0720 21:31:05.417949 task.go:63 Shard file transfer completed I0720 21:31:05.417961 task.go:63 Starting shard file transfer I0720 21:31:05.418150 task.go:63 Shard file transfer completed I0720 21:31:05.418161 task.go:63 Starting shard file transfer I0720 21:31:05.427404 task.go:63 Shard file transfer completed I0720 21:31:05.427438 task.go:63 Starting shard file transfer I0720 21:31:05.430227 task.go:63 Shard file transfer completed I0720 21:31:05.430241 task.go:63 All shards distributed to destination server I0720 21:31:05.430246 task.go:63 Starting shard distribution to destination server I0720 21:31:05.430263 task.go:63 Starting shard file transfer I0720 21:31:05.433240 task.go:63 Shard file transfer completed I0720 21:31:05.433264 task.go:63 Starting shard file transfer I0720 21:31:05.436074 task.go:63 Shard file transfer completed I0720 21:31:05.436097 task.go:63 Starting shard file transfer I0720 21:31:05.436381 task.go:63 Shard file transfer completed I0720 21:31:05.436396 task.go:63 Starting shard file transfer I0720 21:31:05.436654 task.go:63 Shard file transfer completed I0720 21:31:05.436667 task.go:63 Starting shard file transfer I0720 21:31:05.436887 task.go:63 Shard file transfer completed I0720 21:31:05.436899 task.go:63 Starting shard file transfer I0720 21:31:05.439333 task.go:63 Shard file transfer completed I0720 21:31:05.439354 task.go:63 Starting shard file transfer I0720 21:31:05.444033 task.go:63 Shard file transfer completed I0720 21:31:05.444064 task.go:63 All shards distributed to destination server I0720 21:31:05.444070 task.go:63 Mounting EC shards I0720 21:31:05.444083 task.go:63 Starting EC shard mount operation I0720 21:31:05.444934 task.go:63 Successfully mounted EC shards I0720 21:31:05.444950 task.go:63 Starting EC shard mount operation I0720 21:31:05.445696 task.go:63 Successfully mounted EC shards I0720 21:31:05.445706 task.go:63 Starting EC shard mount operation I0720 21:31:05.446396 task.go:63 Successfully mounted EC shards I0720 21:31:05.446402 task.go:63 Verifying EC shards across destinations I0720 21:31:05.447158 task.go:63 EC shard inventory before source deletion I0720 21:31:05.447163 task.go:63 Deleting original volume I0720 21:31:05.447168 task.go:63 Starting original volume deletion from replica servers I0720 21:31:05.447173 task.go:63 Deleting volume from replica server I0720 21:31:05.447669 task.go:63 Successfully deleted volume from replica server I0720 21:31:05.447678 task.go:63 Successfully deleted volume from all replica servers I0720 21:31:05.447689 ec_task.go:252 EC task completed successfully: volume 9500 from 127.0.0.1:46261.46477 with 17 shards distributed --- PASS: TestEcEncodeJulorLayoutConverges (31.01s) === RUN TestCleanupOrphanSkipsWritableSourceReplica W0720 21:31:35.887081 detection.go:747 EC Detection: source replica for volume 955 on 127.0.0.1:34355.43403 is writable; not deleting (may hold writes the EC shards lack) --- PASS: TestCleanupOrphanSkipsWritableSourceReplica (10.41s) === RUN TestCleanupStaleEcShardsBeforeDistribute I0720 21:31:46.313035 task.go:63 Clearing stale EC shards on destination before re-distribute --- PASS: TestCleanupStaleEcShardsBeforeDistribute (10.42s) === RUN TestCleanupStaleEcShardsClearsShardsBeyondSources I0720 21:31:56.735598 task.go:63 Clearing stale EC shards on destination before re-distribute --- PASS: TestCleanupStaleEcShardsClearsShardsBeyondSources (10.43s) === RUN TestCleanupStaleEcShardsCoversTargetsWithoutSources I0720 21:32:07.161150 task.go:63 Clearing stale EC shards on destination before re-distribute --- PASS: TestCleanupStaleEcShardsCoversTargetsWithoutSources (10.42s) === RUN TestCleanupStaleEcShardsSkipsRegularReplicas --- PASS: TestCleanupStaleEcShardsSkipsRegularReplicas (10.41s) === RUN TestCopyVolumeFilesToWorkerUsesCurrentCompactionRevision I0720 21:32:27.986628 task.go:63 Starting volume file copy from source server I0720 21:32:27.987160 task.go:63 Volume index file copied successfully I0720 21:32:27.987827 task.go:63 Volume data file copied successfully --- PASS: TestCopyVolumeFilesToWorkerUsesCurrentCompactionRevision (10.43s) === RUN TestMarkReplicasReadonlyPersists --- PASS: TestMarkReplicasReadonlyPersists (10.43s) === RUN TestGenerateEcShardsLocallyStampsEncodeIdentity I0720 21:32:48.859648 task.go:63 Starting volume file copy from source server I0720 21:32:48.860022 task.go:63 Volume index file copied successfully I0720 21:32:48.860361 task.go:63 Volume data file copied successfully I0720 21:32:48.860702 ec_encoder.go:138 encodeDatFile /tmp/TestGenerateEcShardsLocallyStampsEncodeIdentity271572667/001/952.dat size:120 with EC context 10+4 (total: 14) I0720 21:32:48.875322 task.go:63 EC shard generated I0720 21:32:48.875343 task.go:63 EC shard generated I0720 21:32:48.875349 task.go:63 EC shard generated I0720 21:32:48.875354 task.go:63 EC shard generated I0720 21:32:48.875361 task.go:63 EC shard generated I0720 21:32:48.875367 task.go:63 EC shard generated I0720 21:32:48.875372 task.go:63 EC shard generated I0720 21:32:48.875377 task.go:63 EC shard generated I0720 21:32:48.875386 task.go:63 EC shard generated I0720 21:32:48.875392 task.go:63 EC shard generated I0720 21:32:48.875399 task.go:63 EC shard generated I0720 21:32:48.875409 task.go:63 EC shard generated I0720 21:32:48.875414 task.go:63 EC shard generated I0720 21:32:48.875420 task.go:63 EC shard generated I0720 21:32:48.875484 task.go:63 EC index file generated I0720 21:32:48.875566 task.go:63 Volume info file generated I0720 21:32:48.875572 task.go:63 EC bitrot checksum sidecar generated I0720 21:32:48.875576 task.go:63 EC shard generation completed --- PASS: TestGenerateEcShardsLocallyStampsEncodeIdentity (10.45s) === RUN TestGenerateEcShardsLocallyUsesAdminGeneration I0720 21:32:59.753996 task.go:63 Starting volume file copy from source server I0720 21:32:59.755606 task.go:63 Volume index file copied successfully I0720 21:32:59.755849 task.go:63 Volume data file copied successfully I0720 21:32:59.756723 ec_encoder.go:138 encodeDatFile /tmp/TestGenerateEcShardsLocallyUsesAdminGeneration447409891/001/957.dat size:120 with EC context 10+4 (total: 14) I0720 21:32:59.776323 task.go:63 EC shard generated I0720 21:32:59.776422 task.go:63 EC shard generated I0720 21:32:59.776436 task.go:63 EC shard generated I0720 21:32:59.776443 task.go:63 EC shard generated I0720 21:32:59.776449 task.go:63 EC shard generated I0720 21:32:59.776455 task.go:63 EC shard generated I0720 21:32:59.776469 task.go:63 EC shard generated I0720 21:32:59.776475 task.go:63 EC shard generated I0720 21:32:59.776488 task.go:63 EC shard generated I0720 21:32:59.776494 task.go:63 EC shard generated I0720 21:32:59.776500 task.go:63 EC shard generated I0720 21:32:59.776509 task.go:63 EC shard generated I0720 21:32:59.776514 task.go:63 EC shard generated I0720 21:32:59.776520 task.go:63 EC shard generated I0720 21:32:59.776589 task.go:63 EC index file generated I0720 21:32:59.776679 task.go:63 Volume info file generated I0720 21:32:59.776685 task.go:63 EC bitrot checksum sidecar generated I0720 21:32:59.776691 task.go:63 EC shard generation completed --- PASS: TestGenerateEcShardsLocallyUsesAdminGeneration (10.92s) === RUN TestIsStubReplica --- PASS: TestIsStubReplica (0.00s) === RUN TestSelectCanonicalMetricPrefersCredibleOverLowServerStub --- PASS: TestSelectCanonicalMetricPrefersCredibleOverLowServerStub (0.00s) === RUN TestSelectCanonicalMetricSkipsECMetrics --- PASS: TestSelectCanonicalMetricSkipsECMetrics (0.00s) === RUN TestSelectCanonicalMetricAllStubsFallsBackToLowestServer --- PASS: TestSelectCanonicalMetricAllStubsFallsBackToLowestServer (0.00s) === RUN TestSelectCanonicalMetricTieBreaksByServerAmongCredible --- PASS: TestSelectCanonicalMetricTieBreaksByServerAmongCredible (0.00s) === RUN TestSelectCanonicalMetricEmpty --- PASS: TestSelectCanonicalMetricEmpty (0.00s) === RUN TestDetectionEncodesDespiteLowServerStub I0720 21:33:09.816054 detection.go:216 EC Detection: Volume 13 meets all criteria, attempting to create task I0720 21:33:09.816068 detection.go:242 EC Detection: ActiveTopology available, planning destinations for volume 13 I0720 21:33:09.816174 detection.go:258 EC Detection: Successfully planned 14 destinations for volume 13 I0720 21:33:09.816184 detection.go:275 EC Detection: Looking for replica locations for volume 13 I0720 21:33:09.816194 detection.go:281 EC Detection: Found 2 replica locations for volume 13 I0720 21:33:09.816219 detection.go:400 EC Detection: Successfully created EC task for volume 13, adding to results --- PASS: TestDetectionEncodesDespiteLowServerStub (0.00s) === RUN TestDecodeErasureCodingTaskParamsFromPayload --- PASS: TestDecodeErasureCodingTaskParamsFromPayload (0.00s) === RUN TestDecodeErasureCodingTaskParamsFallback --- PASS: TestDecodeErasureCodingTaskParamsFallback (0.00s) === RUN TestDeriveErasureCodingWorkerConfig --- PASS: TestDeriveErasureCodingWorkerConfig (0.00s) === RUN TestBuildErasureCodingProposal --- PASS: TestBuildErasureCodingProposal (0.00s) === RUN TestErasureCodingHandlerRejectsUnsupportedJobType --- PASS: TestErasureCodingHandlerRejectsUnsupportedJobType (0.00s) === RUN TestEmitErasureCodingDetectionDecisionTraceNoTasks --- PASS: TestEmitErasureCodingDetectionDecisionTraceNoTasks (0.00s) === RUN TestErasureCodingDescriptorOmitsLocalExecutionFields --- PASS: TestErasureCodingDescriptorOmitsLocalExecutionFields (0.00s) === RUN TestApplyErasureCodingExecutionDefaultsForcesLocalFields --- PASS: TestApplyErasureCodingExecutionDefaultsForcesLocalFields (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker/tasks/erasure_coding 187.261s === RUN TestExpireSnapshotsExecution exec_test.go:532: expireSnapshots result: expired 2 snapshot(s), deleted 6 unreferenced file(s) --- PASS: TestExpireSnapshotsExecution (0.06s) === RUN TestExpireSnapshotsNothingToExpire --- PASS: TestExpireSnapshotsNothingToExpire (0.00s) === RUN TestRemoveOrphansExecution --- PASS: TestRemoveOrphansExecution (0.03s) === RUN TestRemoveOrphansPreservesReferencedFiles --- PASS: TestRemoveOrphansPreservesReferencedFiles (0.02s) === RUN TestRewriteManifestsExecution exec_test.go:754: rewriteManifests result: rewrote 5 manifests into 1 (5 entries) --- PASS: TestRewriteManifestsExecution (0.07s) === RUN TestRewriteManifestsBelowThreshold --- PASS: TestRewriteManifestsBelowThreshold (0.01s) === RUN TestFullExecuteFlow exec_test.go:925: Full execution results: expire_snapshots: no snapshots expired; remove_orphans: removed 1 orphan file(s) --- PASS: TestFullExecuteFlow (0.02s) === RUN TestDetectWithFakeFiler --- PASS: TestDetectWithFakeFiler (0.02s) === RUN TestDetectWithFilters --- PASS: TestDetectWithFilters (0.03s) === RUN TestConnectToFilerSkipsUnreachableAddresses --- PASS: TestConnectToFilerSkipsUnreachableAddresses (0.00s) === RUN TestConnectToFilerFailsWhenAllAddressesAreUnreachable --- PASS: TestConnectToFilerFailsWhenAllAddressesAreUnreachable (0.00s) === RUN TestDetectSchedulesCompactionWithoutSnapshotPressure --- PASS: TestDetectSchedulesCompactionWithoutSnapshotPressure (0.03s) === RUN TestDetectSchedulesCompactionWithDeleteManifestPresent --- PASS: TestDetectSchedulesCompactionWithDeleteManifestPresent (0.01s) === RUN TestDetectSchedulesSnapshotExpiryDespiteCompactionEvaluationError --- PASS: TestDetectSchedulesSnapshotExpiryDespiteCompactionEvaluationError (0.02s) === RUN TestDetectSchedulesManifestRewriteWithoutSnapshotPressure --- PASS: TestDetectSchedulesManifestRewriteWithoutSnapshotPressure (0.03s) === RUN TestDetectUsesPlanningIndexForRepeatedCompactionScans --- PASS: TestDetectUsesPlanningIndexForRepeatedCompactionScans (0.02s) === RUN TestDetectInvalidatesPlanningIndexWhenCompactionConfigChanges --- PASS: TestDetectInvalidatesPlanningIndexWhenCompactionConfigChanges (0.02s) === RUN TestDetectPlanningIndexPreservesUnscannedSections --- PASS: TestDetectPlanningIndexPreservesUnscannedSections (0.02s) === RUN TestTableNeedsMaintenanceCachesPlanningIndexBuildError --- PASS: TestTableNeedsMaintenanceCachesPlanningIndexBuildError (0.02s) === RUN TestTableNeedsMaintenanceScopesPlanningIndexBuildErrorsPerOperation --- PASS: TestTableNeedsMaintenanceScopesPlanningIndexBuildErrorsPerOperation (0.02s) === RUN TestPersistPlanningIndexUsesMetadataXattrCASGuard --- PASS: TestPersistPlanningIndexUsesMetadataXattrCASGuard (0.00s) === RUN TestDetectDoesNotScheduleManifestRewriteFromDeleteManifestsOnly --- PASS: TestDetectDoesNotScheduleManifestRewriteFromDeleteManifestsOnly (0.01s) === RUN TestDetectSchedulesOrphanCleanupWithoutSnapshotPressure --- PASS: TestDetectSchedulesOrphanCleanupWithoutSnapshotPressure (0.01s) === RUN TestDetectSchedulesOrphanCleanupWithoutSnapshots --- PASS: TestDetectSchedulesOrphanCleanupWithoutSnapshots (0.00s) === RUN TestStalePlanGuard --- PASS: TestStalePlanGuard (0.00s) === RUN TestMetadataVersionCAS --- PASS: TestMetadataVersionCAS (0.01s) === RUN TestMetadataVersionCASDetectsConcurrentUpdate --- PASS: TestMetadataVersionCASDetectsConcurrentUpdate (0.02s) === RUN TestCompactDataFilesMetrics --- PASS: TestCompactDataFilesMetrics (0.02s) === RUN TestExpireSnapshotsMetrics --- PASS: TestExpireSnapshotsMetrics (0.03s) === RUN TestExecuteCompletionOutputValues --- PASS: TestExecuteCompletionOutputValues (0.03s) === RUN TestCompactDataFilesWithPositionDeletes exec_test.go:2668: result: compacted 2 files into 1 (across 1 bins) --- PASS: TestCompactDataFilesWithPositionDeletes (0.03s) === RUN TestCompactDataFilesWithEqualityDeletes exec_test.go:2768: result: compacted 2 files into 1 (across 1 bins) --- PASS: TestCompactDataFilesWithEqualityDeletes (0.03s) === RUN TestCompactDataFilesApplyDeletesDisabled --- PASS: TestCompactDataFilesApplyDeletesDisabled (0.01s) === RUN TestCompactDataFilesWithMixedDeletes --- PASS: TestCompactDataFilesWithMixedDeletes (0.02s) === RUN TestCompactDataFilesSortStrategyUsesAscendingTableSortOrder --- PASS: TestCompactDataFilesSortStrategyUsesAscendingTableSortOrder (0.03s) === RUN TestCompactDataFilesSortStrategyUsesTableSortOrder --- PASS: TestCompactDataFilesSortStrategyUsesTableSortOrder (0.02s) === RUN TestDetectSkipsSortCompactionBinsAboveCap --- PASS: TestDetectSkipsSortCompactionBinsAboveCap (0.01s) === RUN TestDetectSplitsSortCompactionBinsByCap --- PASS: TestDetectSplitsSortCompactionBinsByCap (0.01s) === RUN TestCompactDataFilesSortStrategyRequiresTableSortOrder --- PASS: TestCompactDataFilesSortStrategyRequiresTableSortOrder (0.01s) === RUN TestRewritePositionDeleteFilesExecution --- PASS: TestRewritePositionDeleteFilesExecution (0.02s) === RUN TestRewritePositionDeleteFilesDetection --- PASS: TestRewritePositionDeleteFilesDetection (0.01s) === RUN TestRewritePositionDeleteFilesSkipsSingleFile --- PASS: TestRewritePositionDeleteFilesSkipsSingleFile (0.02s) === RUN TestRewritePositionDeleteFilesRespectsMinInputFiles --- PASS: TestRewritePositionDeleteFilesRespectsMinInputFiles (0.01s) === RUN TestRewritePositionDeleteFilesPreservesUnsupportedMultiTargetDeletes --- PASS: TestRewritePositionDeleteFilesPreservesUnsupportedMultiTargetDeletes (0.04s) === RUN TestRewritePositionDeleteFilesRebuildsMixedDeleteManifests --- PASS: TestRewritePositionDeleteFilesRebuildsMixedDeleteManifests (0.04s) === RUN TestResolveCompactionRewritePlanFallsBackForUnsupportedSortTransform --- PASS: TestResolveCompactionRewritePlanFallsBackForUnsupportedSortTransform (0.00s) === RUN TestParseConfig --- PASS: TestParseConfig (0.00s) === RUN TestParseOperations --- PASS: TestParseOperations (0.00s) === RUN TestExtractMetadataVersion --- PASS: TestExtractMetadataVersion (0.00s) === RUN TestNeedsMaintenanceNoSnapshots --- PASS: TestNeedsMaintenanceNoSnapshots (0.00s) === RUN TestNeedsMaintenanceExceedsMaxSnapshots --- PASS: TestNeedsMaintenanceExceedsMaxSnapshots (0.00s) === RUN TestNeedsMaintenanceWithinLimits --- PASS: TestNeedsMaintenanceWithinLimits (0.00s) === RUN TestNeedsMaintenanceOldSnapshot --- PASS: TestNeedsMaintenanceOldSnapshot (0.00s) === RUN TestCapabilityAndDescriptor --- PASS: TestCapabilityAndDescriptor (0.00s) === RUN TestBuildMaintenanceProposal --- PASS: TestBuildMaintenanceProposal (0.00s) === RUN TestReadResourceGroupConfig --- PASS: TestReadResourceGroupConfig (0.00s) === RUN TestSelectTablesByResourceGroupRoundRobin --- PASS: TestSelectTablesByResourceGroupRoundRobin (0.00s) === RUN TestSelectTablesByResourceGroupCap --- PASS: TestSelectTablesByResourceGroupCap (0.00s) === RUN TestManifestRewritePathConsistency --- PASS: TestManifestRewritePathConsistency (0.00s) === RUN TestManifestRewriteNestedPathConsistency === RUN TestManifestRewriteNestedPathConsistency/nested_two_levels === RUN TestManifestRewriteNestedPathConsistency/nested_one_level --- PASS: TestManifestRewriteNestedPathConsistency (0.00s) --- PASS: TestManifestRewriteNestedPathConsistency/nested_two_levels (0.00s) --- PASS: TestManifestRewriteNestedPathConsistency/nested_one_level (0.00s) === RUN TestNormalizeIcebergPath === RUN TestNormalizeIcebergPath/relative_metadata_path === RUN TestNormalizeIcebergPath/relative_data_path === RUN TestNormalizeIcebergPath/S3_URL === RUN TestNormalizeIcebergPath/absolute_filer_path === RUN TestNormalizeIcebergPath/nested_data_path === RUN TestNormalizeIcebergPath/S3_URL_nested --- PASS: TestNormalizeIcebergPath (0.00s) --- PASS: TestNormalizeIcebergPath/relative_metadata_path (0.00s) --- PASS: TestNormalizeIcebergPath/relative_data_path (0.00s) --- PASS: TestNormalizeIcebergPath/S3_URL (0.00s) --- PASS: TestNormalizeIcebergPath/absolute_filer_path (0.00s) --- PASS: TestNormalizeIcebergPath/nested_data_path (0.00s) --- PASS: TestNormalizeIcebergPath/S3_URL_nested (0.00s) === RUN TestPartitionKey === RUN TestPartitionKey/empty_partition === RUN TestPartitionKey/nil_partition === RUN TestPartitionKey/single_field === RUN TestPartitionKey/multiple_fields_sorted --- PASS: TestPartitionKey (0.00s) --- PASS: TestPartitionKey/empty_partition (0.00s) --- PASS: TestPartitionKey/nil_partition (0.00s) --- PASS: TestPartitionKey/single_field (0.00s) --- PASS: TestPartitionKey/multiple_fields_sorted (0.00s) === RUN TestBuildCompactionBins --- PASS: TestBuildCompactionBins (0.00s) === RUN TestBuildCompactionBinsFiltersLargeFiles --- PASS: TestBuildCompactionBinsFiltersLargeFiles (0.00s) === RUN TestBuildCompactionBinsMinFilesThreshold --- PASS: TestBuildCompactionBinsMinFilesThreshold (0.00s) === RUN TestBuildCompactionBinsMultiplePartitions --- PASS: TestBuildCompactionBinsMultiplePartitions (0.00s) === RUN TestSplitOversizedBinRespectsTargetSize --- PASS: TestSplitOversizedBinRespectsTargetSize (0.00s) === RUN TestSplitOversizedBinDropsImpossibleRunts --- PASS: TestSplitOversizedBinDropsImpossibleRunts (0.00s) === RUN TestBuildCompactionBinsMultipleSpecs --- PASS: TestBuildCompactionBinsMultipleSpecs (0.00s) === RUN TestBuildCompactionBinsSingleSpec --- PASS: TestBuildCompactionBinsSingleSpec (0.00s) === RUN TestParseConfigApplyDeletes --- PASS: TestParseConfigApplyDeletes (0.00s) === RUN TestNormalizeDetectionConfigUsesSharedDefaults --- PASS: TestNormalizeDetectionConfigUsesSharedDefaults (0.00s) === RUN TestParseConfigRewriteStrategy --- PASS: TestParseConfigRewriteStrategy (0.00s) === RUN TestCollectPositionDeletes --- PASS: TestCollectPositionDeletes (0.00s) === RUN TestCollectEqualityDeletes --- PASS: TestCollectEqualityDeletes (0.00s) === RUN TestMergeParquetFilesWithPositionDeletes --- PASS: TestMergeParquetFilesWithPositionDeletes (0.00s) === RUN TestMergeParquetFilesWithEqualityDeletes --- PASS: TestMergeParquetFilesWithEqualityDeletes (0.00s) === RUN TestDetectNilRequest --- PASS: TestDetectNilRequest (0.00s) === RUN TestExecuteNilRequest --- PASS: TestExecuteNilRequest (0.00s) === RUN TestValidateWhereOperations --- PASS: TestValidateWhereOperations (0.00s) === RUN TestSplitWhereConjunctionQuoteAware --- PASS: TestSplitWhereConjunctionQuoteAware (0.00s) === RUN TestPartitionPredicateMatchesUsesPartitionFieldIDs --- PASS: TestPartitionPredicateMatchesUsesPartitionFieldIDs (0.00s) === RUN TestCompactDataFilesWhereFilter --- PASS: TestCompactDataFilesWhereFilter (0.03s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker/tasks/iceberg 1.169s === RUN TestBuildLimiterFromClusterContext_NilCC --- PASS: TestBuildLimiterFromClusterContext_NilCC (0.00s) === RUN TestBuildLimiterFromClusterContext_NoMetadata --- PASS: TestBuildLimiterFromClusterContext_NoMetadata (0.00s) === RUN TestBuildLimiterFromClusterContext_MissingRateKey --- PASS: TestBuildLimiterFromClusterContext_MissingRateKey (0.00s) === RUN TestBuildLimiterFromClusterContext_NonPositiveRate --- PASS: TestBuildLimiterFromClusterContext_NonPositiveRate (0.00s) === RUN TestBuildLimiterFromClusterContext_PositiveRateBuildsLimiter --- PASS: TestBuildLimiterFromClusterContext_PositiveRateBuildsLimiter (0.00s) === RUN TestBuildLimiterFromClusterContext_BurstMissingDefaultsTo2xRate --- PASS: TestBuildLimiterFromClusterContext_BurstMissingDefaultsTo2xRate (0.00s) === RUN TestBuildLimiterFromClusterContext_TinyRateClampsBurstToOne --- PASS: TestBuildLimiterFromClusterContext_TinyRateClampsBurstToOne (0.00s) === RUN TestParseConfigDefaults --- PASS: TestParseConfigDefaults (0.00s) === RUN TestParseConfigIgnoresWorkerValues --- PASS: TestParseConfigIgnoresWorkerValues (0.00s) === RUN TestParseConfigMetaLogRetentionDefaultsToZero --- PASS: TestParseConfigMetaLogRetentionDefaultsToZero (0.00s) === RUN TestParseConfigMetaLogRetentionDaysConvertsToDuration --- PASS: TestParseConfigMetaLogRetentionDaysConvertsToDuration (0.00s) === RUN TestParseConfigMetaLogRetentionNegativeStaysZero --- PASS: TestParseConfigMetaLogRetentionNegativeStaysZero (0.00s) === RUN TestParseConfigWalkerIntervalDefaultsToZero --- PASS: TestParseConfigWalkerIntervalDefaultsToZero (0.00s) === RUN TestParseConfigWalkerIntervalMinutesConvertsToDuration --- PASS: TestParseConfigWalkerIntervalMinutesConvertsToDuration (0.00s) === RUN TestParseConfigWalkerIntervalNegativeStaysZero --- PASS: TestParseConfigWalkerIntervalNegativeStaysZero (0.00s) === RUN TestClusterS3Endpoints_NilContext --- PASS: TestClusterS3Endpoints_NilContext (0.00s) === RUN TestClusterS3Endpoints_EmptyList --- PASS: TestClusterS3Endpoints_EmptyList (0.00s) === RUN TestClusterS3Endpoints_FiltersEmptyEntries --- PASS: TestClusterS3Endpoints_FiltersEmptyEntries (0.00s) === RUN TestClusterS3Endpoints_AllValid --- PASS: TestClusterS3Endpoints_AllValid (0.00s) === RUN TestReadString_MissingKeyReturnsFallback --- PASS: TestReadString_MissingKeyReturnsFallback (0.00s) === RUN TestReadString_NilValueReturnsFallback --- PASS: TestReadString_NilValueReturnsFallback (0.00s) === RUN TestReadString_WrongKindReturnsFallback --- PASS: TestReadString_WrongKindReturnsFallback (0.00s) === RUN TestReadString_StringValueReturned --- PASS: TestReadString_StringValueReturned (0.00s) === RUN TestCapability_AdvertisesJobType --- PASS: TestCapability_AdvertisesJobType (0.00s) === RUN TestDetect_NilRequestErrors --- PASS: TestDetect_NilRequestErrors (0.00s) === RUN TestDetect_NilSenderErrors --- PASS: TestDetect_NilSenderErrors (0.00s) === RUN TestDetect_WrongJobTypeErrors --- PASS: TestDetect_WrongJobTypeErrors (0.00s) === RUN TestDetect_NoS3EndpointsCompletesWithSkipActivity --- PASS: TestDetect_NoS3EndpointsCompletesWithSkipActivity (0.00s) === RUN TestDetect_NoFilerAddressesCompletesWithSkipActivity --- PASS: TestDetect_NoFilerAddressesCompletesWithSkipActivity (0.00s) === RUN TestDetect_HappyPathProposesOneJobWithFirstFilerAddress --- PASS: TestDetect_HappyPathProposesOneJobWithFirstFilerAddress (0.00s) === RUN TestDetect_EmptyJobTypeAccepted --- PASS: TestDetect_EmptyJobTypeAccepted (0.00s) === RUN TestDetect_PropagatesProposalsSendError --- PASS: TestDetect_PropagatesProposalsSendError (0.00s) === RUN TestDetect_PropagatesCompleteSendError --- PASS: TestDetect_PropagatesCompleteSendError (0.00s) === RUN TestDescriptor_BasicShape --- PASS: TestDescriptor_BasicShape (0.00s) === RUN TestDescriptor_AdminConfigFormHasNoWorkersField --- PASS: TestDescriptor_AdminConfigFormHasNoWorkersField (0.00s) === RUN TestDescriptor_WorkerConfigFormIsAbsent --- PASS: TestDescriptor_WorkerConfigFormIsAbsent (0.00s) === RUN TestDescriptor_AdminRuntimeDefaultsHaveNoTimeoutInPractice --- PASS: TestDescriptor_AdminRuntimeDefaultsHaveNoTimeoutInPractice (0.00s) === RUN TestDescriptor_AdminRuntimeDefaultsDailyCadence --- PASS: TestDescriptor_AdminRuntimeDefaultsDailyCadence (0.00s) === RUN TestDescriptor_AdminRuntimeDefaultsEnabledByDefault --- PASS: TestDescriptor_AdminRuntimeDefaultsEnabledByDefault (0.00s) === RUN TestExecute_NilRequestErrors --- PASS: TestExecute_NilRequestErrors (0.00s) === RUN TestExecute_NilJobErrors --- PASS: TestExecute_NilJobErrors (0.00s) === RUN TestExecute_NilSenderErrors --- PASS: TestExecute_NilSenderErrors (0.00s) === RUN TestExecute_WrongJobTypeErrors --- PASS: TestExecute_WrongJobTypeErrors (0.00s) === RUN TestExecute_NoS3EndpointsErrors --- PASS: TestExecute_NoS3EndpointsErrors (0.00s) === RUN TestExecute_MissingFilerAddressErrors --- PASS: TestExecute_MissingFilerAddressErrors (0.00s) === RUN TestExecute_EmptyJobTypeAccepted --- PASS: TestExecute_EmptyJobTypeAccepted (0.00s) === RUN TestSendSuccessCompletion_EmitsCanonicalCompletion --- PASS: TestSendSuccessCompletion_EmitsCanonicalCompletion (0.00s) === RUN TestSendSuccessCompletion_PropagatesSendError --- PASS: TestSendSuccessCompletion_PropagatesSendError (0.00s) === RUN TestLookupBucketsPath_PropagatesGRPCError --- PASS: TestLookupBucketsPath_PropagatesGRPCError (0.00s) === RUN TestLookupBucketsPath_UsesConfiguredDirBuckets --- PASS: TestLookupBucketsPath_UsesConfiguredDirBuckets (0.00s) === RUN TestLookupBucketsPath_EmptyDirBucketsFallsBackToDefault --- PASS: TestLookupBucketsPath_EmptyDirBucketsFallsBackToDefault (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker/tasks/s3_lifecycle 0.041s ? github.com/seaweedfs/seaweedfs/weed/worker/tasks/util [no test files] === RUN TestDecodeVacuumTaskParamsFromPayload --- PASS: TestDecodeVacuumTaskParamsFromPayload (0.00s) === RUN TestDecodeVacuumTaskParamsFallback --- PASS: TestDecodeVacuumTaskParamsFallback (0.00s) === RUN TestDeriveVacuumConfigAllowsZeroValues --- PASS: TestDeriveVacuumConfigAllowsZeroValues (0.00s) === RUN TestMasterAddressCandidates --- PASS: TestMasterAddressCandidates (0.00s) === RUN TestShouldSkipDetectionByInterval --- PASS: TestShouldSkipDetectionByInterval (0.00s) === RUN TestVacuumHandlerRejectsUnsupportedJobType --- PASS: TestVacuumHandlerRejectsUnsupportedJobType (0.00s) === RUN TestBuildExecutorActivity --- PASS: TestBuildExecutorActivity (0.00s) === RUN TestEmitVacuumDetectionDecisionTraceNoTasks --- PASS: TestEmitVacuumDetectionDecisionTraceNoTasks (0.00s) === RUN TestVacuumDescriptorHasNewFields --- PASS: TestVacuumDescriptorHasNewFields (0.00s) === RUN TestVacuumFiltersVolumeState === RUN TestVacuumFiltersVolumeState/ALL_returns_all === RUN TestVacuumFiltersVolumeState/ACTIVE_returns_writable === RUN TestVacuumFiltersVolumeState/FULL_returns_read-only --- PASS: TestVacuumFiltersVolumeState (0.00s) --- PASS: TestVacuumFiltersVolumeState/ALL_returns_all (0.00s) --- PASS: TestVacuumFiltersVolumeState/ACTIVE_returns_writable (0.00s) --- PASS: TestVacuumFiltersVolumeState/FULL_returns_read-only (0.00s) === RUN TestVacuumFiltersLocation === RUN TestVacuumFiltersLocation/dc_filter === RUN TestVacuumFiltersLocation/rack_filter === RUN TestVacuumFiltersLocation/node_filter === RUN TestVacuumFiltersLocation/dc_+_rack --- PASS: TestVacuumFiltersLocation (0.00s) --- PASS: TestVacuumFiltersLocation/dc_filter (0.00s) --- PASS: TestVacuumFiltersLocation/rack_filter (0.00s) --- PASS: TestVacuumFiltersLocation/node_filter (0.00s) --- PASS: TestVacuumFiltersLocation/dc_+_rack (0.00s) PASS ok github.com/seaweedfs/seaweedfs/weed/worker/tasks/vacuum 0.036s ? github.com/seaweedfs/seaweedfs/weed/worker/types [no test files] ? github.com/seaweedfs/seaweedfs/weed/worker/types/base [no test files] FAIL ==> ERROR: A failure occurred in check(). Aborting... ==> ERROR: Build failed, check /home/alhp/workspace/chroot/build_32987531-4dcf-4cc4-8040-fb36af4c4755/build