4ea6f3b06b
fix: invalid checksum on site replication with conforming checksum types ( #21535 )
2025-08-22 07:15:21 -07:00
2718d9a430
CopyObject must preserve checksums and encrypt them if required ( #21399 )
2025-06-25 08:08:54 -07:00
2b34e5b9ae
move to go1.24 ( #21114 )
2025-04-09 07:28:39 -07:00
aeabac9181
Test checksum types for invalid combinations ( #20953 )
2025-02-18 08:24:01 -08:00
b8dde47d4e
fix: multipart replication with single part objects ( #20895 )
...
x-amz-checksum-algorithm is not set, causing all multipart single-part objects
to fail to replicate going via sftp/FTP uploads.
2025-02-05 15:06:02 -08:00
827004cd6d
Add Full Object Checksums and CRC64-NVME ( #20855 )
...
Backport of AIStor PR 247.
Add support for full object checksums as described here:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html
New checksum types are fully supported. Mint tests from https://github.com/minio/minio-go/pull/2026 are now passing.
Includes fixes from https://github.com/minio/minio/pull/20743 for mint tests.
Add using checksums as validation for object content. Fixes #20845 #20849
Fixes checksum replication (downstream PR 250)
2025-01-20 06:49:07 -08:00
5bb31e4883
Disable mint full object tests ( #20743 )
...
Remove expected failures from https://github.com/minio/minio-go/pull/2026
2024-12-09 18:59:22 -08:00
05a6c170bf
Fix PutObject Trailing checksum ( #20456 )
...
PutObject would verify trailing checksums, but not store them.
Fixes #20455
2024-09-19 05:59:07 -07:00
ec49fff583
Accept multipart checksums with part count ( #19680 )
...
Accept multipart uploads where the combined checksum provides the expected part count.
It seems this was added by AWS to make the API more consistent, even if the
data is entirely superfluous on multiple levels.
Improves AWS S3 compatibility.
2024-05-08 09:18:34 -07:00
f4f1c42cba
deprecate usage of sha256-simd ( #19621 )
...
go1.21 already implements the necessary optimizations
2024-04-25 23:31:35 -07:00
95bf4a57b6
logging: Add subsystem to log API ( #19002 )
...
Create new code paths for multiple subsystems in the code. This will
make maintaing this easier later.
Also introduce bugLogIf() for errors that should not happen in the first
place.
2024-04-04 05:04:40 -07:00
dd2542e96c
add codespell action ( #18818 )
...
Original work here, #18474 , refixed and updated.
2024-01-17 23:03:17 -08:00
9b8ba97f9f
feat: add support for GetObjectAttributes API ( #18732 )
2024-01-05 10:43:06 -08:00
9947c01c8e
feat: SSE-KMS use uuid instead of read all data to md5. ( #17958 )
2023-09-18 10:00:54 -07:00
0bc34952eb
fix: under FanOut API avoid repeated md5sum calculation ( #17572 )
...
md5sum calculation has a high CPU overhead, avoid calculating
it repeatedly for similar fanOut calls.
To fix following CPU profiler result
```
(pprof) top10
Showing nodes accounting for 678.68s, 84.67% of 801.54s total
Dropped 1072 nodes (cum <= 4.01s)
Showing top 10 nodes out of 156
flat flat% sum% cum cum%
332.54s 41.49% 41.49% 332.54s 41.49% runtime/internal/syscall.Syscall6
228.39s 28.49% 69.98% 228.39s 28.49% crypto/md5.block
48.07s 6.00% 75.98% 48.07s 6.00% runtime.memmove
28.91s 3.61% 79.59% 28.91s 3.61% github.com/minio/highwayhash.updateAVX2
8.25s 1.03% 80.61% 8.25s 1.03% runtime.futex
8.25s 1.03% 81.64% 10.81s 1.35% runtime.step
6.99s 0.87% 82.52% 22.35s 2.79% runtime.pcvalue
6.67s 0.83% 83.35% 38.90s 4.85% runtime.mallocgc
5.77s 0.72% 84.07% 32.61s 4.07% runtime.gentraceback
4.84s 0.6% 84.67% 10.49s 1.31% runtime.lock2
```
2023-07-05 03:16:05 -07:00
d0a0eb9738
support fan-out objects via PostUpload() ( #17233 )
2023-05-24 22:51:07 -07:00
5677f73794
Add PostObject Checksum ( #17244 )
2023-05-23 07:58:33 -07:00
aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage ( #17218 )
2023-05-16 13:14:37 -07:00
ef2fc0f99e
fix: reduce using memory and temporary files. ( #17206 )
2023-05-15 14:08:54 -07:00
e07c2ab868
Use hash.NewLimitReader for internal multipart calls ( #17191 )
2023-05-12 11:19:08 -07:00
76913a9fd5
Signed trailers for signature v4 ( #16484 )
2023-05-05 19:53:12 -07:00
7fad0c8b41
Remove checksums from HTTP range request, add part checksums ( #17105 )
2023-04-28 08:26:32 -07:00
c133979b8e
Add part count to checksum ( #17035 )
2023-04-14 09:44:45 -07:00
c73ea27ed7
do not log checksum mismatch error, client received the error ( #16246 )
2022-12-14 01:57:40 -08:00
1b0dfb0f58
remove printing map() checksums ( #16028 )
2022-11-08 13:29:24 -08:00
328d660106
support CRC32 Checksums on single drive setup ( #15873 )
2022-10-15 11:58:47 -07:00
ff12080ff5
Remove deprecated io/ioutil ( #15707 )
2022-09-19 11:05:16 -07:00
8e4a45ec41
fix: encrypt checksums in metadata ( #15620 )
2022-08-31 08:13:23 -07:00
a9f1ad7924
Add extended checksum support ( #15433 )
2022-08-29 16:57:16 -07:00
be6ccd129d
fix: typo in FIPS sha256 ( #15024 )
2022-06-01 19:11:04 -07:00
9d07cde385
use crypto/sha256 only for FIPS 140-2 compliance ( #14983 )
...
It would seem like the PR #11623 had chewed more
than it wanted to, non-fips build shouldn't really
be forced to use slower crypto/sha256 even for
presumed "non-performance" codepaths. In MinIO
there are really no "non-performance" codepaths.
This assumption seems to have had an adverse
effect in certain areas of CPU usage.
This PR ensures that we stick to sha256-simd
on all non-FIPS builds, our most common build
to ensure we get the best out of the CPU at
any given point in time.
2022-05-27 06:00:19 -07:00
202d0b64eb
fix: enable go1.17 github ci/cd ( #12997 )
2021-08-18 18:35:22 -07:00
1f262daf6f
rename all remaining packages to internal/ ( #12418 )
...
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg `
2021-06-01 14:59:40 -07:00
Mark Theunissen