Commit Graph

168 Commits

Author SHA1 Message Date
Minio Trusted bfb92a27b7 Update yaml files to latest version RELEASE.2020-12-23T02-24-12Z 2020-12-23 02:43:25 +00:00
Harshavardhana 5c451d1690 update x/net/http2 to address few bugs (#11144)
additionally also configure http2 healthcheck
values to quickly detect unstable connections
and let them timeout.

also use single transport for proxying requests
2020-12-21 21:42:38 -08:00
Anis Elleuch 5434088c51 replication: Ensure to always use nano precision source modtime (#11135) 2020-12-18 11:37:28 -08:00
Harshavardhana 5df61ab96b fix: remove gorilla/rpc/ deps fully after our fork (#11108) 2020-12-15 11:18:06 -08:00
Harshavardhana 705e196b6c upgrade event notification dependencies (#11095) 2020-12-12 20:31:28 -08:00
Harshavardhana 4939987eb8 update deps to latest for some vulnerable deps (#11080)
- github.com/miekg/dns
- update elasticsearch library deps
  to circumvent some aws-sdk-go deps
2020-12-10 13:23:06 -08:00
Harshavardhana 12391ec4ba update nats.io/jwt dependency (#11066)
primarily for a vulnerability report https://nvd.nist.gov/vuln/detail/CVE-2020-26521
2020-12-09 14:30:35 -08:00
Harshavardhana e083471ec4 use argon2 with sync.Pool for better memory management (#11019) 2020-12-03 19:23:19 -08:00
Harshavardhana 80d31113e5 fix: etcd import paths again depend on v3.4.14 release (#11020)
Due to botched upstream renames of project repositories
and incomplete migration to go.mod support, our current
dependency version of `go.mod` had bugs i.e it was
using commits from master branch which didn't have
the required fixes present in release-3.4 branches

which leads to some rare bugs

https://github.com/etcd-io/etcd/pull/11477 provides
a workaround for now and we should migrate to this.

release-3.5 eventually claims to fix all of this
properly until then we cannot use /v3 import right now
2020-12-03 11:35:18 -08:00
Harshavardhana a6113b2315 update minio-go version v7.0.6 (#11012) 2020-12-01 19:15:38 -08:00
Klaus Post f139a19238 Upgrade compress and pgzip package (#10992)
Should provide faster decompression for s3 select and other places where it is used.
2020-11-27 10:10:15 -08:00
Harshavardhana f96ed3769f run go mod tidy 2020-11-24 12:04:15 -08:00
Klaus Post a75fafdbe2 Remove msgp workaround (#10964)
The error in `github.com/philhofer/fwd` was quickly fixed through 
https://github.com/philhofer/fwd/pull/22 - update the dependency 
and remove the workaround.
2020-11-24 11:58:10 -08:00
Shireesh Anjal 14a7ae8586 Remove platform specific structure definitions (#10935)
Instead of having less/more fields inside a structure depending on the
platform (non-linux/linux), it would be better to have the same standard
definition in all platforms, and certain fields of the structure to be
populated or left unpopulated depending on the platform.
2020-11-21 09:41:33 -08:00
Poorna Krishnamoorthy 251c1ef6da Add support for replication of object tags, retention metadata (#10880) 2020-11-19 18:56:09 -08:00
Harshavardhana 9a34fd5c4a Revert "Revert "Add delete marker replication support (#10396)""
This reverts commit 267d7bf0a9.
2020-11-19 18:43:58 -08:00
Harshavardhana 7ff8128f15 update jstream to latest release v1.0.1 (#10909) 2020-11-17 09:13:28 -08:00
Harshavardhana 267d7bf0a9 Revert "Add delete marker replication support (#10396)"
This reverts commit 50c10a5087.

PR is moved to origin/dev branch
2020-11-12 11:43:14 -08:00
Poorna Krishnamoorthy 50c10a5087 Add delete marker replication support (#10396)
Delete marker replication is implemented for V2
configuration specified in AWS spec (though AWS
allows it only in the V1 configuration).

This PR also brings in a MinIO only extension of
replicating permanent deletes, i.e. deletes specifying
version id are replicated to target cluster.
2020-11-10 15:24:14 -08:00
Klaus Post 311ab43d4c Upgrade msgp to official release (#10858)
Code identical, but official release.
2020-11-09 07:52:24 -08:00
Klaus Post 47d715f642 Update msgp to main branch (#10835)
The main branch has error unwrapping to clean up logs on canceled contexts, etc.

Asked for a new release: https://github.com/tinylib/msgp/issues/282
2020-11-05 08:13:03 -08:00
Venkata K Sadineni d8e07f2c41 go mod update golang.org/x/text to address a CVE (#10805) 2020-11-01 19:42:48 -08:00
Klaus Post 6b14c4ab1e Optimize decryptObjectInfo (#10726)
`decryptObjectInfo` is a significant bottleneck when listing objects.

Reduce the allocations for a significant speedup.

https://github.com/minio/sio/pull/40

```
λ benchcmp before.txt after.txt
benchmark                          old ns/op     new ns/op     delta
Benchmark_decryptObjectInfo-32     24260928      808656        -96.67%

benchmark                          old MB/s     new MB/s     speedup
Benchmark_decryptObjectInfo-32     0.04         1.24         31.00x

benchmark                          old allocs     new allocs     delta
Benchmark_decryptObjectInfo-32     75112          48996          -34.77%

benchmark                          old bytes     new bytes     delta
Benchmark_decryptObjectInfo-32     287694772     4228076       -98.53%
```
2020-10-29 09:34:20 -07:00
Harshavardhana 23773bb32b update NTP package for accurate time resolution fixes (#10670) 2020-10-14 12:29:20 +05:30
Poorna Krishnamoorthy dbbed6f7f0 update minio-go dependency (#10634) 2020-10-06 08:37:09 -07:00
Shireesh Anjal f1418a50f0 add NVMe drive info [model num, serial num, drive temp. etc.] (#10613)
* add NVMe drive info [model num, serial num, drive temp. etc.]
* Ignore fuse partitions
* Add the nvme logic only for linux
* Move smart/nvme structs to a separate file

Co-authored-by: wlan0 <sidharthamn@gmail.com>
2020-10-04 10:18:46 -07:00
Harshavardhana c6a9a94f94 fix: optimize ServerInfo() handler to avoid reading config (#10626)
fixes #10620
2020-10-02 16:19:44 -07:00
Anis Elleuch d1ed1da8c6 build: check-gen should exit with some changes in generated code (#10590) 2020-10-02 11:10:39 -07:00
Harshavardhana 66174692a2 add '.healing.bin' for tracking currently healing disk (#10573)
add a hint on the disk to allow for tracking fresh disk
being healed, to allow for restartable heals, and also
use this as a way to track and remove disks.

There are more pending changes where we should move
all the disk formatting logic to backend drives, this
PR doesn't deal with this refactor instead makes it
easier to track healing in the future.
2020-09-28 19:39:32 -07:00
Anis Elleuch e0c04a2da0 Makefile: Check for any non committed auto-generated code (#10579)
Always check if the auto-generated code is still compatible with the
existing written code to avoid a possible forgetting or sometimes a non
intentional change.
2020-09-28 13:33:34 -07:00
Praveen raj Mani b880796aef Set the maximum open connections limit in PG and MySQL target configs (#10558)
As the bulk/recursive delete will require multiple connections to open at an instance,
The default open connections limit will be reached which results in the following error

```FATAL:  sorry, too many clients already```

By setting the open connections to a reasonable value - `2`, We ensure that the max open connections
will not be exhausted and lie under bounds.

The queries are simple inserts/updates/deletes which is operational and sufficient with the
the maximum open connection limit is 2.

Fixes #10553

Allow user configuration for MaxOpenConnections
2020-09-24 22:20:30 -07:00
Harshavardhana 02c1a08a5b fix: make sure to lock CopyObject for in-place updates (#10492) 2020-09-15 20:44:48 -07:00
Gaige B Paulsen 859ef52886 update for smartos build (solaris too) (#10378) 2020-08-31 10:19:25 -07:00
Harshavardhana 7e80afdd7f remove nancy ignore list, as upstream issue is fixed 2020-08-26 15:45:42 -07:00
Harshavardhana caad314faa add ruleguard support, fix all the reported issues (#10335) 2020-08-24 12:11:20 -07:00
Praveen raj Mani d0c910a6f3 Support https and basic-auth for elasticsearch notification target (#10332) 2020-08-23 09:43:48 -07:00
Harshavardhana 81c90ae430 move to gorilla mux 1.8.0 (#10328) 2020-08-22 23:14:02 -07:00
Harshavardhana 83a82d818e allow lock tolerance to match storage-class drive tolerance (#10270) 2020-08-14 18:17:14 -07:00
poornas 79e21601b0 fix: web handlers to enforce replication (#10249)
This PR also preserves source ETag for replication
2020-08-12 17:32:24 -07:00
Harshavardhana 1e2ebc9945 feat: time to bring back http2.0 support (#10230)
Bonus move our CI/CD to go1.14
2020-08-10 09:02:29 -07:00
Harshavardhana cd04600862 update minio-go version to latest v7.0.3 (#10201)
- Add methods to set/remove replication rules (poornas)
- fix: only SSE-C headers should be applied to destination (Harshavardhana)
- fix: avoid data race by copying the buffer (Harshavardhana)
- remove deprecated build badges (Harshavardhana)
- fix: handle readFull bug with certain readers (Harshavardhana)
- fix a typo in README.md (Julien K)
- lifecycle: Fix marshaling expiration date/days (Anis Elleuch)
- add replication-status, expiration headers (Harshavardhana)
- Return object's version id in StatObject(Anis Elleuch)
- display appropriate funcName with nested callers (Harshavardhana)
- allow KMS tests to be run in the CI/CD (Harshavardhana)
- fix: removing lifecycle properly (Harshavardhana)
- feat: Add ListenNotification API to listen for all events (Harshavardhana)
2020-08-05 08:14:36 -07:00
Harshavardhana db6bba709a update CREDITS with minio/selfupdate project 2020-07-31 23:58:30 -07:00
poornas b46ab7e921 Rename replication target handler (#10142)
Rename replication target handler to a generic bucket target handler
2020-07-28 11:50:47 -07:00
Harshavardhana 3a73f1ead5 refactor server update behavior (#10107) 2020-07-23 08:03:31 -07:00
poornas c43da3005a Add support for server side bucket replication (#9882) 2020-07-21 17:49:56 -07:00
Harshavardhana 4bfc50411c fix: return versionId in tagging APIs (#10068) 2020-07-16 22:38:58 -07:00
Anis Elleuch 778e9c864f Move dependency from minio-go v6 to v7 (#10042) 2020-07-14 09:38:05 -07:00
Harshavardhana 143f9371c6 fix: loading users regression
additionally also move to latest gorilla/mux master
to fix the DNS style bucket routing regression

resolves #10022
resolves #10023
2020-07-11 14:03:27 -07:00
Harshavardhana 72e0745e2f fix: migrate to go.etcd.io import path (#9987)
with the merge of https://github.com/etcd-io/etcd/pull/11823
etcd v3.5.0 will now have a properly imported versioned path

this fixes our pending migration to newer repo
2020-07-07 19:04:29 -07:00
Harshavardhana c087a05b43 fix: simplify data structure before release (#9968)
- additionally upgrade to msgp@v1.1.2
- change StatModTime,StatSize fields as
  simple Size/ModTime
- reduce 50000 entries per List batch to 10000
  as client needs to wait too long to see the
  first batch some times which is not desired
  and it is worth we write the data as soon
  as we have it.
2020-07-04 12:25:53 -07:00